quote-pipeline

Usage

• Install dependancies

pip install -r requirements.txt

• Place FastText model in a file called model.bin under project root (or set your filepath with config.core.fasttext_path)

• Run

python main.py

For the configuration options, see config.py.

Docker

docker build -t quote-pipeline .
docker run -d \
    -e KAFKA_BOOTSTRAP_SERVERS=<kafka-ip:port> \
    -e ES_HOST=<elasticsearch-ip:port> \
    quote-pipeline

Docker image is also available at lsirepfl/quote-pipeline.

Elasticsearch

An explicit mapping for the output index is provided in mapping.sh.

ES_HOST="elasticsearch-ip:9200" INDEX="index-name" bash mapping.sh