Dockerized setup of SPEEDD environment
This project configures the SPEEDD multi-node dockerized environment. It's comprised of the following components:
- Zookeeper
- Storm cluster:
- Storm nimbus
- Storm supervisor
- Storm UI
- Kafka broker
- SPEEDD UI
- SPEEDD client
- Install docker toolkit:
Note: Because of an issue with default docker machine you have to create a new machine with overlay storage driver using the following command:
docker-machine create -d virtualbox --engine-storage-driver overlay overlay
After creating the overlay machine please update the start.sh file under your Docker Toolbox installation folder to have the following line:
Instead of: VM=default
Should be: VM=overlay
- Start docker-machine and open the terminal window (e.g. Docker Quickstart Terminal)
- Build prerequisite images (this step is required because we use our own forked version of the storm and kafka images)
3.1 Build storm images
- git clone https://github.com/speedd-project/storm-docker.git
- cd storm-docker
- ./rebuild.sh 3.2 Build kafka image
- git clone https://github.com/speedd-project/kafka-docker.git
- cd kafka-docker
- docker build -t wurstmeister/kafka-docker --rm=true .
- Create the 'projects' folder in your home directory and check out the speedd project's source code into it. Then build the speedd-runtime project (
mvn clean install -DskipTests assembly:assembly
- run it from the speedd-runtime folder)
Start a SPEEDD cluster for traffic management use case:
docker-compose -f docker-compose.yml -f docker-compose-tm.yml up -d
Note: The client and UI containers will wait for 2 minutes to have the kafka brokers up and topics initialized. Also, it might take a few minutes to storm cluster to fully initialize. Please take this into account before accessing the UI or starting SPEEDD topology.
Start a SPEEDD cluster for credit card fraud management use case:
docker-compose -f docker-compose.yml -f docker-compose-ccf.yml up -d
Destroy the SPEEDD cluster:
docker-compose -f docker-compose.yml -f docker-compose-tm.yml stop
Find docker machine ip
docker-machine ls
to see the running machine name (should beoverlay
)docker-machine ip overlay
Open SSH to the client:
ssh root@<docker-machine-ip> -p 49022
Note: The password is initialized to 'speedd'
Deploy SPEEDD topology (example for the traffic management use case):
- Open ssh to the client container
cd /opt/speedd/speedd-runtime/scripts/traffic
./start-speedd-runtime-docker
Stream events into SPEEDD (example for the traffic management use case):
- Open ssh to the client container
cd /opt/speedd/speedd-runtime/scripts/traffic
playevents-traffic-docker
Open SPEEDD UI:
- open http://<docker-machine-ip>:43000 in your browser
Open Storm UI:
- open http://<docker-machine-ip>:49080 in your browser