Skip to content

Latest commit

 

History

History
181 lines (116 loc) · 3.86 KB

README.md

File metadata and controls

181 lines (116 loc) · 3.86 KB

Running with Spring Cloud Data Flow

Starting Spring Cloud Data Flow

cd .dataflow
. versions
docker-compose up -d

Running shell

docker-compose exec dataflow-server java -jar shell.jar

Services

Sample use case

Below we will build, deploy and create simple stream that writes HTTP requests to the file.

  1. Build

    mvn clean package -f ../pom.xml

    NOTE: target directory is mapped under /target on dataflow-server service in docker-compose.yml.

  2. Register HTTP source

    • using shell

      docker-compose exec dataflow-server java -jar shell.jar
      app register --name http-pro --type source --uri "file:///target/http-source-0.0.1-SNAPSHOT.jar"

    • using REST API

      curl 'http://localhost:9393/apps/source/http-pro' -i -X POST -d 'uri=file%3A%2F%2F%2Ftarget%2Fhttp-source-0.0.1-SNAPSHOT.jar'

  3. Define stream

    We will define stream from HTTP Source to the file: http-to-file

    • using shell

      stream create --definition "http-pro --port=8090 | file --directory=/tmp --name=http-requests" --name http-to-file

    • using REST API

      curl -X POST -d "name=http-to-file&definition=http-pro --port=8090 | file --directory=/tmp --name=http-requests" localhost:9393/streams/definitions?deploy=false

  4. Validate stream

    • using shell

      stream validate --name http-to-file

    • using REST API

      curl 'http://localhost:9393/streams/validation/http-to-file' -i -X GET

  5. Deploy stream

    • using shell

      stream deploy --name http-to-file

    • using REST API

      curl 'http://localhost:9393/streams/deployments/http-to-file' -i -X POST

  6. Get stream details

    • using shell

      stream info --name http-to-file

    • using REST API

      curl 'http://localhost:9393/streams/deployments/http-to-file' -i

  7. Test stream

    • generate request

      docker-compose exec skipper-server curl -H 'Content-Type: application/json' -d '{"name":"john"}' http://localhost:8090

    • monitor file

      docker-compose exec skipper-server tail -f /tmp/http-requests

    • consume kafka topic

      docker-compose exec kafka kafka-topics --bootstrap-server=kafka:9092 --list
docker-compose exec kafka kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic http-to-file.http-pro

  8. Undeploy stream

    • using shell

      stream undeploy --name http-to-file

    • using REST API

      curl 'http://localhost:9393/streams/deployments/http-to-file' -i -X DELETE

  9. Destroy stream

    • using shell

      stream destroy --name http-to-file

    • using REST API

      curl 'http://localhost:9393/streams/definitions/http-to-file' -i -X DELETE

  10. Unregister Http source

    • using shell

      app unregister --type source --name http-pro

    • using REST API

      curl 'http://localhost:9393/apps/source/http-pro' -i -X DELETE

Troubleshooting

Browse skipper logs for executed command:

docker-compose logs skipper-server | grep 'Command to be executed:'

Execute command manualy:

docker-compose exec skipper-server <command>