Skip to content

How to use dotnet core to send and consume data from Kafka topics

Notifications You must be signed in to change notification settings

Azadehkhojandi/Kafka-dotnet-HelloWorld

Repository files navigation

Option 1:

  • Setp 1: Install kafka on your local machine directly

Run ./0-download-kafka.sh , ./1-start-kafka-zookeeper.sh and ./2-start-kafka-server.sh and ./3-create-topic.sh

  • Setp 2: monitor and test
  1. Run ./kafka-status.sh, it should return back list of topics and brokers
  2. Run ./4-start-producer.sh my-topic in one terminal and leave it open and Run ./5-start-consumer.sh my-topic in another terminal. If you type a message in producer and hit enter the message will appears in consumer terminal.

Option 2:

  • Setp 1: Install kafka on your local machine via docker Run docker-compose up -d

alt text

  • Setp 2: monitor and test
  1. Run Producer app in one terminal and leave it open cd Producer dotnet restore dotnet build cd producer dotnet run

  2. Run consumer in another terminal and leave it open cd Consumer dotnet restore dotnet build cd Consumer dotnet run

alt text

Hints

If you run the dotnet code on windows and run your docker file on Linux sub system (wsl) you might get the following issue:

0.0.0.0:9092/bootstrap: Connect to ipv4#0.0.0.0:9092 failed: The requested address is not valid in its context... (after 1000ms in state CONNECT) 1/1 brokers are down

The solution for it is running the consumer and producer app on Linux sub system (wsl) too.

Install .Net Core 2.2 on UBuntu

curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg

sudo mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg

sudo sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-bionic-prod bionic main" > /etc/apt/sources.list.d/dotnetdev.list'

sudo apt-get install apt-transport-https

sudo apt-get update

sudo apt-get install dotnet-sdk-2.2

References

https://kafka.apache.org/quickstart https://github.com/confluentinc/confluent-kafka-dotnet https://www.hugopicado.com/2017/11/22/getting-started-with-net-core-and-kafka.html https://github.com/wurstmeister/kafka-docker https://github.com/confluentinc/cp-docker-images/blob/master/examples/cp-all-in-one/docker-compose.yml https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

About

How to use dotnet core to send and consume data from Kafka topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published