Skip to content

nnson1610/cdc-pulsar-to-mysql

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

REQUIREMENT

  • Docker version 23.0.5

HOW TO RUN

  1. Install mysql in local:

  2. Config mysql credentials in cdc.yaml

  3. Run docker compose command:

  docker-compose up --build

To rerun specific service

  docker-compose up --build --force-recreate <service_name> 

To go to docker cmd

  docker ps
  docker exec -it <container_id> /bin/sh
  1. Go to docker cmd and run:
  bin/pulsar-admin source localrun  --sourceConfigFile cdc.yaml
  1. View topics:
  bin/pulsar-admin topics list public/default
  1. Consume message:
  bin/pulsar-client consume -s "sub-tank" public/default/dbserver1.inventory.tank -n 0

MYSQL

  1. Go to 'mysql' docker cmd (pwd: password):
  mysql -u user -p inventory;
  use inventory;
  1. MySQL commands:
  create table <table_name> ( <column1>: <datatype>, <column2>: <datatype2>) 
  insert into <table_name> values (<value1>,<value2>)

PULSAR MANAGER

  • GET XSRF-TOKEN:
      curl --location --request GET 'http://localhost:7750/pulsar-manager/csrf-token' \
  • Create manage user
      curl --location --request PUT 'http://localhost:7750/pulsar-manager/users/superuser' \
      --header 'X-XSRF-TOKEN: {REPLACE_THIS_WITH_ABOVE_TOKEN}'\
      --header 'Cookie: XSRF-TOKEN={REPLACE_THIS_WITH_ABOVE_TOKEN};' \
      --header 'Content-Type: application/json' \
      --data-raw '{"name": "admin", "password": "apachepulsar", "description": "test", "email": "[email protected]"}'
  • Go to management URL: http://localhost:9527/#/environments
      ServiceURL: http://pulsar:8080
      Bookie URL: http://pulsar:6650

ISSUES

  1. Bookie handle is not available -> stop, remove all pulsar manager + pulsar container -> remove folder ./tools (which is created automatically when running)

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published