Skip to content

Latest commit

 

History

History
289 lines (222 loc) · 7 KB

README.md

File metadata and controls

289 lines (222 loc) · 7 KB
Raw

Kafky

Kafka Terminal Client entirely written in RUST.

Features

  • auth
    • mutual ssl
    • plain
  • environments & credentials in a sharded configuration file ~/.kafky/config.yml
  • create/delete/get topic
  • get consumer groups
    • lag calculation
  • consume messages from multiple topics
    • auto commit
    • different reset
    • json format
    • key / value
  • produce messages
    • message history / search
  • brew installation

Installation

Cargo

$ cargo install --git https://github.com/n3wtron/kafky

Brew

Work in progress

Configuration

location $HOME/.kafky/config.yml

---
environments:
  - name: sample-env
    brokers:
      - "localhost:9094"
    truststore:
      #path:
      #base64:
      pem: |
        -----BEGIN CERTIFICATE-----
        DQEJARYAMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCJ9WRanG/fUvcfKiGl\n                    DQEJARYAMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCJ9WRanG/fUvcfKiGl\n                    DQEJARYAMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCJ9WRanG/fUvcfKiGl\n                    DQEJARYAMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCJ9WRanG/fUvcfKiGl\n                    DQEJARYAMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCJ9WRanG/fUvcfKiGl
        -----END CERTIFICATE-----
    credentials:
      - name: plain-cred
        plain:
          username: kafka-user
          password: kafka-password
      - name: ssl-cred
        ssl:
          certificate:
            path: /my.cert.pem
            #pem:
            #base64:
          privateKey:
            base64: bXkgcHJpdmF0ZSBrZXk=
            #pem:
            #path:
            password: my-cert-password

Commands

Get topics

USAGE:
    kafky get topics --output-format <format>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -o, --output-format <format>     [default: text]  [possible values: table, text, json]
list topics
$ kafly -e sample-env -c plain-cred get topics
bar
foo

table format

$ kafly -e sample-env -c plain-cred get topics -o table
TOPIC               PARTITION  LEADER  ISR    REPLICAS
foo                 0          3       3      3
foo                 0          1       1      1
bar                 0          1       1      1
bar                 1          1       1      1
bar                 2          1       1      1

json format

$ kafly -e sample-env -c plain-cred get topics -o json
{"name":"test-c","partitions":[{"id":0,"leader":3,"replicas":[3],"isrs":[3]}]}
{"name":"test-b","partitions":[{"id":0,"leader":1,"replicas":[1],"isrs":[1]}]}
{"name":"bar","partitions":[{"id":0,"leader":1,"replicas":[1],"isrs":[1]},{"id":1,"leader":1,"replicas":[1],"isrs":[1]},{"id":2,"leader":1,"replicas":[1],"isrs":[1]}]}

Get Consumer groups

USAGE:
    kafky get consumer-groups [OPTIONS] --output-format <format>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -o, --output-format <format>     [default: table]  [possible values: table, json]
    -g, --group <groups>...         filter by group names
        --timeout <timeout>         timeout in seconds reading __consumer_offsets topic [default: 10]
    -t, --topic <topics>...         filter by topic names

WARNING: This command is reading from the __consumer_offsets topic, for --timeout seconds (default is 10secs), increase it to have more reliable data

Example

$> kafky -e sample-env -c plain-cred get consumer-groups

GROUP     TOPIC  PARTITION  OFFSET  LAG
ssl-host  bar    0          10      14
ssl-host  bar    2          10      12
ssl-host  bar    1          14      8
test      bar    2          11      11
test      bar    0          11      13
test      bar    1          16      6
my-host   bar    1          22      0
my-host   bar    0          22      2
my-host   bar    2          22      0

Consume

USAGE:
    kafky consume [FLAGS] [OPTIONS] --consumer-group <CONSUMER GROUP NAME> --topic <TOPIC_NAME>...

FLAGS:
    -a, --autocommit    
        --earliest      read from the earliest offset
    -h, --help          Prints help information
        --latest        read from the latest offset (default)
        --timestamp     print timestamp message (works only with text format)
    -V, --version       Prints version information

OPTIONS:
    -c, --consumer-group <CONSUMER GROUP NAME>     [default: $HOSTNAME]
    -o, --output-format <format>                   [default: text]  [possible values: json, text]
    -k, --key-separator <key-separator>           
    -t, --topic <TOPIC_NAME>...

Examples

Simple
$ kafky -e sample-env -c plain-cred consume -t bar

Consume messages from bar topic in the sample-env environment using plain-cred credential

By default, it will consume from the latest offset without commit

From the beginning
$ kafky -e sample-env -c plain-cred consume -t bar --earliest
Auto commit
$ kafky -e sample-env -c plain-cred consume -t bar --auto-commit --consumer-group kafky-group

By default, the consumer group is the $HOSTNAME

Key separator
$ kafky -e sample-env -c plain-cred consume -t bar --key-separator ::
Multiple topics
$ kafky -e sample-env -c plain-cred consume -t bar -t foo
Message timestamp
$ kafky -e sample-env -c plain-cred consume -t bar --timestamp
JSON format
$ kafky -e sample-env -c plain-cred consume -t bar -o json

Produce

USAGE:
    kafky produce [OPTIONS] --topic <TOPIC_NAME>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -k, --key-separator <key-separator>    
    -t, --topic <TOPIC_NAME>

Example

$ kafky -e sample-env -c plain-cred produce -t bar --key-separator ::
bar <- my-key::my-payload
bar <- _

Create Topics

USAGE:
    kafky create topics --partitions <partitions> --replication-factor <replication_factor> --topic <topic>...

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -p, --partitions <partitions>                    number of partitions
    -r, --replication-factor <replication_factor>    replication factor
    -t, --topic <topic>...
Example
$ kafky -e sample-env -c plain-cred create topics -t topic-1 -t topic-2 --partition 3 --replication-factor 1

Delete Topics

USAGE:
    kafky delete topics [FLAGS] --topic <topic>...

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information
    -y               without confirmation

OPTIONS:
    -t, --topic <topic>...    topic name
Example
$ kafky -e sample-env -c plain-cred delete topics -t topic-1 -t topic-2
Are you sure that you want to delete the topic-1, topic-2 topics? [y/N]
_

Config

$ kafky config edit