Skip to content

Multiple Kafka Clusters

Jump to bottom
Naresh Kumar Vudutha edited this page Feb 20, 2019 · 3 revisions

This tutorial is helpful for users who are running multiple kafka clusters and would like to use Cruise Control UI (CCFE).

Goal

By the end of this exercise you will have single CCFE managing all your kafka clusters.

This is how the data flow looks like in this setup

                                                                     +-----------------------+
                                +-------------+                      |                       |
                                |             |       +------------->+ cc-1 (10.0.0.1:9090)  |
+----------------+  GET /       |             |       |              |                       |
|                |              |  nginx (or) |       |              +-----------------------+
|   User / CCFE  +<------------>+  Apache     |       |
|                |   CCFE FILES |             |       |              +-----------------------+
+-------+--------+              |  10.0.0.4   |       |              |cc-2 (10.0.0.2:9090)   |
        ^                       |  port 80    |       +------------->+                       |
        |                       |             |       |              +-----------------------+
        |                       |             |       |
        +---------------------->+             +<------+              +-----------------------+
      {GET,POST}                |             |       |              |                       |
      /cc-N/kafkacruisecontrol  +-------------+       +------------->+ cc-3 (10.0.0.3:9090)  |
                                                   CC REST           +-----------------------+
                                                   API (/kafkacruisecontrol)

Assumptions

  1. Zookeeper(s) is/are up and running.
  2. Three Kafka Clusters (1 or more) are setup and running.
  3. Three Cruise Control (1 or more) instances are setup and running.
    1. CC-1 Running on ip 10.0.0.1 & port 9090 with hostname cc-1.example.com
    2. CC-2 Running on ip 10.0.0.2 & port 9090 with hostname cc-2.example.com
    3. CC-3 Running on ip 10.0.0.3 & port 9090 with hostname cc-3.example.com
  4. Cruise Control Servers are exposing their REST API on URL Path /kafkacruisecontrol/*

Requirements

You need a webserver like Apache, Nginx which supports serving static content and at the same time supports Reverse Proxy.

  1. Lets say, this webserver is running on ip 10.0.0.4 & port 80 with hostname ccfe.example.com
  2. Document root for the webserver is configured at /var/www/html/
  3. Webserver can reach all CC-{1,2,3} nodes on their ip & ports
    1. TCP from 10.0.0.4:* is allowed to access TCP on 10.0.0.1:9090 # cc-1.example.com
    2. TCP from 10.0.0.4:* is allowed to access TCP on 10.0.0.2:9090 # cc-2.example.com
    3. TCP from 10.0.0.4:* is allowed to access TCP on 10.0.0.3:9090 # cc-3.example.com
  4. Webserver (10.0.0.4:80) is accessible from selective users (or ip address, subnets).

In general, please deny everything and allow only the traffic thats accepted.

Configure Webserver

All the common steps are shown below. Wherever there are dedicated steps required for the targeted webserver, they are highlighted as needed.

CCFE with Nginx (or) Apache

  1. Download the latest compiled artifacts from https://github.com/linkedin/cruise-control-ui/releases page

This will have the following files once extracted.

/home/user/Downloads $ tar zxvf cruise-control-ui.tar.gz 
cruise-control-ui/
cruise-control-ui/dist/
cruise-control-ui/README.txt
cruise-control-ui/dist/index.html
cruise-control-ui/dist/static/
cruise-control-ui/dist/static/cc-logo.png
cruise-control-ui/dist/static/css/
cruise-control-ui/dist/static/js/
cruise-control-ui/dist/static/config.csv
cruise-control-ui/dist/static/js/manifest.js
cruise-control-ui/dist/static/js/vendor.js
cruise-control-ui/dist/static/js/app.js
cruise-control-ui/dist/static/css/app.css
cruise-control-ui/dist/static/css/app.css.map
  1. Copy the cruise-control-ui.tar.gz to the webserver.
scp cruise-control-ui.tar.gz [email protected]:/tmp/
  1. Extract the cruise-control-ui.tar.gz inside the server root folder.
ssh [email protected]
cd /var/www/html/
sudo tar zxvf /tmp/cruise-control-ui.tar.gz
  1. Update the config.csv so that webserver can reach the Cruise Control Server(s)
ssh [email protected]
cd /var/www/html/
mv /var/www/html/cruise-control-ui/dist/* /var/www/html/
rmdir /var/www/html/cruise-control-ui/dist
rmdir /var/www/html/cruise-control-ui

add the URL portion to config.csv in such a way that the webserver (nginx/apache) can uniquely identify requests coming from browser and reverse-proxy them to cruise control.

cat /var/www/html/
cat config.csv
region-1,cc-one,/cc-1/kafkacruisecontrol
region-1,cc-two,/cc-2/kafkacruisecontrol
region-1,cc-three,/cc-3/kafkacruisecontrol

NOTE: /cc-$N/kafkacruisecontrol is relative to webserver which is listening on 10.0.0.4 & port 80. So all requests made by CCFE will go to http://10.0.0.4:80/cc-$N/kafkacruisecontrol/ end point(s), depending on which cluster user is looking at, on the CCFE.

  1. Webserver specific configuration

Update nginx config, so that the custom url convention we have come-up with is made available to nginx.

Make these changes in the appropriate nginx server directive.

// 10.0.0.1 is first CC (i.e cc-1.example.com)
location /cc-1/kafkacruisecontrol/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_buffering off;
    proxy_pass http://10.0.0.1:9090/kafkacruisecontrol/;
}

// 10.0.0.2 is second CC (i.e cc-2.example.com)
location /cc-2/kafkacruisecontrol/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_buffering off;
    proxy_pass http://10.0.0.2:9090/kafkacruisecontrol/;
}

// 10.0.0.3 is third CC (i.e cc-3.example.com)
location /cc-3/kafkacruisecontrol/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_buffering off;
    proxy_pass http://10.0.0.3:9090/kafkacruisecontrol/;
}

Update Apache config, so that the custom url convention we have come-up with is known to webserver.

Make these changes in the appropriate Apache server <Directory> path.

// 10.0.0.1 is first CC (i.e cc-1.example.com)
ProxyPass "/cc-1/kafkacruisecontrol/" "http://10.0.0.1:9090/kafkacruisecontrol/"
ProxyPassReverse "/cc-1/kafkacruisecontrol/" "http://10.0.0.1:9090/kafkacruisecontrol/"

// 10.0.0.2 is second CC (i.e cc-2.example.com)
ProxyPass "/cc-2/kafkacruisecontrol/" "http://10.0.0.2:9090/kafkacruisecontrol/"
ProxyPassReverse "/cc-2/kafkacruisecontrol/" "http://10.0.0.2:9090/kafkacruisecontrol/"

// 10.0.0.3 is third CC (i.e cc-3.example.com)
ProxyPass "/cc-3/kafkacruisecontrol/" "http://10.0.0.3:9090/kafkacruisecontrol/"
ProxyPassReverse "/cc-3/kafkacruisecontrol/" "http://10.0.0.3:9090/kafkacruisecontrol/"
  1. Hit the Cruise Control host & port in the browser to access CCFE

http://cc-host.example.com/

This will show UI with 3 drop down links in the top navigation bar. Select the cluster that you want to operate next.

Few Notes

URL portion in the config.csv (third value in CSV row) is relative to the webserver in which the CCFE is deployed. So, please take some extra caution understanding how the URL routing happens.

Security

Since we are deploying CCFE within a webserver. Please secure it by using proper firewalls or Authentication provided by the webserver. Any unauthorized use will compromise your application security.

Clone this wiki locally