readme: there is no -solo.yml manifest; various lint changes to README; fix name of Stark & Wayne

README.md

@@ -6,20 +6,20 @@ This is a BOSH release and deployment manifest deploy the Confluent Platform on
 
 Containers are fun, but getting stateful workloads in them can get a bit arkward. The abstraction layers it brins only add more complexity to the primitives they require only in the name of cloud independance.
 
-What if a cloud agnostic resource orchestrator existed and that was closer to the underlying cloud native resources. Enters [Bosh](https://bosh.io). As stated by the project presentation : 
+What if a cloud agnostic resource orchestrator existed and that was closer to the underlying cloud native resources. Enters [BOSH](https://bosh.io). As stated by the project presentation:
 
 > BOSH is a project that unifies release engineering, deployment, and lifecycle management of small and large-scale cloud software. BOSH can provision and deploy software over hundreds of VMs. It also performs monitoring, failure recovery, and software updates with zero-to-minimal downtime.
 
 Long story short, Bosh let you declare a desired state of your software and the underlying physical infrastructure (disk, vm instance, networks, OS, etc) and ensures that it meets the actual state. It was inspired by Google Borg. OS upgrade? Bosh will update on a rolling deployment. Same for software or resource change. A VM stops responding?  Bosh will detach disk, trash the VM and recreate a new one before re-attaching persistent disks and restarting the processes. Think of it as a Kubernetes for your IaaS resources.
 
 ## Getting started on Bosh
 
-[Stark and Wayne](https://starkandwayne.com) provides an incredible [Bosh tutorial](http://ultimateguidetobosh.com/). That is a recommeded first step to enter the world of Bosh.
+[Stark & Wayne](https://starkandwayne.com) provides an incredible [BOSH tutorial](http://ultimateguidetobosh.com/). That is a recommeded first step to enter the world of Bosh.
 
 ## TL;DR - I just want to deploy
 
 * [AWS deployment instructions](doc/aws-instructions.md)
-* GCP Deployment instructions - sooooon
+* [GCP Deployment instructions](gcp-instructions.md)
 * vSphere Deployment instructions - sooooon
 * Virtual Box deployment instructions - sooooon
 
@@ -37,16 +37,11 @@ A lot of security features are to be implemented. For a complete state of the bi
 
 This current iteration was successully tested on AWS and GCP cpis.
 
-## Deploy single collocated VM
-
-```plain
-bosh deploy confluent-platform-bosh-release/manifests/confluent-platform-solo.yml -o confluent-platform-bosh-release/manifests/operators/create.yml
-```
-
 ## Deploy Confluent Platform Cluster
 
-```
-bosh deploy confluent-platform-bosh-release/manifests/confluent-platform.yml -o confluent-platform-bosh-release/manifests/operators/create.yml
+```plain
+bosh deploy confluent-platform-bosh-release/manifests/confluent-platform.yml \
+    -o confluent-platform-bosh-release/manifests/operators/create.yml
 ```
 
 ## Updates
@@ -55,21 +50,21 @@ When new versions of `confluent-platform-bosh-release` are released the `manifes
 
 ```plain
 export BOSH_ENVIRONMENT=<bosh-alias>
-export BOSH_DEPLOYMENT=confluent-platform-dev
+export BOSH_DEPLOYMENT=confluent-platform
 cd confluent-platform-bosh-release
 git pull
 cd -
-bosh deploy confluent-platform-bosh-release/manifests/confluent-platform-solo.yml
+bosh deploy confluent-platform-bosh-release/manifests/confluent-platform.yml
 ```
 
 ## Development
 
 To iterate on this BOSH release, use the `create.yml` manifest when you deploy:
 
 ```plain
-bosh deploy manifests/confluent-platform-solo.yml -o manifests/operators/create.yml
+bosh deploy manifests/confluent-platform.yml -o manifests/operators/create.yml
 ```
 
 ## Acknowledgement
 
-Big shout out to [Stark and Wayne](https://starkandwayne.com) for their inspiration with their [Kafka Bosh Release](https://github.com/cloudfoundry-community/kafka-boshrelease). The openjdk package used by release is provided by them.
+Big shout out to [Stark & Wayne](https://starkandwayne.com) for their inspiration with their [Kafka Bosh Release](https://github.com/cloudfoundry-community/kafka-boshrelease). The openjdk package used by release is provided by them.

config/blobs.yml

@@ -1,8 +1,11 @@
-confluent-platform/confluent-5.2.1-2.12.zip:
-  size: 530385649
-  object_id: 1084d580-918f-4c92-7dc8-d390b8739e2b
-  sha: sha256:079fee8774671769fbb2124428f5f75224d0c8836d36efede643b597727944b9
+confluent-platform/confluent-5.3.0-2.12.zip:
+  size: 798771747
+  sha: sha256:fedbd2b80ec39afa815c908f287e4ab7704e508fa7847139fec6f65ea9e7623d
 java/jdk8u192-b03.tar.gz:
   size: 45670457
   object_id: 5a70262f-3127-4a35-6685-b271ba939661
   sha: sha256:5d8203117cad2ed7ef1e20d951f3c1b1515f725484e35cc10c61307e66018efe
+minio/mc:
+  size: 16605184
+  object_id: 0fb6f283-7aea-4c8a-5157-c6d3a509680f
+  sha: sha256:67280ce05acdd656156ca39b266f2931889ed2b58b703300639b1ccba645a6b3

doc/gcp-instructions.md

@@ -10,62 +10,74 @@ Example CIDR : 10.0.10.0/16
 
 ### Create subnets
 
-#### Infrastructure
-
-* Example subnet 1 name : infrastructure
-* Example subnet 1 CIDR : 10.0.10.0/24
-* Example subnet 1 region : northamerica-northeast1
-
-#### Confluent Platform
-
-* Example subnet 2 name : confluent-platform
-* Example subnet 2 CIDR : 10.0.20.0/24
-* Example subnet 2 region : northamerica-northeast1
+| Name | CIDR | Region |
+|---|---|---|
+| instrastructure | 10.0.10.0/24 | northamerica-northeast1 |
+| confluent-platform | 10.0.20.0/24 | northamerica-northeast1 |
 
 ### Create Firewall rules
 
 | Name | Targets | Filters | Protocols / ports | Network |
 | ------------- | ------------- | ------------- | ------------- | ------------- |
 | bosh-allow-ssh | allow-ssh | IP ranges: 0.0.0.0/0  | tcp:22 | cp-bosh |
 | bosh-unrestricted | confluent-platform | Tags: confluent-platform  | all | cp-bosh |
-|  bosh-allow-control-center | allow-control-center | IP ranges: 0.0.0.0/0 | tcp:9021 | cp-bosh |
-
-### Create a TCP Load Balancer for Confluent Server
-
-TODO
+| bosh-allow-control-center | allow-control-center | IP ranges: 0.0.0.0/0 | tcp:9021 | cp-bosh |
+| bosh-allow-ksql | allow-ksql | IP ranges: 0.0.0.0/0 | tcp:8088 | cp-bosh |
 
 ### Create unmanaged instance groups for Control Center
 
-Instance Group 1 Name : cp-control-center
-Instance Group 1 Zone : northamerica-northeast1-a
-Instance Group 1 Network : cp-bosh
-Instance Group 1 Subnet : confluent-platform
-
-
-Instance Group 2 Name : cp-control-center
-Instance Group 2 Zone : northamerica-northeast1-b
-Instance Group 2 Network : cp-bosh
-Instance Group 2 Subnet : confluent-platform
+| Number | Zone | Name | Network | Subnet |
+|---|---|---|---|---|
+| 1 | northamerica-northeast1-a | cp-control-center | cp-bosh | confluent-platform |
+| 2 | northamerica-northeast1-b | cp-control-center | cp-bosh | confluent-platform |
+| 3 | northamerica-northeast1-c | cp-control-center | cp-bosh | confluent-platform |
 
+### Create unmanaged instance groups for KSQL
 
-Instance Group 3 Name : cp-control-center
-Instance Group 3 Zone : northamerica-northeast1-c
-Instance Group 3 Network : cp-bosh
-Instance Group 3 Subnet : confluent-platform
+| Number | Zone | Name | Network | Subnet |
+|---|---|---|---|---|
+| 1 | northamerica-northeast1-a | cp-ksql | cp-bosh | confluent-platform |
+| 2 | northamerica-northeast1-b | cp-ksql | cp-bosh | confluent-platform |
+| 3 | northamerica-northeast1-c | cp-ksql | cp-bosh | confluent-platform |
 
 ### Create an Http Load Balancer for Control Center
 
 #### Backend services
 
 Instance Group : cp-control-center
+
 Port number : 9021
+
 Health check : HTTP on :9021/
+
 Backend Services : cp-control-center
 
 #### Frontend
 
 protocol :  http
+
 port : 80
+
+ip : Reserved ipv4
+
+### Create an Http Load Balancer for KSQL
+
+#### Backend services
+
+Instance Group : cp-ksql
+
+Port number : 8088
+
+Health check : HTTP on :8088/
+
+Backend Services : cp-ksql
+
+#### Frontend
+
+protocol :  https
+
+port : 443
+
 ip : Reserved ipv4
 
 ### Create a jumpbox to run Bosh CLI Commands

doc/state-of-security.md

@@ -1,91 +1,44 @@
 # State of security implementation
 
-- [ ] Broker
-  - [X] Brokers to brokers
-    - [X] Encryption
-    - [X] Authentication
-  - [ ] Metric reporter
-    - [X] Encryption
-    - [X] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-
-- [ ] Connect
-  - [ ] Workers to Brokers
-    - [X] Encryption
-    - [X] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-  - [ ] Rest API
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] RBAC
-  - [ ] Interceptors
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-
-- [ ] KSQL
-  - [ ] KSQL nodes to Brokers
-    - [X] Encryption
-    - [X] Authentication
-    - [ ] ACL
-  - [ ] Rest API
-    - [ ] Encryption
-    - [ ] Authentication
-  - [ ] Schema Registry
-    - [ ] Encryption
-    - [ ] Authentication
-  - [ ] Interceptors
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] ACL
-
-- [ ] Schema Registry
-  - [ ] Schema registry to Brokers
-    - [X] Encryption
-    - [X] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-  - [ ] Rest API
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] RBAC
-  - [ ] Interceptors
-    - [ ] SSL
-    - [ ] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-
-- [ ] Control Center
-  - [ ] Rest API
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-  - [ ] Brokers
-    - [X] Encryption
-    - [X] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-  - [ ] Connect
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-  - [ ] KSQL
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] ACL
-    - [ ] RBAC
+- [X] Encryption
+  - [X] Kafka Broker
+    - [X] mTLS for broker intercommunication
+    - [X] mTLS between Metric Reporters and Kafka Cluster
+  - [X] Kafka Connect  
+    - [X] mTLS with Kafka cluster
+    - [X] Https for Connect REST endpoints
+  - [X] Schema Registry
+    - [X] mTLS with Kafka cluster
+    - [X] Https for REST endpoints
+  - [X] KSQL
+    - [X] mTLS with Kafka cluster
+    - [X] Https for REST endpoints
+  - [X] Control Center
+    - [X] mTLS with Kafka cluster
+    - [X] Https for REST endpoints
+- [ ] Authentication
+  - [ ] Kafka Broker
+    - [X] SASL for broker intercommunication
+    - [X] SASL between Metric Reporters and Kafka Cluster (to test)
+  - [X] Kafka Connect
+    - [X] SASL with Kafka cluster
+    - [X] REST endpoints
   - [ ] Schema Registry
-    - [ ] Encryption
-    - [ ] Authentication
-    - [ ] ACL
-    - [ ] RBAC
-  - [ ] Zookeeper
-    - [ ] Authentication
-
-- [ ] Zookeeper
-  - [ ] Authentication
+    - [X] SASL with Kafka cluster
+    - [ ] REST endpoints
+  - [X] KSQL
+    - [X] SASL for with Kafka cluster
+    - [X] REST endpoints
+  - [X] Control Center
+    - [X] SASL with Kafka cluster
+    - [X] Basic Auth for REST endpoints
+- [ ] Kafka Topics ACL
+  - [ ] Kafka Connect
+  - [ ] Schema Regisry
+  - [ ] KSQL
+  - [ ] Control Center
+- [ ] RBAC
+  - [ ] Kafka Connect
+  - [ ] Schema Regisry
+  - [ ] KSQL
+  - [ ] Control Center