Proxy test environment

[alex-dabija]

Stories

-As a Giant Swarm engineer, I want to have a test environment with a plain proxy in a separate VPC in order test the implementation of private networking with proxy support.

Background

Talax uses a transit gateway through which they route all their network traffic. Also, any HTTP and HTTPS traffic going to the Internet must go through a plain proxy. This proxy has a known domain name.

Requirements

• a VPC with 2 subnets (one private and one public) need to be created;
• the transit gateway created during mc-bootstrap cluster creation has to be attached to the VPC;
• a plain proxy for HTTP and HTTPS traffic must be part of the test environment;
• a public Route53 hosted zone needs to be created;
• the proxy domain name must be configured in the public hosted zone;
• HTTP and HTTPS must be routed to go through the plain proxy;
• any other HTTP and HTTPS traffic not routed through the proxy must be blocked at the transit gateway level;
• a NAT gateway must be configured in order to access the Internet through the proxy;
• the proxy must be created in a private subnet;
• a public subnet is needed for the NAT gateway.

Questions

• What automation technology should be used to get the test environment as soon as possible?

Probably, Terraform would be the fastest option to get the AWS resources created (VPC, Route53 hosted zone, etc.). I don't know if the plain proxy can be created with Terraform.
Also, an operator can be used to create all the required AWS resources.

• Does AWS offer a managed plain proxy service for HTTP and HTTPS traffic?

Unknown.

• Should the transit gateway be part of the test environment?

The transit gateway can just be attached to the management cluster VPC if it exists in the test environment. The transit gateway can be referenced in the management cluster configuration before it's created and it can be attached durring cluster creation.
The transit gateway needs to be created and attached to a few VPCs (management cluster VPC and the test environment VPC) before the management cluster creation can continue. The mc-bootstrap needs to be aware of the test environment.

• How are routing rules updated?

The routing rules need to be updated when a management or workload clusters is created. Maybe, transit gateway propagation rules or prefix lists could be used to ease the configuration.
Talax will have the routes already configured because they will know all the CIDR blocks before any cluster is created. This probably doesn't make much sense for a dynamic test environment.

Resources

• Transit gateway route tables;
• Transit gateway prefix lists refrences;
• Prefix lists;
• Giant Swarm test proxy environment diagram;

[alex-dabija]

An operator will be used to create the test environment as agreed during team Hydra planning.

[alex-dabija]

The VPN is not part of this story because at the moment the Kubernetes API is still public. The VPN story needs to be implemented before the clusters are configured to use a private load balancer for the Kubernetes API.

[calvix]

squid proxy app https://github.com/giantswarm/squid-proxy-app

outgoing proxy app stack that will deploy cluster and app together
https://github.com/giantswarm/outgoing-proxy-stack

[alex-dabija]

The proxy test environment is created, if enabled, at the same time when a new cluster is created. We successfully routed traffic through the Squid proxy for a pod manually configured with proxy settings.

Further testing is blocked for now until we finish the following two stories:

• CAPA private networking #1355;
• CAPI providers work behind an HTTP(S) proxy #1424.

[calvix]

still partially blocked, but starting to prepare some stuff for test proxy MC

[alex-dabija]

All the cloud-init kubeadm phases finished successfully.

[calvix]

cluster machines can boot successfully, all machines join the kubernetes cluster, now I need to sort the core apps that are necessary to run the cluster