Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigAnimal: Azure-BAH private networking #4858

Merged
merged 8 commits into from
Oct 5, 2023
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Connecting from Azure
navTitle: From Azure
deepToC: true
redirects:
- /biganimal/release/using_cluster/connecting_your_cluster/01_connecting_from_azure
- /biganimal/release/using_cluster/02_connecting_your_cluster/01_connecting_from_azure/01_private_endpoint
Expand All @@ -10,8 +11,6 @@ Three different methods enable you to connect to your cluster from your applicat

## Azure private endpoint (recommended)

While other methods for connecting your cluster from your application's virtual network in Azure are available, we strongly recommend using the Azure private endpoint method.

Azure private endpoint is a network interface that securely connects a private IP
address from your Azure virtual network (VNet) to an external service. You grant access only to a single cluster instead of the entire BigAnimal resource virtual network, thus ensuring maximum network isolation. Other advantages include:
- You need to configure the Private Link only once. Then you can use multiple private endpoints to connect applications from many different VNets.
Expand All @@ -23,8 +22,24 @@ Private endpoints are the same mechanism used by first-party Azure services such
If you set up a private endpoint and want to change to a public network, you must remove the private endpoint resources before making the change.
!!!

### Using BigAnimal's cloud account

When using BigAnimal's cloud account, when creating a cluster, you provide BigAnimal with your Azure subscription ID (see [Networking](/biganimal/latest/getting_started/creating_a_cluster/#network-logs--telemetry-section)). BigAnimal, in turn, provides you with a private link alias, which you can use to connect to your cluster privately.

1. When creating your cluster, on the Cluster Settings tab, in the Network section:
ccestes marked this conversation as resolved.
Show resolved Hide resolved
1. Select **Private**.

1. Enter your application's Azure subscription ID.

1. After the cluster is created, go to the cluster details to see the corresponding endpoint service name. You need the service name while creating a private endpoint.

1. Create a private endpoint in the client's VNet. The steps for creating a private endpoint in the client's VNet are the same whether you're using BigAnimal's cloud or your own. See [Step 1: Create an Azure private endpoint](#step-1-create-an-azure-private-endpoint) and [Step 2: Create an Azure Private DNS Zone for the private endpoint](#step-2-create-an-azure-private-dns-zone-for-the-private-endpoint).

1. In your application's Azure account, select **Private Link Center**, and then select **Private endpoints**. Select the endpoint you created previously, and use the service name provided in the details section in BigAnimal to access your cluster.

### Using your Azure account

## Private endpoint example
#### Example

This example shows how to connect your cluster using Azure private endpoint.

Expand All @@ -46,7 +61,7 @@ Assume that your cluster is on a subscription called `development` and is being
- Virtual network subnet: `snet-client`


### Prerequisites
#### Prerequisites

To walk through an example in your own environment, you need:

Expand All @@ -67,11 +82,11 @@ To walk through an example in your own environment, you need:
In this example, you create an Azure private endpoint in your client VM's virtual network. After you create the private endpoint, you can use its private IP address to access the Postgres cluster. You must perform this procedure for every virtual network you want to connect from.


### Step 1: Create an Azure private endpoint
#### Step 1: Create an Azure private endpoint

Create an Azure private endpoint in each client virtual network that needs to connect to your BigAnimal cluster. You can create the private endpoint using either the [Azure portal](#using-the-azure-portal) or the [Azure CLI](#using-the-azure-cli).

#### Using the Azure portal
##### Using the Azure portal

1. If you prefer to create the private endpoint using the Azure portal, on the upper-left side of the screen, select **Create a resource > Networking > Private Link**. Alternatively. in the search box enter `Private Link`.

Expand Down Expand Up @@ -129,7 +144,7 @@ you created by entering the following details:

10. Proceed to [Accessing the cluster](#accessing-the-cluster).

#### Using the Azure CLI
##### Using the Azure CLI

If you prefer to create the private endpoint using the Azure CLI, either use your local terminal with an Azure CLI profile already configured or open a new Azure Cloud Shell using the Azure portal.

Expand Down Expand Up @@ -160,7 +175,7 @@ az network private-endpoint create \

- `subscription` is the Azure subscription in which to create the private endpoint.

### Accessing the cluster
#### Accessing the cluster
You have successfully built a tunnel between your client VM's virtual network and the cluster. You can now access the cluster from the private endpoint in your client VM. The private endpoint's private IP address is associated with an independent virtual network NIC. Get the private endpoint's private IP address using the following commands:

```shell
Expand All @@ -185,7 +200,7 @@ edb_admin=>

```

### Step 2: Create an Azure Private DNS Zone for the private endpoint
#### Step 2: Create an Azure Private DNS Zone for the private endpoint

EDB strongly recommends using a [private Azure DNS zone](https://docs.microsoft.com/en-us/azure/dns/private-dns-privatednszone) with the private endpoint to establish a connection with a cluster. You can't validate TLS certificates using `verify-full` when connecting to an IP address.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,49 @@ PostgresAuditLogs_CL
| project record_log_time_s, record_error_severity_s, record_message_s
| sort by record_log_time_s desc
```

### Using BigAnimal's cloud account

To access your Postgres cluster logs, when using BigAnimal's cloud account, generate a SAS token from BigAnimal and use it to download the logs.

1. In the BigAnimal portal, select **Clusters**, select your cluster, and select the **Monitoring & Logging** tab.

1. Select **Generate Token** and copy the SAS token. The SAS token is a sensitive value and shouldn't be made publicly available. The following is a sample SAS token:
ccestes marked this conversation as resolved.
Show resolved Hide resolved

```
https://blobsamples.blob.core.windows.net/?sv=2022-11-02&ss=b&srt=sco&sp=rwlc&se=2023-05-24T09:51:36Z&st=2023-05-24T01:51:36Z&spr=https&sig=<signature>
```

1. Enter the `azcopy` command to download the Postgres logs from BigAnimal. For example:

```
azcopy copy '$TOKEN' . --recursive
INFO: Scanning...
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

Job aa4b74a0-bc92-be4e-551c-47aec1c1cfc3 has started
Log file is located at: /Users/sample_user/.azcopy/aa4b74a0-bc92-be4e-551c-47aec1c1cfc3.log

100.0 %, 5 Done, 0 Failed, 0 Pending, 0 Skipped, 5 Total, 2-sec Throughput (Mb/s): 0.5375


Job aa4b74a0-bc92-be4e-551c-47aec1c1cfc3 summary
Elapsed Time (Minutes): 0.0333
Number of File Transfers: 5
Number of Folder Property Transfers: 0
Number of Symlink Transfers: 0
Total Number of Transfers: 5
Number of File Transfers Completed: 5
Number of Folder Transfers Completed: 0
Number of File Transfers Failed: 0
Number of Folder Transfers Failed: 0
Number of File Transfers Skipped: 0
Number of Folder Transfers Skipped: 0
TotalBytesTransferred: 134416
Final Job Status: Completed
$ tail p-a1b2c3d4d5/kubernetes-logs/p-a1b2c3d4d5/2023/09/26/13/19/azure_customer_postgresql_cluster.var.log.containers.p-a1b2c3d4d5-1_p-a1b2c3d4d5_postgres-c798aa19ea0481c8d9575f025405b3ad9212816ca7e928f997473055499a692c.log
{"@timestamp":"2023-09-26T13:19:19.572442Z","level":"info","ts":"2023-09-26T13:19:19Z","logger":"wal-archive","msg":"Archived WAL file","logging_pod":"p-a1b2c3d4d5-1","walName":"pg_wal/000000010000000000000006","startTime":"2023-09-26T13:19:18Z","endTime":"2023-09-26T13:19:19Z","elapsedWalTime":1.060413255,"stream":"stdout","logtag":"F","message":"{\"level\":\"info\",\"ts\":\"2023-09-26T13:19:19Z\",\"logger\":\"wal-archive\",\"msg\":\"Archived WAL
```

## Metrics

Expand Down