Skip to content

Commit

Permalink
Renaming ESA to DMA (#245)
Browse files Browse the repository at this point in the history
* Renaming ESA to DMA

* * added new gif
* updated portal experience

* filename updates

* linting

* Fixing someone elses linting errors!!

* fixing broken links

* * updated file names
* updated broken links

* check links automatically

* updated link checker

* check failed links

* updated paths and linting yml

* fixed few more links

* esa to dma

* updated default tags across all deployments

Co-authored-by: Marvin Buss <[email protected]>
  • Loading branch information
esbran and marvinbuss authored Dec 13, 2021
1 parent f45e7f7 commit 4c33814
Show file tree
Hide file tree
Showing 28 changed files with 158 additions and 157 deletions.
34 changes: 17 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,35 @@
# Enterprise-Scale Analytics - Data Management Zone
# Data Management & Analytics Scenario - Data Management Zone

## Objective

The [Enterprise-Scale Analytics](https://aka.ms/adopt/datamanagement) architecture provides a prescriptive data platform design coupled with Azure best practices and design principles. These principles serve as a compass for subsequent design decisions across critical technical domains. The architecture will continue to evolve alongside the Azure platform and is ultimately driven by the various design decisions that organizations must make to define their Azure data journey.
The [Data Management & Analytics scenario](https://aka.ms/adopt/datamanagement) provides a prescriptive data platform design coupled with Azure best practices and design principles. These principles serve as a compass for subsequent design decisions across critical technical domains. The architecture will continue to evolve alongside the Azure platform and is ultimately driven by the various design decisions that organizations must make to define their Azure data journey.

The Enterprise-Scale Analytics architecture consists of two core building blocks:
The Data Management & Analytics architecture consists of two core building blocks:

1. *Data Management Zone* which provides all data management and data governance capabilities for the data platform of an organization.
1. *Data Landing Zone* which is a logical construct and a unit of scale in the Enterprise-Scale Analytics architecture that enables data retention and execution of data workloads for generating insights and value with data.
1. *Data Landing Zone* which is a logical construct and a unit of scale in the Data Management & Analytics architecture that enables data retention and execution of data workloads for generating insights and value with data.

The architecture is modular by design and allows organizations to start small with a single Data Management Zone and Data Landing Zone, but also allows to scale to a multi-subscription data platform environment by adding more Data Landing Zones to the architecture. Thereby, the reference design allows to implement different modern data platform patterns like data-mesh, data-fabric as well as traditional datalake architectures. Enterprise-Scale Analytics has been very well aligned with the data-mesh approach, and is ideally suited to help organizations build data products and share these across business units of an organization. If core recommendations are followed, the resulting target architecture will put the customer on a path to sustainable scale.
The architecture is modular by design and allows organizations to start small with a single Data Management Zone and Data Landing Zone, but also allows to scale to a multi-subscription data platform environment by adding more Data Landing Zones to the architecture. Thereby, the reference design allows to implement different modern data platform patterns like data-mesh, data-fabric as well as traditional datalake architectures. Data Management & Analytics has been very well aligned with the data-mesh approach, and is ideally suited to help organizations build data products and share these across business units of an organization. If core recommendations are followed, the resulting target architecture will put the customer on a path to sustainable scale.

![Enterprise-Scale Analytics](/docs/images/EnterpriseScaleAnalytics.gif)
![Data Management & Analytics](/docs/images/DataManagementAnalytics.gif)

---

_The Enterprise-Scale Analytics architecture represents the strategic design path and target technical state for your Azure data platform._
_The Data Management & Analytics architecture represents the strategic design path and target technical state for your Azure data platform._

---

This respository describes the Data Management Zone, which is classified as data management hub. It is the heart of the Enterprise-Scale Analytics architecture pattern and enables central governance of data assets across all Data Landing Zones. Enterprise-Scale Anayltics targets the deployment of a single Data Management Zone instance inside a tenant of an organization.
This repository describes the Data Management Zone, which is classified as data management hub. It is the heart of the Data Management & Analytics architecture pattern and enables central governance of data assets across all Data Landing Zones. Data Management and Analytics scenario targets the deployment of a single Data Management Zone instance inside a tenant of an organization.

> **Note:** Before getting started with the deployment, please make sure you are familiar with the [complementary documentation in the Cloud Adoption Framework](https://aka.ms/adopt/datamanagement). After deploying your Data Management Zone, please move on to the [Data Landing Zone](https://github.com/Azure/data-landing-zone) deployment to create an environment in which you can start working on generating insights and value with data. The minimal recommended setup consists of a single Data Management Zone and a single [Data Landing Zone](https://github.com/Azure/data-landing-zone).
## Deploy Enterprise-Scale Analytics
## Deploy Data Management & Analytics Scenario

The Enterprise-Scale Analytics architecture is modular by design and allows customers to start with a small footprint and grow over time. In order to not end up in a migration project, customers should decide upfront how they want to organize data domains across Data Landing Zones. All Enterprise-Scale Analytics architecture building blocks can be deployed through the Azure Portal as well as through GitHub Actions workflows and Azure DevOps Pipelines. The template repositories contain sample YAML pipelines to more quickly get started with the setup of the environments.
The Data Management & Analytics architecture is modular by design and allows customers to start with a small footprint and grow over time. In order to not end up in a migration project, customers should decide upfront how they want to organize data domains across Data Landing Zones. All Data Management & Analytics architecture building blocks can be deployed through the Azure Portal as well as through GitHub Actions workflows and Azure DevOps Pipelines. The template repositories contain sample YAML pipelines to more quickly get started with the setup of the environments.

| Reference implementation | Description | Deploy to Azure | Link |
|:---------------------------|:------------|:----------------|------|
| Enterprise-Scale Analytics | Deploys a Data Management Zone and one or multiple [Data Landing Zone](https://github.com/Azure/data-landing-zone) all at once. Provides less options than the the individual Data Management Zone and Data Landing Zone deployment options. Helps you to quickly get started and make yourself familiar with the reference design. For more advanced scenarios, please deploy the artifacts individually. |[![Deploy To Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#blade/Microsoft_Azure_CreateUIDef/CustomDeploymentBlade/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-management-zone%2Fmain%2Fdocs%2Freference%2FenterpriseScaleAnalytics.json/uiFormDefinitionUri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-management-zone%2Fmain%2Fdocs%2Freference%2Fportal.enterpriseScaleAnalytics.json) | |
| Data Management & Analytics Scenario | Deploys a Data Management Zone and one or multiple [Data Landing Zone](https://github.com/Azure/data-landing-zone) all at once. Provides less options than the the individual Data Management Zone and Data Landing Zone deployment options. Helps you to quickly get started and make yourself familiar with the reference design. For more advanced scenarios, please deploy the artifacts individually. |[![Deploy To Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#blade/Microsoft_Azure_CreateUIDef/CustomDeploymentBlade/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-management-zone%2Fmain%2Fdocs%2Freference%2FdataManagementAnalytics.json/uiFormDefinitionUri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-management-zone%2Fmain%2Fdocs%2Freference%2Fportal.dataManagementAnalytics.json) | |
| Data Management Zone | Deploys a single Data Management Zone to a subscription. |[![Deploy To Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#blade/Microsoft_Azure_CreateUIDef/CustomDeploymentBlade/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-management-zone%2Fmain%2Finfra%2Fmain.json/uiFormDefinitionUri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-management-zone%2Fmain%2Fdocs%2Freference%2Fportal.dataManagementZone.json) | [Repository](https://github.com/Azure/data-management-zone) |
| Data Landing Zone | Deploys a single Data Landing Zone to a subscription. Please deploy a Data Management Zone first. |[![Deploy To Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#blade/Microsoft_Azure_CreateUIDef/CustomDeploymentBlade/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-landing-zone%2Fmain%2Finfra%2Fmain.json/uiFormDefinitionUri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-landing-zone%2Fmain%2Fdocs%2Freference%2Fportal.dataLandingZone.json) | [Repository](https://github.com/Azure/data-landing-zone) |
| Data Product Batch | Deploys a Data Workload template for Data Batch Analysis to a resource group inside a [Data Landing Zone](https://github.com/Azure/data-landing-zone). Please deploy a Data Management Zone and [Data Landing Zone](https://github.com/Azure/data-landing-zone) first. |[![Deploy To Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#blade/Microsoft_Azure_CreateUIDef/CustomDeploymentBlade/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-product-batch%2Fmain%2Finfra%2Fmain.json/uiFormDefinitionUri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fdata-product-batch%2Fmain%2Fdocs%2Freference%2Fportal.dataProduct.json) | [Repository](https://github.com/Azure/data-product-batch) |
Expand All @@ -40,13 +40,13 @@ The Enterprise-Scale Analytics architecture is modular by design and allows cust

To deploy the Data Management Zone into your Azure Subscription, please follow the step-by-step instructions:

1. [Prerequisites](/docs/EnterpriseScaleAnalytics-Prerequisites.md)
2. [Create repository](/docs/EnterpriseScaleAnalytics-CreateRepository.md)
3. [Setting up Service Principal](/docs/EnterpriseScaleAnalytics-ServicePrincipal.md)
1. [Prerequisites](/docs/DataManagementAnalytics-Prerequisites.md)
2. [Create repository](/docs/DataManagementAnalytics-CreateRepository.md)
3. [Setting up Service Principal](/docs/DataManagementAnalytics-ServicePrincipal.md)
4. Template Deployment
1. [GitHub Action Deployment](/docs/EnterpriseScaleAnalytics-GitHubActionsDeployment.md)
2. [Azure DevOps Deployment](/docs/EnterpriseScaleAnalytics-AzureDevOpsDeployment.md)
5. [Known Issues](/docs/EnterpriseScaleAnalytics-KnownIssues.md)
1. [GitHub Action Deployment](/docs/DataManagementAnalytics-GitHubActionsDeployment.md)
2. [Azure DevOps Deployment](/docs/DataManagementAnalytics-AzureDevOpsDeployment.md)
5. [Known Issues](/docs/DataManagementAnalytics-KnownIssues.md)

## Contributing

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ The following table explains each of the parameters:
| Parameter | Description | Sample value |
|:--------------------------------------------|:-------------|:-------------|
| **AZURE_SUBSCRIPTION_ID** | Specifies the subscription ID of the Data Management Zone where all the resources will be deployed | <div style="width: 36ch">`xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`</div> |
| **AZURE_LOCATION** | Specifies the region where you want the resources to be deployed. Please check [Supported Regions](/docs/EnterpriseScaleAnalytics-Prerequisites.md). | `northeurope` |
| **AZURE_LOCATION** | Specifies the region where you want the resources to be deployed. Please check [Supported Regions](/docs/DataManagementAnalytics-Prerequisites.md#supported-regions). | `northeurope` |
| **AZURE_RESOURCE_MANAGER _CONNECTION_NAME** | Specifies the resource manager connection name in Azure DevOps. More details on how to create the resource manager service connection in Azure DevOps was described in the previous paragraph or [here](https://docs.microsoft.com/azure/devops/pipelines/library/connect-to-azure?view=azure-devops#create-an-azure-resource-manager-service-connection-with-an-existing-service-principal). | `my-connection-name` |

### Configure `params.dev.json`
Expand Down Expand Up @@ -136,11 +136,11 @@ After following the instructions and updating the parameters and variables in yo

**Congratulations!** You have successfully executed all steps to deploy the template into your environment through Azure DevOps.

Now, you can navigate to the pipeline that you have created as part of step 5 and monitor it as each service is deployed. If you run into any issues, please check the [Known Issues](/docs/EnterpriseScaleAnalytics-KnownIssues.md) first and open an [issue](https://github.com/Azure/data-management-zone/issues) if you come accross a potential bug in the repository.
Now, you can navigate to the pipeline that you have created as part of step 5 and monitor it as each service is deployed. If you run into any issues, please check the [Known Issues](/docs/DataManagementAnalytics-KnownIssues.md) first and open an [issue](https://github.com/Azure/data-management-zone/issues) if you come accross a potential bug in the repository.

## Add Root Collection Administrator role to Purview account

Since the Purview account is being deployed using a Service Principal, at this point, the only identity that has access to the Purview data plane, is the Service Principal. Use the [instructions in this guide](https://docs.microsoft.com/azure/purview/tutorial-metadata-policy-collections-apis#add-the-root-collection-administrator-role) to assign additional user accounts to the Purview Root Collection as Collection Admin.

>[Previous](/docs/EnterpriseScaleAnalytics-ServicePrincipal.md)
>[Next](/docs/EnterpriseScaleAnalytics-KnownIssues.md)
>[Previous](/docs/DataManagementAnalytics-ServicePrincipal.md)
>[Next](/docs/DataManagementAnalytics-KnownIssues.md)
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ First, you must generate your own respository based off this template respositor
1. Optionally, to include the directory structure and files from all branches in the template and not just the default branch, select **Include all branches**.
1. Click **Create repository from template**.

>[Previous](/docs/EnterpriseScaleAnalytics-Prerequisites.md)
>[Next](/docs/EnterpriseScaleAnalytics-ServicePrincipal.md)
>[Previous](/docs/DataManagementAnalytics-Prerequisites.md)
>[Next](/docs/DataManagementAnalytics-ServicePrincipal.md)
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ The following table explains each of the parameters:
| Parameter | Description | Sample value |
|:--------------------------------------------|:-------------|:-------------|
| **AZURE_SUBSCRIPTION_ID** | Specifies the subscription ID of the Data Management Zone where all the resources will be deployed | <div style="width: 36ch">`xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`</div> |
| **AZURE_LOCATION** | Specifies the region where you want the resources to be deployed. Please check [Supported Regions](/docs/EnterpriseScaleAnalytics-Prerequisites.md) | `northeurope` |
| **AZURE_LOCATION** | Specifies the region where you want the resources to be deployed. Please check [Supported Regions](/docs/DataManagementAnalytics-Prerequisites.md#supported-regions) | `northeurope` |

### Configure `params.dev.json`

Expand Down Expand Up @@ -89,11 +89,11 @@ After following the instructions and updating the parameters and variables in yo

**Congratulations!** You have successfully executed all steps to deploy the template into your environment through GitHub Actions.

Now, you can navigate to the **Actions** tab of the main page of the repository, where you will see a workflow with the name `Data Management Zone Deployment` running. Click on it to see how it deploys the environment. If you run into any issues, please check the [Known Issues](/docs/EnterpriseScaleAnalytics-KnownIssues.md) first and open an [issue](https://github.com/Azure/data-management-zone/issues) if you come accross a potential bug in the repository.
Now, you can navigate to the **Actions** tab of the main page of the repository, where you will see a workflow with the name `Data Management Zone Deployment` running. Click on it to see how it deploys the environment. If you run into any issues, please check the [Known Issues](/docs/DataManagementAnalytics-KnownIssues.md) first and open an [issue](https://github.com/Azure/data-management-zone/issues) if you come accross a potential bug in the repository.

## Add Root Collection Administrator role to Purview account

Since the Purview account is being deployed using a Service Principal, at this point, the only identity that has access to the Purview data plane, is the Service Principal. Use the [instructions in this guide](https://docs.microsoft.com/azure/purview/tutorial-metadata-policy-collections-apis#add-the-root-collection-administrator-role) to assign additional user accounts to the Purview Root Collection as Collection Admin.

>[Previous](/docs/EnterpriseScaleAnalytics-ServicePrincipal.md)
>[Next](/docs/EnterpriseScaleAnalytics-KnownIssues.md)
>[Previous](/docs/DataManagementAnalytics-ServicePrincipal.md)
>[Next](/docs/DataManagementAnalytics-KnownIssues.md)
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ This error message appears, if the Purview soft-limit has been reached inside yo
**Solution:**
This error message appears, when deploying Purview and the `Microsoft.EventHub`, `Microsoft.Storage` and/or `Microsoft.Purview` Resource Provider (RP) is not registered for the subscription or it was registered during early preview phase. If the `Microsoft.Purview` RP is already registered, please un-register and re-register it. We have released a fix to our Deploy To Azure Buttons to overcome this issue until it gets fixed on the Purview side. If you still run into this problem, please open an Issue in this repo and start (re-)registering the three Resource Providers manually through the `Azure Portal`, `Azure Powershell` or `Azure CLI` as mentioned [here](https://docs.microsoft.com/azure/azure-resource-manager/management/resource-providers-and-types). Once that is done, use the "Redeploy" button on the failed deployment to restart the deployment and successfully deploy the components.
This error message appears, when deploying Purview and the `Microsoft.EventHub`, `Microsoft.Storage` and/or `Microsoft.Purview` Resource Provider (RP) is not registered for the subscription or it was registered during early preview phase. If the `Microsoft.Purview` RP is already registered, please un-register and re-register it. We have released a fix to our Deploy To Azure Buttons to overcome this issue until it gets fixed on the Purview side. If you still run into this problem, please open an Issue in this repository and start (re-)registering the three Resource Providers manually through the `Azure Portal`, `Azure Powershell` or `Azure CLI` as mentioned [here](https://docs.microsoft.com/azure/azure-resource-manager/management/resource-providers-and-types). Once that is done, use the "Redeploy" button on the failed deployment to restart the deployment and successfully deploy the components.
>[Previous (Option (a) GitHub Actions)](/docs/EnterpriseScaleAnalytics-GitHubActionsDeployment.md)
>[Previous (Option (b) Azure DevOps)](/docs/EnterpriseScaleAnalytics-AzureDevOpsDeployment.md)
>[Previous (Option (a) GitHub Actions)](/docs/DataManagementAnalytics-GitHubActionsDeployment.md)
>[Previous (Option (b) Azure DevOps)](/docs/DataManagementAnalytics-AzureDevOpsDeployment.md)
Loading

0 comments on commit 4c33814

Please sign in to comment.