Skip to content

Commit

Permalink
Merge pull request #36 from Azure-Samples/t2-datalakestore-2020-10-26
Browse files Browse the repository at this point in the history
[T2] datalakestore
  • Loading branch information
00Kai0 authored Nov 19, 2020
2 parents 360d711 + 336ee1c commit 2907f13
Show file tree
Hide file tree
Showing 3 changed files with 199 additions and 0 deletions.
87 changes: 87 additions & 0 deletions samples/datalakestore/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
page_type: sample
languages:
- python
products:
- azure
description: "These code samples will show you how to manage datalakestore using Azure SDK for Python."
urlFragment: datalakestore
---

# Getting started - Managing datalakestore using Azure Python SDK

These code samples will show you how to manage datalakestore using Azure SDK for Python.

## Features

This project framework provides examples for the following services:

### datalakestore
* [] Using the Azure SDK for Python - datalakestore Management Library [azure-mgmt-datalake-store](https://pypi.org/project/azure-mgmt-datalake-store/) for the [datalakestore API](https://docs.microsoft.com/en-us/rest/api/datalakestore/)

## Getting Started

### Prerequisites

1. Before we run the samples, we need to make sure we have setup the credentials. Follow the instructions in [register a new application using Azure portal](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal) to obtain `subscription id`,`client id`,`client secret`, and `application id`

2. Store your credentials an environment variables.
For example, in Linux-based OS, you can do
```bash
export AZURE_TENANT_ID="xxx"
export AZURE_CLIENT_ID="xxx"
export AZURE_CLIENT_SECRET="xxx"
export SUBSCRIPTION_ID="xxx"
```

### Installation

1. If you don't already have it, [install Python](https://www.python.org/downloads/).

This sample (and the SDK) is compatible with Python 2.7, 3.3, 3.4, 3.5 and 3.6.

2. General recommendation for Python development is to use a Virtual Environment.
For more information, see https://docs.python.org/3/tutorial/venv.html

Install and initialize the virtual environment with the "venv" module on Python 3 (you must install [virtualenv](https://pypi.python.org/pypi/virtualenv) for Python 2.7):

```
python -m venv mytestenv # Might be "python3" or "py -3.6" depending on your Python installation
cd mytestenv
source bin/activate # Linux shell (Bash, ZSH, etc.) only
./scripts/activate # PowerShell only
./scripts/activate.bat # Windows CMD only
```

### Quickstart

1. Clone the repository.

```
git clone https://github.com/Azure-Samples/azure-samples-python-management.git
```

2. Install the dependencies using pip.

```
cd azure-samples-python-management/samples/datalakestore
pip install -r requirements.txt
```

## Demo

A demo app is included to show how to use the project.

To run the complete demo, execute `python example.py`

To run each individual demo, point directly to the file. For example (i.e. not complete list):

1. `python manage_datalakestore.py`

If the script starts with `disable_***.py`, it means that it is unavailable now.

The sample files do not have dependency each other and each file represents an individual end-to-end scenario. Please look at the sample that contains the scenario you are interested in

## Resources

- https://github.com/Azure/azure-sdk-for-python
109 changes: 109 additions & 0 deletions samples/datalakestore/manage_datalakestore.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# --------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------

import os

from azure.identity import DefaultAzureCredential
from azure.mgmt.datalake.store import DataLakeStoreAccountManagementClient
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.datalake.store import models

# - other dependence -
# - end -


def main():

SUBSCRIPTION_ID = os.environ.get("SUBSCRIPTION_ID", None)
GROUP_NAME = "testdatalakestore1"
LOCATION = 'eastus2'

# Create client
# # For other authentication approaches, please see: https://pypi.org/project/azure-identity/
resource_client = ResourceManagementClient(
credential=DefaultAzureCredential(),
subscription_id=SUBSCRIPTION_ID
)
datalakestore_client = DataLakeStoreAccountManagementClient(
credential=DefaultAzureCredential(),
subscription_id=SUBSCRIPTION_ID
)
# - init depended client -
# - end -

# Create resource group
resource_client.resource_groups.create_or_update(
GROUP_NAME,
{"location": LOCATION}
)

# - init depended resources -
# - end -

# Create datalakestore
# define account params
ACCOUNT_NAME = 'testaccount'

params_create = models.CreateDataLakeStoreAccountParameters(
location=LOCATION,
identity=models.EncryptionIdentity(),
encryption_config=models.EncryptionConfig(
type=models.EncryptionConfigType.service_managed
),
encryption_state=models.EncryptionState.enabled,
tags={
'tag1': 'value1'
}
)

# params_create_no_encryption = models.CreateDataLakeStoreAccountParameters(
# location=location,
# tags={
# 'tag1': 'value1'
# }
# )

datalakestore = datalakestore_client.accounts.begin_create(
GROUP_NAME,
ACCOUNT_NAME,
params_create
).result()
print("Create datalakestore:\n{}".format(datalakestore))

# Get datalakestore
datalakestore = datalakestore_client.accounts.get(
GROUP_NAME,
ACCOUNT_NAME
)
print("Get datalakestore:\n{}".format(datalakestore))

# Update datalakestore
datalakestore = datalakestore_client.accounts.begin_update(
GROUP_NAME,
ACCOUNT_NAME,
models.UpdateDataLakeStoreAccountParameters(
tags={
'tag2': 'value2'
}
)
).result()
print("Update datalakestore:\n{}".format(datalakestore))

# Delete datalakestore
datalakestore = datalakestore_client.accounts.begin_delete(
GROUP_NAME,
ACCOUNT_NAME
).result()
print("Delete datalakestore.\n")

# Delete Group
resource_client.resource_groups.begin_delete(
GROUP_NAME
).result()


if __name__ == "__main__":
main()
3 changes: 3 additions & 0 deletions samples/datalakestore/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
azure-identity
azure-mgmt-resource==15.0.0
azure-mgmt-datalake-store==1.0.0b1

0 comments on commit 2907f13

Please sign in to comment.