Added `auth describe` command #1244

andrewnester · 2024-02-29T13:07:22Z

Changes

This command provide details on auth configuration user is using as well as authenticated user and auth mechanism used.

Relies on databricks/databricks-sdk-go#838 (tests will fail until merged)

Examples of output

Workspace: https://test.com
User: [email protected]
Authenticated with: pat
-----
Configuration:
  ✓ auth_type: pat
  ✓ host: https://test.com (from bundle)
  ✓ profile: DEFAULT (from --profile flag)
  ✓ token: ******** (from /Users/andrew.nester/.databrickscfg config file)

DATABRICKS_AUTH_TYPE=azure-msi databricks auth describe -p "Azure 2"
Unable to authenticate: inner token: Post "https://foobar.com/oauth2/token": AADSTS900023: Specified tenant identifier foobar_aaaaaaa' is neither a valid DNS name, nor a valid external domain. See https://login.microsoftonline.com/error?code=900023
-----
Configuration:
  ✓ auth_type: azure-msi (from DATABRICKS_AUTH_TYPE environment variable)
  ✓ azure_client_id: 8470f3ba-aaaa-bbbb-cccc-xxxxyyyyzzzz (from /Users/andrew.nester/.databrickscfg config file)
  ~ azure_client_secret: ******** (from /Users/andrew.nester/.databrickscfg config file, not used for auth type azure-msi)
  ~ azure_tenant_id: foobar_aaaaaaa (from /Users/andrew.nester/.databrickscfg config file, not used for auth type azure-msi)
  ✓ azure_use_msi: true (from /Users/andrew.nester/.databrickscfg config file)
  ✓ host: https://foobar.com (from /Users/andrew.nester/.databrickscfg config file)
  ✓ profile: Azure 2 (from --profile flag)

For account

Unable to authenticate: default auth: databricks-cli: cannot get access token: Error: token refresh: Post "https://xxxxxxx.com/v1/token": http 400: {"error":"invalid_request","error_description":"Refresh token is invalid"}
. Config: host=https://xxxxxxx.com, account_id=ed0ca3c5-fae5-4619-bb38-eebe04a4af4b, profile=ACCOUNT-ed0ca3c5-fae5-4619-bb38-eebe04a4af4b
-----
Configuration:
  ✓ account_id: ed0ca3c5-fae5-4619-bb38-eebe04a4af4b (from /Users/andrew.nester/.databrickscfg config file)
  ✓ auth_type: databricks-cli (from /Users/andrew.nester/.databrickscfg config file)
  ✓ host: https://xxxxxxxxx.com (from /Users/andrew.nester/.databrickscfg config file)
  ✓ profile: ACCOUNT-ed0ca3c5-fae5-4619-bb38-eebe04a4af4b

Tests

Added unit tests

pietern

Very nice!

Could you see if the auth detail synthesis can be pulled into a library? I'd like to see if we can include information like this as part of authentication error messages.

It happens every now and then that folks make an incorrect assumption on how they have setup auth and then beat their heads over it not working. If we include (some of) this information in those errors, those incorrect assumptions would surface immediately.

cmd/auth/describe.go

bundle/config/workspace.go

cmd/auth/describe.go

cmd/root/auth.go

## Changes Added Config.GetAuthDetails and use it in config debug string Needed for databricks/cli#1244 ## Tests - [x] `make test` passing - [x] `make fmt` applied - [x] relevant integration tests applied

pietern

The output for me includes:

Configuration:
  ✓ auth_type: pat (from dynamic configuration)
  ✓ databricks_cli_path: ./.databricks/databricks (from DATABRICKS_CLI_PATH environment variable)

The auth_type is resolved by authenticating and duplicates the "Authenticated with" field line at the top of the output. It would be good to mask it out if it is not explicitly set (through env var or as part of the databrickscfg).

The CLI path is set by the CLI itself. Would be good to mask out as well if it is the same as the value set by the CLI itself.

libs/databrickscfg/loader.go

pietern · 2024-03-13T13:21:21Z

bundle/config/workspace.go

+	for k := range config.ConfigAttributes {
+		attr := &config.ConfigAttributes[k]
+		if !attr.IsZero(cfg) {
+			cfg.SetAttrSource(attr, config.Source{Type: config.SourceType("bundle")})


We could even include the file/line where the attr was defined.

bundle/config/workspace.go

cmd/auth/describe.go

juliacrawf-db · 2024-03-26T03:06:32Z

cmd/auth/describe.go

+
+var authTemplate = `{{"Workspace:" | bold}} {{.Details.Host}}
+{{if .Username }}{{"User:" | bold}} {{.Username}}{{end}}
+{{"Authenticated with:" | bold}} {{.Details.AuthType}}


As a new user, it took me a hot second to figure out what 'pat' was in the example output, and I also wondered what other types there were, and had to do a bit of searching. Low priority, but just a thought - could we output a friendlier string instead of just the exact field value?

If we do, we should do both (can be a follow-up to this, IMO). The literal value is what someone would use when hardcoding it in a profile or in the DATABRICKS_AUTH_TYPE environment variable.

cmd/auth/describe.go

juliacrawf-db · 2024-03-26T04:15:11Z

I think this is great.

For the success output, it seems like some of the info is duplicated in the two sections of the output (Workspace=host, Authenticated with=auth_type) but because the words are different ...will they always be the same? If so, is the info needed in both sections?

cmd/auth/describe.go

Co-authored-by: Julia Crawford (Databricks) <[email protected]>

cmd/auth/describe.go

pietern · 2024-03-26T12:01:16Z

cmd/auth/describe.go

+		if err != nil {
+			return err
+		}
+		cmd.OutOrStdout().Write(buf)


I noticed the "Source" struct doesn't include JSON tags in its definition, so it uses the uppercased field names in the JSON output. Should be fixed in a follow up SDK change.

Or it could be fixed here with a type copy.

@pietern good catch, I'll do the follow up

pietern · 2024-03-26T12:56:26Z

cmd/auth/describe.go

+			profile = "default"
+		}
+		details.Configuration["profile"] = &config.AttrConfig{Value: profile, Source: config.Source{Type: config.SourceDynamicConfig}}
+	}


Could we also change the source for DATABRICKS_CLI_PATH here?

Now every run outputs:

✓ databricks_cli_path: ./cli (from DATABRICKS_CLI_PATH environment variable)

And shouldn't it show up as ~ btw if the auth type is not OAuth btw?

And shouldn't it show up as ~ btw if the auth type is not OAuth btw?

yeah, we would need to mark it with correct auth_types in Go SDK. Since it's a minor thing, we'll do the follow up on this

cmd/root/auth.go

cmd/auth/describe.go

cmd/root/auth.go

CLI: * Added `auth describe` command ([#1244](#1244)). * Fixed message for successful auth describe run ([#1336](#1336)). Bundles: * Make bundle validation print text output by default ([#1335](#1335)). * Use UserName field to identify if service principal is used ([#1310](#1310)). * Allow unknown properties in the config file for template initialization ([#1315](#1315)). * Remove support for DATABRICKS_BUNDLE_INCLUDES ([#1317](#1317)). * Make `bundle.deployment` optional in the bundle schema ([#1321](#1321)). * Add allow list for resources when bundle `run_as` is set ([#1233](#1233)). * Fix the generated DABs JSON schema ([#1322](#1322)). * Make bundle loaders return diagnostics ([#1319](#1319)). * Add `bundle debug terraform` command ([#1294](#1294)). * Allow specifying CLI version constraints required to run the bundle ([#1320](#1320)). Internal: * Retain location information of variable reference ([#1333](#1333)). * Define `dyn.Mapping` to represent maps ([#1301](#1301)). * Return `diag.Diagnostics` from mutators ([#1305](#1305)). * Fix flaky test in `libs/process` ([#1314](#1314)). * Move path field to bundle type ([#1316](#1316)). * Load bundle configuration from mutator ([#1318](#1318)). * Return diagnostics from `config.Load` ([#1324](#1324)). * Return warning for nil primitive types during normalization ([#1329](#1329)). * Include `dyn.Path` in normalization warnings and errors ([#1332](#1332)). * Make normalization return warnings instead of errors ([#1334](#1334)). API Changes: * Added `databricks lakeview migrate` command. * Added `databricks lakeview unpublish` command. * Changed `databricks ip-access-lists get` command . New request type is . OpenAPI commit e316cc3d78d087522a74650e26586088da9ac8cb (2024-04-03) Dependency updates: * Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0 ([#1326](#1326)).

Breaking Change: * Add allow list for resources when bundle `run_as` is set ([#1233](#1233)). * Make bundle validation print text output by default ([#1335](#1335)). CLI: * Added `auth describe` command ([#1244](#1244)). * Fixed message for successful auth describe run ([#1336](#1336)). Bundles: * Use UserName field to identify if service principal is used ([#1310](#1310)). * Allow unknown properties in the config file for template initialization ([#1315](#1315)). * Remove support for DATABRICKS_BUNDLE_INCLUDES ([#1317](#1317)). * Make `bundle.deployment` optional in the bundle schema ([#1321](#1321)). * Fix the generated DABs JSON schema ([#1322](#1322)). * Make bundle loaders return diagnostics ([#1319](#1319)). * Add `bundle debug terraform` command ([#1294](#1294)). * Allow specifying CLI version constraints required to run the bundle ([#1320](#1320)). Internal: * Retain location information of variable reference ([#1333](#1333)). * Define `dyn.Mapping` to represent maps ([#1301](#1301)). * Return `diag.Diagnostics` from mutators ([#1305](#1305)). * Fix flaky test in `libs/process` ([#1314](#1314)). * Move path field to bundle type ([#1316](#1316)). * Load bundle configuration from mutator ([#1318](#1318)). * Return diagnostics from `config.Load` ([#1324](#1324)). * Return warning for nil primitive types during normalization ([#1329](#1329)). * Include `dyn.Path` in normalization warnings and errors ([#1332](#1332)). * Make normalization return warnings instead of errors ([#1334](#1334)). API Changes: * Added `databricks lakeview migrate` command. * Added `databricks lakeview unpublish` command. * Changed `databricks ip-access-lists get` command . New request type is . OpenAPI commit e316cc3d78d087522a74650e26586088da9ac8cb (2024-04-03) Dependency updates: * Bump github.com/databricks/databricks-sdk-go from 0.36.0 to 0.37.0 ([#1326](#1326)).

andrewnester requested review from pietern and shreyas-goenka February 29, 2024 13:07

andrewnester added the do-not-merge label Feb 29, 2024

andrewnester requested review from ilia-db and kartikgupta-db March 1, 2024 10:10

pietern reviewed Mar 4, 2024

View reviewed changes

cmd/auth/describe.go Outdated Show resolved Hide resolved

bundle/config/workspace.go Show resolved Hide resolved

cmd/auth/describe.go Show resolved Hide resolved

cmd/root/auth.go Show resolved Hide resolved

cmd/root/auth.go Show resolved Hide resolved

andrewnester mentioned this pull request Mar 4, 2024

Added Config.GetAuthDetails databricks/databricks-sdk-go#838

Merged

3 tasks

andrewnester force-pushed the feature/auth-describe branch from 3301b27 to ef16d3c Compare March 4, 2024 14:46

andrewnester requested a review from pietern March 4, 2024 14:50

andrewnester removed the do-not-merge label Mar 8, 2024

pietern reviewed Mar 13, 2024

View reviewed changes

ilia-db mentioned this pull request Mar 18, 2024

Don't attempt auth in auth profiles --skip-validate #1282

Merged

andrewnester added 5 commits March 25, 2024 14:51

Added auth describe command

01d4926

fix

8092795

use updated sdk

1cd5db3

fixed tests

d560094

fixes

c56cb6e

andrewnester force-pushed the feature/auth-describe branch from 8e91948 to c56cb6e Compare March 25, 2024 14:35

no trailing dots

42e9374

andrewnester requested a review from pietern March 25, 2024 14:39

juliacrawf-db reviewed Mar 26, 2024

View reviewed changes

cmd/auth/describe.go Outdated Show resolved Hide resolved

juliacrawf-db reviewed Mar 26, 2024

View reviewed changes

cmd/auth/describe.go Outdated Show resolved Hide resolved

Update cmd/auth/describe.go

755b683

Co-authored-by: Julia Crawford (Databricks) <[email protected]>

pietern reviewed Mar 26, 2024

View reviewed changes

andrewnester added 2 commits March 28, 2024 13:01

Merge branch 'main' into feature/auth-describe

80a2408

remove --account flag

8af0a4e

andrewnester requested a review from pietern March 28, 2024 13:32

pietern reviewed Apr 2, 2024

View reviewed changes

fixes

566bc90

andrewnester requested a review from pietern April 2, 2024 14:45

fixed comment

6413e24

andrewnester enabled auto-merge April 2, 2024 14:50

pietern approved these changes Apr 2, 2024

View reviewed changes

fix test

edcc7e5

andrewnester added this pull request to the merge queue Apr 3, 2024

Merged via the queue into main with commit 8c144a2 Apr 3, 2024
5 checks passed

andrewnester deleted the feature/auth-describe branch April 3, 2024 08:20

andrewnester mentioned this pull request Apr 3, 2024

Release v0.217.0 #1337

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added `auth describe` command #1244

Added `auth describe` command #1244

andrewnester commented Feb 29, 2024 •

edited

Loading

pietern left a comment

pietern left a comment

pietern Mar 13, 2024

juliacrawf-db Mar 26, 2024

pietern Mar 26, 2024

juliacrawf-db commented Mar 26, 2024 •

edited

Loading

pietern Mar 26, 2024

pietern Mar 26, 2024

andrewnester Mar 28, 2024 •

edited

Loading

pietern Mar 26, 2024

pietern Mar 26, 2024

andrewnester Mar 28, 2024

Added auth describe command #1244

Added auth describe command #1244

Conversation

andrewnester commented Feb 29, 2024 • edited Loading

Changes

Tests

pietern left a comment

Choose a reason for hiding this comment

pietern left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

juliacrawf-db commented Mar 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewnester Mar 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Added `auth describe` command #1244

Added `auth describe` command #1244

andrewnester commented Feb 29, 2024 •

edited

Loading

juliacrawf-db commented Mar 26, 2024 •

edited

Loading

andrewnester Mar 28, 2024 •

edited

Loading