Add `bundle summary` to display URLs for deployed resources #1731

lennartkats-db · 2024-08-29T10:22:55Z

Changes

Adds a textual output to the databricks bundle summary command, which includes URLs of deployed resources.

Example usage:

$ databricks bundle summary
Name: my_pipeline
Target: dev
Workspace:
  Host: https://domain.databricks.com
  User: [email protected]
  Path: /Users/[email protected]/.bundle/my_pipeline/dev
Resources:
  Jobs:
    my_project_job:
      Name: [dev lennart] my_project_job
      URL:  https://domain.databricks.com/jobs/206899209187287?o=6051921418418893
  Pipelines:
    my_project_pipeline:
      Name: [dev lennart] my_project_pipeline
      URL:  https://domain.databricks.com/pipelines/3f849fd5-ba7d-47fa-a34c-c6bf034b4f58?o=6051921418418893

Notes:

The top headers of the output are the same as those from the existing bundle validate command
URLs are colored light blue in the output
For resources that haven't been deployed yet, we show (not deployed) in place of the URL

bundle/config/resources/job.go

pietern

First pass. Didn't look at the changes to the render package yet, and want to take a closer look as they may impact the output of the validate command.

bundle/config/mutator/initialize_urls.go

lennartkats-db · 2024-10-05T03:47:35Z

@pietern could you take another look? I addressed your initial comments. I was hoping to do a major pass over this in the airplane today; I'd really like to get this change in.

…into cp-summary-with-urls

pietern

There are conflicts on the rendering code, there are more conflicts incoming, and they need to be moved around anyway. I can take care of that. Can you look at the inline comment on the org ID test, then I can do the rest.

bundle/config/mutator/initialize_urls_test.go

bundle/config/mutator/initialize_urls.go

bundle/config/mutator/initialize_urls_test.go

bundle/render/render_text_output.go

bundle/render/render_text_output_test.go

lennartkats-db

@pietern I reviewed the changes in https://github.com/databricks/cli/pull/1731/files/aea4a6e63409deb5f10b9e792aba4a782ea0c585..7658dbdff2872e8b7b503e407ca5b127873e60c8; please take a look

bundle/config/mutator/initialize_urls_test.go

bundle/config/resources.go

Co-authored-by: Lennart Kats (databricks) <[email protected]>

CLI: * Added JSON input validation for CLI commands ([#1771](#1771)). Bundles: * Support Git worktrees for `sync` ([#1831](#1831)). * Add `bundle summary` to display URLs for deployed resources ([#1731](#1731)). * Added a warning when incorrect permissions used for `/Workspace/Shared` bundle root ([#1821](#1821)). * Show actionable errors for collaborative deployment scenarios ([#1386](#1386)). * Fix path to repository-wide exclude file ([#1837](#1837)). * Fixed typo in converting cluster permissions ([#1826](#1826)). * Ignore metastore permission error during template generation ([#1819](#1819)). * Handle normalization of `dyn.KindTime` into an any type ([#1836](#1836)). * Added support for pip options in environment dependencies ([#1842](#1842)). * Fix race condition when restarting continuous jobs ([#1849](#1849)). * Fix pipeline in default-python template not working for certain workspaces ([#1854](#1854)). * Add "output" flag to the bundle sync command ([#1853](#1853)). Internal: * Move utility functions dealing with IAM to libs/iamutil ([#1820](#1820)). * Remove unused `IS_OWNER` constant ([#1823](#1823)). * Assert SDK version is consistent in the CLI generation process ([#1814](#1814)). * Fixed unmarshalling json input into `interface{}` type ([#1832](#1832)). * Fix `TestAccFsMkdirWhenFileExistsAtPath` in isolated Azure environments ([#1833](#1833)). * Add behavioral tests for examples from the YAML spec ([#1835](#1835)). * Remove Terraform conversion function that's no longer used ([#1840](#1840)). * Encode assumptions about the dashboards API in a test ([#1839](#1839)). * Add script to make testing of code on branches easier ([#1844](#1844)). API Changes: * Added `databricks disable-legacy-dbfs` command group. OpenAPI commit cf9c61453990df0f9453670f2fe68e1b128647a2 (2024-10-14) Dependency updates: * Upgrade TF provider to 1.54.0 ([#1852](#1852)). * Bump github.com/databricks/databricks-sdk-go from 0.48.0 to 0.49.0 ([#1843](#1843)).

CLI: * Added JSON input validation for CLI commands ([#1771](#1771)). * Support Git worktrees for `sync` ([#1831](#1831)). Bundles: * Add `bundle summary` to display URLs for deployed resources ([#1731](#1731)). * Added a warning when incorrect permissions used for `/Workspace/Shared` bundle root ([#1821](#1821)). * Show actionable errors for collaborative deployment scenarios ([#1386](#1386)). * Fix path to repository-wide exclude file ([#1837](#1837)). * Fixed typo in converting cluster permissions ([#1826](#1826)). * Ignore metastore permission error during template generation ([#1819](#1819)). * Handle normalization of `dyn.KindTime` into an any type ([#1836](#1836)). * Added support for pip options in environment dependencies ([#1842](#1842)). * Fix race condition when restarting continuous jobs ([#1849](#1849)). * Fix pipeline in default-python template not working for certain workspaces ([#1854](#1854)). * Add "output" flag to the bundle sync command ([#1853](#1853)). Internal: * Move utility functions dealing with IAM to libs/iamutil ([#1820](#1820)). * Remove unused `IS_OWNER` constant ([#1823](#1823)). * Assert SDK version is consistent in the CLI generation process ([#1814](#1814)). * Fixed unmarshalling json input into `interface{}` type ([#1832](#1832)). * Fix `TestAccFsMkdirWhenFileExistsAtPath` in isolated Azure environments ([#1833](#1833)). * Add behavioral tests for examples from the YAML spec ([#1835](#1835)). * Remove Terraform conversion function that's no longer used ([#1840](#1840)). * Encode assumptions about the dashboards API in a test ([#1839](#1839)). * Add script to make testing of code on branches easier ([#1844](#1844)). API Changes: * Added `databricks disable-legacy-dbfs` command group. OpenAPI commit cf9c61453990df0f9453670f2fe68e1b128647a2 (2024-10-14) Dependency updates: * Upgrade TF provider to 1.54.0 ([#1852](#1852)). * Bump github.com/databricks/databricks-sdk-go from 0.48.0 to 0.49.0 ([#1843](#1843)).

## Changes This builds on the functionality added in #1731 that produces a URL for every resource. Adds `bundle/resources` package to deal with resource lookups and command completion. The new functionality is similar to the lookup and command completion functionality located in `bundle/run`. It differs in that it doesn't gracefully deal with ambiguous references to resources, now that we explicitly validate this doesn't occur in the bundle configuration. It still allows resources to be looked up with their fully qualified key, `<plural type>.<key>`. ## Tests * Added unit tests for resource lookup and completion * Manually confirmed that `bundle open` prompts, accepts a key argument, and opens a browser

lennartkats-db added 2 commits August 24, 2024 20:55

Add a textual bundle summary command

869637d

Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls

462ee2e

lennartkats-db commented Aug 29, 2024

View reviewed changes

bundle/config/resources/job.go Show resolved Hide resolved

Fix typo

1325fd8

lennartkats-db marked this pull request as ready for review August 29, 2024 14:46

lennartkats-db requested review from pietern and ilia-db September 5, 2024 08:51

lennartkats-db requested a review from andrewnester September 12, 2024 10:19

Merge branch 'main' into cp-summary-with-urls

44110d1

lennartkats-db requested a review from fjakobs September 20, 2024 12:29

pietern reviewed Sep 23, 2024

View reviewed changes

Address reviewer comments

189d40b

lennartkats-db requested a review from pietern October 5, 2024 03:47

lennartkats-db added 2 commits October 4, 2024 20:48

Merge branch 'cp-summary-with-urls' of github.com:lennartkats-db/cli …

8cd03d1

…into cp-summary-with-urls

Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls

7911c67

lennartkats-db force-pushed the cp-summary-with-urls branch from 233e1be to 785409b Compare October 5, 2024 04:14

Fix test failure based on code from main

65bad56

lennartkats-db force-pushed the cp-summary-with-urls branch from 785409b to 65bad56 Compare October 5, 2024 04:18

pietern reviewed Oct 11, 2024

View reviewed changes

bundle/config/mutator/initialize_urls_test.go Outdated Show resolved Hide resolved

lennartkats-db added 2 commits October 11, 2024 14:43

Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls

ef2400f

Fix test name

d54f641

lennartkats-db enabled auto-merge October 11, 2024 13:43

lennartkats-db requested a review from pietern October 11, 2024 13:43

Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls

6218539

pietern reviewed Oct 14, 2024

View reviewed changes

Styling fix

aea4a6e

lennartkats-db requested a review from pietern October 17, 2024 08:24

lennartkats-db and others added 2 commits October 17, 2024 10:25

Merge remote-tracking branch 'databricks/main' into cp-summary-with-urls

2765a41

Remove unused name field

13049a5

pietern added 6 commits October 17, 2024 15:58

Add singular/plural titles to config.SupportedResources()

2c8bb75

Use plural title resource header

85bc79f

require -> assert

dccd70b

Include test case for clusters

b1dd60c

Always use ID to synthesize URLs

7bf2ec3

Use net/url to build resource URLs such that proper escaping is done

7658dbd

lennartkats-db commented Oct 17, 2024

View reviewed changes

bundle/config/mutator/initialize_urls_test.go Outdated Show resolved Hide resolved

bundle/config/mutator/initialize_urls_test.go Show resolved Hide resolved

bundle/config/resources.go Outdated Show resolved Hide resolved

lennartkats-db and others added 3 commits October 17, 2024 18:46

Merge branch 'main' into cp-summary-with-urls

530a819

Update bundle/config/resources.go

0cd3fcd

Co-authored-by: Lennart Kats (databricks) <[email protected]>

Update bundle/config/mutator/initialize_urls_test.go

c45b058

Co-authored-by: Lennart Kats (databricks) <[email protected]>

pietern changed the title ~~Add a bundle summary command that includes URLs of everything that is deployed~~ Add bundle summary to display URLs for deployed resources Oct 18, 2024

pietern approved these changes Oct 18, 2024

View reviewed changes

lennartkats-db added this pull request to the merge queue Oct 18, 2024

Merged via the queue into databricks:main with commit c5043c3 Oct 18, 2024
5 checks passed

lennartkats-db deleted the cp-summary-with-urls branch October 18, 2024 06:53

andrewnester mentioned this pull request Oct 23, 2024

[Release] Release v0.231.0 #1856

Merged

pietern mentioned this pull request Oct 23, 2024

Add command to open a resource in the browser #1846

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `bundle summary` to display URLs for deployed resources #1731

Add `bundle summary` to display URLs for deployed resources #1731

lennartkats-db commented Aug 29, 2024 •

edited by pietern

Loading

pietern left a comment

lennartkats-db commented Oct 5, 2024

pietern left a comment

lennartkats-db left a comment

Add bundle summary to display URLs for deployed resources #1731

Add bundle summary to display URLs for deployed resources #1731

Conversation

lennartkats-db commented Aug 29, 2024 • edited by pietern Loading

Changes

pietern left a comment

Choose a reason for hiding this comment

lennartkats-db commented Oct 5, 2024

pietern left a comment

Choose a reason for hiding this comment

lennartkats-db left a comment

Choose a reason for hiding this comment

Add `bundle summary` to display URLs for deployed resources #1731

Add `bundle summary` to display URLs for deployed resources #1731

lennartkats-db commented Aug 29, 2024 •

edited by pietern

Loading