Added support for Databricks Apps in DABs

[andrewnester]

Changes

Now it's possible to configure new app resource in bundle and point it to the custom source_code_path location where Databricks App code is defined.

On databricks bundle deploy DABs will create an app. All consecutive databricks bundle deploy execution will update an existing app if there are any updated

On databricks bundle run <my_app> DABs will execute app deployment. If the app is not started yet, it will start the app first.

Bundle configuration

bundle:
  name: apps

variables:
  my_job_id:
    description: "ID of job to run app"
    lookup:
      job: "My Job"
  databricks_name:
    description: "Name for app user"
  additional_flags:
    description: "Additional flags to run command app"
    default: ""
  my_app_config:
    type: complex
    description: "Configuration for my Databricks App"
    default:
      command:
        - flask
        - --app
        - hello
        - run
        - ${var.additional_flags}
      env:
        - name: DATABRICKS_NAME
          value: ${var.databricks_name}

resources:
  apps:
    my_app:
      name: "anester-app" # required and has to be unique
      description: "My App"
      source_code_path: ./app # required and points to location of app code
      config: ${var.my_app_config}
      resources:
        - name: "my-job"
          description: "A job for app to be able to run"
          job:
            id: ${var.my_job_id}
            permission: "CAN_MANAGE_RUN"
      permissions:
        - user_name: "[email protected]"
          level: "CAN_VIEW"
        - service_principal_name: "my_sp"
          level: "CAN_MANAGE"

targets:
  dev:
    variables:
      databricks_name: "Andrew (from dev)"
      additional_flags: --debug
  
  prod:
    variables:
      databricks_name: "Andrew (from prod)"

Execution

1. databricks bundle deploy -t dev
2. databricks bundle run my_app -t dev

If app is started

✓ Getting the status of the app my-app
✓ App is in RUNNING state
✓ Preparing source code for new app deployment.
✓ Deployment is pending
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>

If app is not started

✓ Getting the status of the app my-app
✓ App is in UNAVAILABLE state
✓ Starting the app my-app
✓ App is starting...
....
✓ App is starting...
✓ App is started!
✓ Preparing source code for new app deployment.
✓ Downloading source code from /Workspace/Users/...
✓ Starting app with command: flask --app hello run --debug
✓ App started successfully
You can access the app at <app-url>

Tests

Added unit and config tests + manual test.

--- PASS: TestAccDeployBundleWithApp (404.59s)
PASS
coverage: 36.8% of statements in ./...
ok      github.com/databricks/cli/internal/bundle       405.035s        coverage: 36.8% of statements in ./...

[andrewnester]

All of the TF changes regenerated based on local TF override with this branch databricks/terraform-provider-databricks#4099

Will be updated when the change is released in TF provider and provider updated in CLI

[eng-dev-ecosystem-bot]

Test Details: go/deco-tests/12276608279

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

• PR number: 1928
• Commit SHA: 33d034a50f066597f12b34ed1a9583837b0fb5a1

Checks will be approved automatically on success.