[FEATURE] Local and Brickflow Metadata Cleanup

[brent-johnson]

A really common problem experienced by developers is when deployments fail because metadata in the local and/or databricks workspace does not match what resources are actually deployed to databricks.

The error message often looks like:
Error: cannot read job: User ff59715d-07ae-465b-9a29-5539c4e7f687 does not have Admin or Manage Run or Owner or View permissions on job 1020683971056165

Often times even under the instruction of support developers that try to resolve this issue manually end up making mistakes. This wastes the time of everyone involved.

Cloud Information

• AWS
• Azure
• GCP
• Other

I'm proposing extending the CLI to include a 'clean' feature that performs this cleanup per project.

The command would look something like:
bf projects clean --project test_project --env qa

Describe alternatives you've considered
I wrote a shell script that performed this same functionality and have tested it to success, but it would be better if it was integrated into the CLI.