diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 52bcc14..a7bd6ad 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -7,15 +7,14 @@ operating environment, schemachange version and python version. Whenever possibl also include a brief, self-contained code example that demonstrates the problem. We have -included [issue templates for reporting bugs, requesting features and seeking clarifications.](https://github.com/Snowflake-Labs/schemachange/issues/new/choose) -Choose the appropriate issue template to contribute to the repository. +included [issue templates](https://github.com/Snowflake-Labs/schemachange/issues/new/choose) for reporting bugs, requesting features and seeking clarifications. Choose the appropriate issue template to contribute to the repository. ## Contributing code -Thanks for your interest in contributing code to schemachange! +Thank you for your interest in contributing code to schemachange! -+ If this is your first time contributing to a project on GitHub, please read through - our [guide to contributing to schemachange](guide-to-contributing-to-schemachange). ++ If this is your first time contributing to a project on GitHub, please continue reading through + [our guide to contributing to schemachange](#guide-to-contributing-to-schemachange). + There are many online tutorials to help you [learn git](https://try.github.io/). For discussions of specific git workflows, see these discussions on [linux git workflow](https://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html), @@ -23,27 +22,29 @@ Thanks for your interest in contributing code to schemachange! ### Guide to contributing to schemachange +> **IMPORTANT** : You will need to follow the [provisioning and schemachange setup instructions](../demo/README.MD) to ensure you can run GitHub actions against your Snowflake account before placing a PR with main schemachange repository so that your PR can be merged into schemachange master branch. + 1. If you are a first-time contributor + Go to [Snowflake-Labs/Schemachange](https://github.com/Snowflake-Labs/schemachange) and click the "fork" button to create your own copy of the project. + [Clone](https://github.com/git-guides/git-clone) the project to your local computer ```shell - git clone https://github.com/your-username/schemachange.git - + # Replace with your Github User Name otherwise + # you will not be able to clone from the fork you created earlier. + git clone https://github.com//schemachange.git ``` + Change the directory ```shell cd schemachange - ``` + + Add upstream repository: ```shell git remote add upstream https://github.com/Snowflake-Labs/schemachange - ``` + Now, `git remote -v` will show two [remote](https://github.com/git-guides/git-remote) repositories named: @@ -54,7 +55,6 @@ Thanks for your interest in contributing code to schemachange! ```shell git checkout main git pull upstream main --tags - ``` 2. Create and Activate a Virtual Environment @@ -68,7 +68,7 @@ Thanks for your interest in contributing code to schemachange! of [this](https://docs.python.org/3/library/venv.html#how-venvs-work) table: | Platform | Shell | Command | - |----------|------------|---------------------------------------| + |----------|------------|---------------------------------------| | POSIX | bash/zsh | `$ source /bin/activate` | | POSIX | fish | `$ source /bin/activate.fish` | | POSIX | csh/tcsh | `$ source /bin/activate.csh` | @@ -76,12 +76,14 @@ Thanks for your interest in contributing code to schemachange! | Windows | cmd.exe | `C:\> \Scripts\activate.bat` | | Windows | PowerShell | `PS C:\> \Scripts\Activate.ps1` | - 3. With your virtual environment activated, upgrade pip + 3. With your virtual environment activated, upgrade `pip` + ```bash python -m pip install --upgrade pip ``` 4. Install the repo as an "editable" package with development dependencies + ```bash pip install -e .[dev] ``` @@ -90,27 +92,30 @@ Thanks for your interest in contributing code to schemachange! + Create a branch for the features you want to work on. Since the branch name will appear in the merge message, use a sensible name such as 'update-build-library-dependencies': - ```shell - git checkout -b update-build-library-dependencies - ``` + ```shell + git checkout -b update-build-library-dependencies + ``` + Commit locally as you progress ( [git add](https://github.com/git-guides/git-add) and [git commit](https://github.com/git-guides/git-commit) ). Use a properly formatted commit message. Be sure to - document any changed behavior. + document any changed behavior in the [CHANGELOG.md](../CHANGELOG.md) file to help us collate the changes for a specific release. + +4. Test your contribution locally -4. Test your contribution ```bash python -m pytest ``` + PS: Please add test cases to the features you are developing so that over time, we can capture any lapse in functionality changes. + +5. Push your contribution to GitHub -5. To submit your contribution - + [Push](https://github.com/git-guides/git-push) your changes back to your fork on GitHub + [Push](https://github.com/git-guides/git-push) your changes back to your fork on GitHub ```shell git push origin update-build-library-dependencies - ``` +6. Raise a Pull Request to merge your contribution into the a Schemachange Release + Go to GitHub. The new branch will show up with a green [Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests#initiating-the-pull-request) button. Make sure the title and message are clear, concise and self-explanatory. Then click the button to submit diff --git a/.github/workflows/dev-pytest.yml b/.github/workflows/dev-pytest.yml index 4cfa9cb..94abf1e 100644 --- a/.github/workflows/dev-pytest.yml +++ b/.github/workflows/dev-pytest.yml @@ -20,7 +20,7 @@ jobs: matrix: scenario-name: [ 'basics_demo', 'citibike_demo', 'citibike_demo_jinja'] os: ["ubuntu-latest"] - python-version: ["3.11"] + python-version: ["3.12"] runs-on: ${{ matrix.os }} if: ${{ github.event.label.name == 'ci-run-tests' || github.event_name == 'push' || github.event_name == 'workflow_dispatch' }} diff --git a/.github/workflows/master-pytest.yml b/.github/workflows/master-pytest.yml index 80ba7aa..5caa505 100644 --- a/.github/workflows/master-pytest.yml +++ b/.github/workflows/master-pytest.yml @@ -25,7 +25,7 @@ jobs: matrix: scenario-name: [ 'basics_demo', 'citibike_demo', 'citibike_demo_jinja'] os: ["ubuntu-latest", "macos-latest", "windows-latest"] - python-version: ["3.8", "3.9", "3.10", "3.11"] + python-version: ["3.9", "3.10", "3.11", "3.12"] runs-on: ${{ matrix.os }} if: ${{ github.event.label.name == 'ci-run-tests' || github.event_name == 'push' || github.event_name == 'workflow_dispatch' }} env: diff --git a/CHANGELOG.md b/CHANGELOG.md index f409859..5d1c933 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,10 +5,13 @@ All notable changes to this project will be documented in this file. ## [4.0.0] - TBD ### Added -- use of structlog for standard log outputs +- Use of `structlog~=24.1.0` for standard log outputs +- Verified Schemachange against Python 3.12 ### Changed -- Refactored the main cli.py to be more modular to aide in future development and testing. +- Refactored the main cli.py into multiple modules - config, session. +- Updated contributing guidelines and demo readme content to help contributors setup local snowflake account to run the github actions in their fork before pushing the PR to upstream repository. +- Removed tests against Python 3.8 [End of Life on 2024-10-07](https://devguide.python.org/versions/#supported-versions) ## [3.7.0] - 2024-07-22 diff --git a/README.md b/README.md index ae9fe7a..93dcd74 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,8 @@ blog post. For the complete list of changes made to schemachange check out the [CHANGELOG](CHANGELOG.md). +To learn more about making a contribution to schemachange, please see our [Contributing guide](.github/CONTRIBUTING.md). + **Please note** that schemachange is a community-developed tool, not an official Snowflake offering. It comes with no support or warranty. @@ -490,7 +492,9 @@ compatibility with versions prior to 3.2. This is the main command that runs the deployment process. -`usage: schemachange deploy [-h] [--config-folder CONFIG_FOLDER] [-f ROOT_FOLDER] [-m MODULES_FOLDER] [-a SNOWFLAKE_ACCOUNT] [-u SNOWFLAKE_USER] [-r SNOWFLAKE_ROLE] [-w SNOWFLAKE_WAREHOUSE] [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG]` +```bash +usage: schemachange deploy [-h] [--config-folder CONFIG_FOLDER] [-f ROOT_FOLDER] [-m MODULES_FOLDER] [-a SNOWFLAKE_ACCOUNT] [-u SNOWFLAKE_USER] [-r SNOWFLAKE_ROLE] [-w SNOWFLAKE_WAREHOUSE] [-d SNOWFLAKE_DATABASE] [-s SNOWFLAKE_SCHEMA] [-c CHANGE_HISTORY_TABLE] [--vars VARS] [--create-change-history-table] [-ac] [-v] [--dry-run] [--query-tag QUERY_TAG] +``` | Parameter | Description | |----------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------| diff --git a/demo/README.MD b/demo/README.MD index 24fc262..a36dadf 100644 --- a/demo/README.MD +++ b/demo/README.MD @@ -13,30 +13,24 @@ not break any existing functionality. - [setup_schemachange_schema.sql](provision/setup_schemachange_schema.sql): Contains the SQL variables to track the individual demo scenarios in its own change history table. -### Contributors - -As a contributor, you will have to set up schemachange demo database and schemachange schema (See Initialize and Setup -scripts below). Along with that you will also set up the following Secrets in your forked repository so that the GitHub -actions can set up, test and teardown the temporary schema it creates to test the changes to your code in the master and -dev branches respectively. - -- SCHEMACHANGE_SNOWFLAKE_PASSWORD -- SCHEMACHANGE_SNOWFLAKE_USER -- SCHEMACHANGE_SNOWFLAKE_ACCOUNT - ### Consumers - If you are consumer who is installing schemachange and wants to test-run the demo, then you will have to set the following environment variables. - - SNOWFLAKE_ACCOUNT: This will be the account identifier for your snowflake account. - - SNOWFLAKE_USER: This will be the user that will connect to you snowflake account. - - SNOWFLAKE_PASSWORD: This is the password for the user (SNOWFLAKE_USER) that will connect to the snowflake account. + - SNOWFLAKE_ACCOUNT: This will be the account identifier for your Snowflake account. + - SNOWFLAKE_USER: This will be the user that will connect to your Snowflake account (`SNOWFLAKE_ACCOUNT`). + - SNOWFLAKE_PASSWORD: This is the password for the user (`SNOWFLAKE_USER`) that will connect to your Snowflake + account (`SNOWFLAKE_ACCOUNT`). - SCENARIO_NAME: This will be demo folder you intend to experiment with. For starters, `basics_demo`, `citibike_demo` or `citibike_demo_jinja` are included with the repo that will set the root folder value in the respective schemachange-config.yml file. - - SNOWFLAKE_WAREHOUSE: This will be the warehouse you set up for the demo. Default setup is SCHEMACHANGE_DEMO_WH - - SNOWFLAKE_DATABASE Keyed to SCHEMACHANGE_DEMO - - SNOWFLAKE_ROLE Keyed to SCHEMACHANGE_DEMO-DEPLOY + - SNOWFLAKE_WAREHOUSE: This will be the warehouse you set up for the demo in your Snowflake + account (`SNOWFLAKE_ACCOUNT`). Default setup is SCHEMACHANGE_DEMO_WH. + - SNOWFLAKE_DATABASE Keyed to SCHEMACHANGE_DEMO. You can update this to the database in your snowflake account + (`SNOWFLAKE_ACCOUNT`). You will also need to update the provision scripts accordingly. + - SNOWFLAKE_ROLE Keyed to SCHEMACHANGE_DEMO-DEPLOY. This particular role name was chosen to test hyphenated roles + in schemachange. You can update this value to match the role you have setup in your snowflake + account (`SNOWFLAKE_ACCOUNT`). The scripts in the `provision` folder can be used to set up up your demo database along with a schema in that database that will house the change tracking tables needed to set up and teardown the schemas used to test a working version of @@ -47,6 +41,17 @@ the demo DDL scripts. - The [setup](provision/setup_schemachange_schema.sql) script creates the `SCHEMACHANGE` schema in the database that you created in the initialize step. +### Contributors + +As a contributor, you will have to set up schemachange demo database and schemachange schema (See Initialize and Setup +scripts below). Along with that you will also set up the following Secrets in your forked repository so that the GitHub +actions can set up, test and teardown the temporary schema it creates to test the changes to your code in the master and +dev branches respectively. + +- SCHEMACHANGE_SNOWFLAKE_PASSWORD +- SCHEMACHANGE_SNOWFLAKE_USER +- SCHEMACHANGE_SNOWFLAKE_ACCOUNT + # Setup The setup scripts are included to build the schema needed by the GitHub Actions Workflow to avoid conflict across jobs