Skip to content

Commit

Permalink
catch table or view not exist error in spark 3.0+ (dbt-labs#331)
Browse files Browse the repository at this point in the history
  • Loading branch information
ChenyuLInx authored Apr 13, 2022
1 parent bb5075e commit fbf9abe
Show file tree
Hide file tree
Showing 6 changed files with 18 additions and 5 deletions.
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.1.0b1
current_version = 1.2.0a1
parse = (?P<major>\d+)
\.(?P<minor>\d+)
\.(?P<patch>\d+)
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ jobs:

runs-on: ubuntu-latest

outputs:
is_alpha: ${{ steps.check-is-alpha.outputs.is_alpha }}

steps:
- name: Check out the repository
uses: actions/checkout@v2
Expand Down Expand Up @@ -150,6 +153,14 @@ jobs:
- name: Check wheel contents
run: |
check-wheel-contents dist/*.whl --ignore W007,W008
- name: Check if this is an alpha version
id: check-is-alpha
run: |
export is_alpha=0
if [[ "$(ls -lh dist/)" == *"a1"* ]]; then export is_alpha=1; fi
echo "::set-output name=is_alpha::$is_alpha"
- uses: actions/upload-artifact@v2
with:
name: dist
Expand All @@ -158,6 +169,8 @@ jobs:
test-build:
name: verify packages / python ${{ matrix.python-version }} / ${{ matrix.os }}

if: needs.build.outputs.is_alpha == 0

needs: build

runs-on: ${{ matrix.os }}
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
- Use dbt.tests.adapter.basic in test suite ([#298](https://github.com/dbt-labs/dbt-spark/issues/298), [#299](https://github.com/dbt-labs/dbt-spark/pull/299))
- Make internal macros use macro dispatch to be overridable in child adapters ([#319](https://github.com/dbt-labs/dbt-spark/issues/319), [#320](https://github.com/dbt-labs/dbt-spark/pull/320))
- Override adapter method 'run_sql_for_tests' ([#323](https://github.com/dbt-labs/dbt-spark/issues/323), [#324](https://github.com/dbt-labs/dbt-spark/pull/324))

- when a table or view doesn't exist, 'adapter.get_columns_in_relation' will return empty list instead of fail ([#328]https://github.com/dbt-labs/dbt-spark/pull/328)

### Contributors
- [@JCZuurmond](https://github.com/dbt-labs/dbt-spark/pull/279) ( [#279](https://github.com/dbt-labs/dbt-spark/pull/279))
Expand Down
2 changes: 1 addition & 1 deletion dbt/adapters/spark/__version__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version = "1.1.0b1"
version = "1.2.0a1"
2 changes: 1 addition & 1 deletion dbt/adapters/spark/impl.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ def get_columns_in_relation(self, relation: Relation) -> List[SparkColumn]:
# CDW would just return and empty list, normalizing the behavior here
errmsg = getattr(e, "msg", "")
if (
f"Table or view not found: {relation}" in errmsg or
"Table or view not found" in errmsg or
"NoSuchTableException" in errmsg
):
pass
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ def _get_dbt_core_version():


package_name = "dbt-spark"
package_version = "1.1.0b1"
package_version = "1.2.0a1"
dbt_core_version = _get_dbt_core_version()
description = """The Apache Spark adapter plugin for dbt"""

Expand Down

0 comments on commit fbf9abe

Please sign in to comment.