Merge branch 'upstream'

README.md

@@ -175,10 +175,10 @@ how to set up a development environment.
 - [Superset SIPs](https://github.com/orgs/apache/projects/170) - The status of Superset's SIPs (Superset Improvement Proposals) for both consensus and implementation status.
 
 Understanding the Superset Points of View
+
 - [The Case for Dataset-Centric Visualization](https://preset.io/blog/dataset-centric-visualization/)
 - [Understanding the Superset Semantic Layer](https://preset.io/blog/understanding-superset-semantic-layer/)
 
-
 - Getting Started with Superset
  - [Superset in 2 Minutes using Docker Compose](https://superset.apache.org/docs/installation/docker-compose#installing-superset-locally-using-docker-compose)
  - [Installing Database Drivers](https://superset.apache.org/docs/configuration/databases#installing-database-drivers)
@@ -196,8 +196,8 @@ Understanding the Superset Points of View
  - [Mixed Time Series Charts](https://preset.io/events/mixed-time-series-visualization-in-superset-workshop/)
  - [How the Bing Team Customized Superset for the Internal Self-Serve Data & Analytics Platform](https://preset.io/events/how-the-bing-team-heavily-customized-superset-for-their-internal-data/)
  - [Live Demo: Visualizing MongoDB and Pinot Data using Trino](https://preset.io/events/2021-04-13-visualizing-mongodb-and-pinot-data-using-trino/)
- - [Introduction to the Superset API](https://preset.io/events/introduction-to-the-superset-api/)
- - [Building a Database Connector for Superset](https://preset.io/events/2021-02-16-building-a-database-connector-for-superset/)
+ - [Introduction to the Superset API](https://preset.io/events/introduction-to-the-superset-api/)
+ - [Building a Database Connector for Superset](https://preset.io/events/2021-02-16-building-a-database-connector-for-superset/)
 
 - Visualizations
  - [Creating Viz Plugins](https://superset.apache.org/docs/contributing/creating-viz-plugins/)
@@ -207,6 +207,7 @@ Understanding the Superset Points of View
 - [Superset API](https://superset.apache.org/docs/rest-api)
 
 ## Repo Activity
+
 <a href="https://next.ossinsight.io/widgets/official/compose-last-28-days-stats?repo_id=39464018" target="_blank" align="center">
  <picture>
  <source media="(prefers-color-scheme: dark)" srcset="https://next.ossinsight.io/widgets/official/compose-last-28-days-stats/thumbnail.png?repo_id=39464018&image_size=auto&color_scheme=dark" width="655" height="auto" />

RELEASING/README.md

@@ -115,7 +115,7 @@ source set_release_env.sh 1.5.1rc1 [email protected]
 
 The script will output the exported variables. Here's example for 1.5.1rc1:
 
-```
+```env
 -------------------------------
 Set Release env variables
 SUPERSET_VERSION=1.5.1
@@ -264,13 +264,13 @@ python changelog.py --previous_version 1.5.0 --current_version ${SUPERSET_GITHUB
 
 Finally, bump the version number on `superset-frontend/package.json` (replace with whichever version is being released excluding the RC version):
 
-```
+```json
 "version": "0.38.0"
 ```
 
 Commit the change with the version number, then git tag the version with the release candidate and push to the branch:
 
-```
+```bash
 # add changed files and commit
 git add ...
 git commit ...
@@ -366,7 +366,7 @@ The script will interactively ask for extra information needed to fill out the e
 voting description, it will generate a passing, non passing or non conclusive email.
 Here's an example:
 
-```
+```text
 A List of people with +1 binding vote (ex: Max,Grace,Krist): Daniel,Alan,Max,Grace
 A List of people with +1 non binding vote (ex: Ville): Ville
 A List of people with -1 vote (ex: John):
@@ -516,16 +516,22 @@ reference), and whether to force the `latest` Docker tag on the
 generated images.
 
 ### Npm Release
+
 You might want to publish the latest @superset-ui release to npm
+
 ```bash
 cd superset/superset-frontend
 ```
+
 An automated GitHub action will run and generate a new tag, which will contain a version number provided as a parameter.
+
 ```bash
 export GH_TOKEN={GITHUB_TOKEN}
 npx lerna version {VERSION} --conventional-commits --create-release github --no-private --yes --message {COMMIT_MESSAGE}
 ```
+
 This action will publish the specified version to npm registry.
+
 ```bash
 npx lerna publish from-package --yes
 ```

docs/docs/configuration/configuring-superset.mdx

@@ -321,7 +321,103 @@ CUSTOM_SECURITY_MANAGER = CustomSsoSecurityManager
  }
  ]
  ```
+### Keycloak-Specific Configuration using Flask-OIDC
+If you are using Keycloak as OpenID Connect 1.0 Provider, the above configuration based on [`Authlib`](https://authlib.org/) might not work. In this case using [`Flask-OIDC`](https://https://pypi.org/project/flask-oidc/) is a viable option.
 
+Make sure the pip package [`Flask-OIDC`](https://https://pypi.org/project/flask-oidc/) is installed on the webserver. This was succesfully tested using version 2.2.0. This package requires [`Flask-OpenID`](https://pypi.org/project/Flask-OpenID/) as a dependency.
+
+The following code defines a new security manager. Add it to a new file named `keycloak_security_manager.py`, placed in the same directory as your `superset_config.py` file.
+```python
+from flask_appbuilder.security.manager import AUTH_OID
+from superset.security import SupersetSecurityManager
+from flask_oidc import OpenIDConnect
+from flask_appbuilder.security.views import AuthOIDView
+from flask_login import login_user
+from urllib.parse import quote
+from flask_appbuilder.views import ModelView, SimpleFormView, expose
+from flask import (
+ redirect,
+ request
+)
+import logging
+
+class OIDCSecurityManager(SupersetSecurityManager):
+
+ def __init__(self, appbuilder):
+ super(OIDCSecurityManager, self).__init__(appbuilder)
+ if self.auth_type == AUTH_OID:
+ self.oid = OpenIDConnect(self.appbuilder.get_app)
+ self.authoidview = AuthOIDCView
+
+class AuthOIDCView(AuthOIDView):
+
+ @expose('/login/', methods=['GET', 'POST'])
+ def login(self, flag=True):
+ sm = self.appbuilder.sm
+ oidc = sm.oid
+
+ @self.appbuilder.sm.oid.require_login
+ def handle_login():
+ user = sm.auth_user_oid(oidc.user_getfield('email'))
+
+ if user is None:
+ info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
+ user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'),
+ info.get('email'), sm.find_role('Gamma'))
+
+ login_user(user, remember=False)
+ return redirect(self.appbuilder.get_url_for_index)
+
+ return handle_login()
+
+ @expose('/logout/', methods=['GET', 'POST'])
+ def logout(self):
+ oidc = self.appbuilder.sm.oid
+
+ oidc.logout()
+ super(AuthOIDCView, self).logout()
+ redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login
+
+ return redirect(
+ oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))
+```
+Then add to your `superset_config.py` file:
+```python
+from keycloak_security_manager import OIDCSecurityManager
+from flask_appbuilder.security.manager import AUTH_OID, AUTH_REMOTE_USER, AUTH_DB, AUTH_LDAP, AUTH_OAUTH
+import os
+
+AUTH_TYPE = AUTH_OID
+SECRET_KEY: 'SomethingNotEntirelySecret'
+OIDC_CLIENT_SECRETS = '/path/to/client_secret.json'
+OIDC_ID_TOKEN_COOKIE_SECURE = False
+OIDC_OPENID_REALM: '<myRealm>'
+OIDC_INTROSPECTION_AUTH_METHOD: 'client_secret_post'
+CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
+
+# Will allow user self registration, allowing to create Flask users from Authorized User
+AUTH_USER_REGISTRATION = True
+
+# The default user self registration role
+AUTH_USER_REGISTRATION_ROLE = 'Public'
+```
+Store your client-specific OpenID information in a file called `client_secret.json`. Create this file in the same directory as `superset_config.py`:
+```json
+{
+ "<myOpenIDProvider>": {
+ "issuer": "https://<myKeycloakDomain>/realms/<myRealm>",
+ "auth_uri": "https://<myKeycloakDomain>/realms/<myRealm>/protocol/openid-connect/auth",
+ "client_id": "https://<myKeycloakDomain>",
+ "client_secret": "<myClientSecret>",
+ "redirect_uris": [
+ "https://<SupersetWebserver>/oauth-authorized/<myOpenIDProvider>"
+ ],
+ "userinfo_uri": "https://<myKeycloakDomain>/realms/<myRealm>/protocol/openid-connect/userinfo",
+ "token_uri": "https://<myKeycloakDomain>/realms/<myRealm>/protocol/openid-connect/token",
+ "token_introspection_uri": "https://<myKeycloakDomain>/realms/<myRealm>/protocol/openid-connect/token/introspect"
+ }
+}
+```
 ## LDAP Authentication
 
 FAB supports authenticating user credentials against an LDAP server.

docs/docs/contributing/howtos.mdx

@@ -434,7 +434,7 @@ To debug Flask running in POD inside a kubernetes cluster, you'll need to make s
  add: ["SYS_PTRACE"]
 ```
 
-See (set capabilities for a container)[https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-capabilities-for-a-container] for more details.
+See [set capabilities for a container](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-capabilities-for-a-container) for more details.
 
 Once the pod is running as root and has the `SYS_PTRACE` capability it will be able to debug the Flask app.
 
@@ -590,6 +590,7 @@ Finally, for the translations to take effect we need to compile translation cata
 binary MO files for the backend using `pybabel`.
 
 ```bash
+# inside the project root
 pybabel compile -d superset/translations
 ```
 

scripts/tests/README.md

@@ -26,48 +26,56 @@ You can use a different DB backend by defining `SUPERSET__SQLALCHEMY_DATABASE_UR
 
 This script will not install any dependencies for you, so you must be on an already set virtualenv
 
-## Use:
+## Usage
 
 To show all supported switches:
-```$bash
+
+```bash
 scripts/tests/run.sh --help
 ```
 
 From the superset repo root directory:
 
 - Example run all tests:
-```$bash
+
+```bash
 scripts/tests/run.sh
 ```
 
 - Example run a single test module:
-```$bash
+
+```bash
 scripts/tests/run.sh --module tests/charts/api_tests.py
 ```
 
 - Example run a single test:
-```$bash
+
+```bash
 scripts/tests/run.sh --module tests/charts/api_tests.py::TestChartApi::test_get_charts
 ```
 
 - Example run a single test, without any init procedures. Init procedures include:
  resetting test database, db upgrade, superset init, loading example data. If your tests
  are idempotent, after the first run, subsequent runs are really fast
-```$bash
+
+```bash
 scripts/tests/run.sh --module tests/charts/api_tests.py::TestChartApi::test_get_charts --no-init
 ```
 
 - Example for not recreating the test DB (will still run all the tests init procedures)
-```$bash
+
+```bash
 scripts/tests/run.sh --module tests/charts/api_tests.py::TestChartApi::test_get_charts --no-reset-db
 ```
 
 - Example for not running tests just initialize the test DB (drop/create, upgrade and load examples)
-```$bash
+
+```bash
 scripts/tests/run.sh --no-tests
 ```
 
 - Example for just resetting the tests DB
-```$bash
+
+```bash
 scripts/tests/run.sh --reset-db --no-tests
 ```

superset-frontend/src/components/ListView/types.ts

@@ -117,7 +117,10 @@ export enum FilterOperator {
  DatasetIsCertified = 'dataset_is_certified',
  DashboardHasCreatedBy = 'dashboard_has_created_by',
  ChartHasCreatedBy = 'chart_has_created_by',
- DashboardTags = 'dashboard_tags',
- ChartTags = 'chart_tags',
- SavedQueryTags = 'saved_query_tags',
+ DashboardTagByName = 'dashboard_tags',
+ DashboardTagById = 'dashboard_tag_id',
+ ChartTagByName = 'chart_tags',
+ ChartTagById = 'chart_tag_id',
+ SavedQueryTagByName = 'saved_query_tags',
+ SavedQueryTagById = 'saved_query_tag_id',
 }

superset-frontend/src/pages/ChartList/index.tsx

@@ -614,7 +614,7 @@ function ChartList(props: ChartListProps) {
  key: 'tags',
  id: 'tags',
  input: 'select',
- operator: FilterOperator.ChartTags,
+ operator: FilterOperator.ChartTagById,
  unfilteredLabel: t('All'),
  fetchSelects: loadTags,
  },

superset-frontend/src/pages/DashboardList/index.tsx

@@ -547,7 +547,7 @@ function DashboardList(props: DashboardListProps) {
  key: 'tags',
  id: 'tags',
  input: 'select',
- operator: FilterOperator.DashboardTags,
+ operator: FilterOperator.DashboardTagById,
  unfilteredLabel: t('All'),
  fetchSelects: loadTags,
  },

superset-frontend/src/pages/SavedQueryList/index.tsx

@@ -501,7 +501,7 @@ function SavedQueryList({
  id: 'tags',
  key: 'tags',
  input: 'select',
- operator: FilterOperator.SavedQueryTags,
+ operator: FilterOperator.SavedQueryTagById,
  fetchSelects: loadTags,
  },
  ]

superset-websocket/README.md

@@ -56,7 +56,8 @@ In addition to periodic socket connection cleanup, the internal _channels_ regis
 ## Install
 
 Install dependencies:
-```
+
+```bash
 npm ci
 ```
 
@@ -71,12 +72,14 @@ Configuration via environment variables is also supported which can be helpful i
 Configure the Superset Flask app to enable global async queries (in `superset_config.py`):
 
 Enable the `GLOBAL_ASYNC_QUERIES` feature flag:
-```
+
+```python
 "GLOBAL_ASYNC_QUERIES": True
 ```
 
 Configure the following Superset values:
-```
+
+```python
 GLOBAL_ASYNC_QUERIES_TRANSPORT = "ws"
 GLOBAL_ASYNC_QUERIES_WEBSOCKET_URL = "ws://<host>:<port>/"
 ```
@@ -86,7 +89,8 @@ Note that the WebSocket server must be run on the same hostname (different port)
 Note also that `localhost` and `127.0.0.1` are not considered the same host. For example, if you're pointing your browser to `localhost:<port>` for Superset, then the WebSocket url will need to be configured as `localhost:<port>`.
 
 The following config values must contain the same values in both the Flask app config and `config.json`:
-```
+
+```text
 GLOBAL_ASYNC_QUERIES_REDIS_CONFIG
 GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX
 GLOBAL_ASYNC_QUERIES_JWT_COOKIE_NAME
@@ -114,26 +118,28 @@ The application is tracking a couple of metrics with `statsd` using the [hot-sho
 ## Running
 
 Running locally via dev server:
-```
+
+```bash
 npm run dev-server
 ```
 
 Running in production:
-```
+
+```bash
 npm run build && npm start
 ```
 
 ## Health check
 
 The WebSocket server supports health checks via one of:
 
-```
+```text
 GET /health
 ```
 
 OR
 
-```
+```text
 HEAD /health
 ```