Release 5.10.0

### BREAKING CHANGES:
* Dropped support for Python < 3.10.
* Dropped support for MySQL < 8.0.11.
* The minimum supported version of SQLite is increased from 3.21.0 to 3.27.0.
* Backend: Removed all views and forms related to login, registration and password reset.
* Backend: Disabled session auth for swagger ui.
* Frontend: Remove ``SPA_STATIC_FILES_PROVIDERS`` and all code related to serving webpack assets.
* Removed middlewares required for admin site from main ``MIDDLEWARE`` settings.
* Removed default views and classes for manifest and gui. Now uses only FastAPI endpoints.
* Removed useless attribute ``json`` from ``PWAManifest``.
* Removed useless templates.
* Move ``check_request_etag`` to ``vstutils.utils``.
* Minimum node version is set to 20.
* Dropped support of ``Centrifugo`` versions less than 5.
* Remove ``CustomTranslations``.

### Changelog:
* Feature(backend): Migrate to Django 5.0.
* Feature(backend): Migrate to DRF 3.15.
* Feature(backend): Provide support for ``GeneratedField``.
* Feature(backend): Provide default value to schema for named binary fields.
* Feature(backend,frontend): Migrate to OAuth2 authorization.
* Feature(backend): Provide support for json files as arguments to ``run_task``.
* Feature(backend): Provide new ``SimpleFileAction`` for file response operations.
* Feature(backend): Call original registration serializer in email confirmation serializer.
* Feature(backend): Use uvloop if installed for uwsgi workers.
* Feature(backend): Allow to override project of webpush notifications.
* Feature(backend): Render ``manifest.json`` via FastAPI endpoints.
* Feature(backend): Moving admin panel to separated application.
* Feature(backend): Enabled logging for authlib.
* Feature(backend): Allow to provide ``JWT`` access token and id token claims from user instance.
* Feature(backend): Minimal required Pydantic serialization support.
* Feature(backend): Provide async middlewares support.
* Feature(backend): Use ormsgpack for session serializing by default.
* Feature(backend): Use Rust optimization for Vigenère cipher encode/decode.
* Feature(backend): Upgrade to Centrifugo 5.
* Feature(backend): Use async client for centrifugo as default.
* Feature(backend): Added allow bulk argument to nested decorator.
* Feature(backend): Added allow bulk argument to nested decorator (for nested create/add operations).
* Feature(frontend): Send token only on protected routes.
* Feature(frontend): Use userinfo endpoint response instead of loading user.
* Feature(frontend): Cache oauth2 authority and client id.
* Feature(frontend): Create pages for login, registration and password reset.
* Feature(frontend): Add onBefore signal for actions.
* Fix(backend): Exception on ``EmptyResultSet`` for recursive queries.
* Fix(backend): Schema generation when ``APIView`` is used.
* Fix(backend): Add enum values for django filters.
* Fix(backend): Fixed popUp translations.
* Fix(backend): Result serializer ignored if it not Serializer.
* Fix(backend): Remove session_engine health check.
* Fix(backend): Change content disposition as ``inline`` for static files.
* Fix(backend): Dropping sessions on refreshing tokens.
* Fix(backend): Support SQS fifo queues.
* Fix(backend): Cache manifest with etag header.
* Fix(backend): Add filter class for MtM same as Fk.
* Fix(backend): Prevent duplicates when add_in_vary is used.
* Fix(backend): Handle BadSignatureError when validating token.
* Fix(backend): Enable http cache for schema.
* Fix(backend): User settings saving.
* Fix(backend): Issue with integer min and max validators on DecimalField.
* Fix(backend): Provide translations to confirmation email and password reset email.
* Fix(frontend): Send refresh call for OIDC token when it was refused on ``userinfo`` call.
* Fix(frontend): Fixed card word break.
* Fix(frontend): Use ``shallowRef`` for deep fk tree to prevent errors on large trees.
* Fix(frontend): Add missing Vue import.
* Fix(frontend): Fix csv file input buttons type.
* Fix(frontend): Prevent failure when invalid bulk sent.
* Fix(frontend): Respect minItems and maxItems values in array of DeepFkField.
* Fix(frontend): Prevent selection of items with children in DeepFkField when only_last_child is set to True.
* Fix(frontend): Fix fk lookup when target view created using action on parent view.
* Refactoring(backend): Create ``vstutils.exception.NotModifiedException`` for 304 returns.
* Chore(backend): Update ``vstcompile`` to ``3.0``.
* Chore(backend): Change timing name from ``total`` to ``full`` in ``add_server_timing_header`` middleware.
* Chore(backend): Update dependencies, typing and docs translations.
* Chore(backend): Disable session auth for swagger ui.
* Chore(backend): Remove CustomTranslations.
* Chore(backend): Build ``vstutils.tools:get_file_value`` with rust.
* Chore(frontend): Replace ``webpack`` and ``jest`` with ``vite`` and ``vitest``.
* Chore(frontend): Vendor ``JavaScript-autoComplete`` due to import problems in modern builds.
* Chore(frontend): Split frontend by adding some dynamic imports.
* Chore(build): Use manylinux image for build wheels.

See merge request vst/vst-utils!636

.gitignore

@@ -1,5 +1,7 @@
 node_modules
-vstutils/static/bundle
+vstutils/static/spa
+vstutils_utils/target
+vstutils_tools/target
 *.pyc
 *.c
 *.so

.gitlab-ci.yml

@@ -13,7 +13,6 @@ variables:
   RESTORE_CACHE_ATTEMPTS: 3
   DJANGO_LOG_LEVEL: 'CRITICAL'
   TOX_ARGS: "--workdir /cache/.tox_vstutils_${CI_COMMIT_REF_NAME}"
-  CC: "ccache gcc"
   UWSGI_PROFILE: "minimal"
 
 stages:
@@ -30,6 +29,7 @@ stages:
   variables:
     TOX_ENVS: ""
     YARN_CACHE_FOLDER: "${CI_PROJECT_DIR}/yarn"
+    CC: "ccache gcc"
   before_script:
     - >
       if [ "${CI_COMMIT_REF_NAME}" != "master" -a "${CI_COMMIT_REF_NAME}" != "developer" ]; then
@@ -50,7 +50,7 @@ stages:
 
 .js_tests_template: &branch_js_tests
   <<: *branch_tests
-  image: registry.gitlab.com/vstconsulting/images:node18-tests
+  image: registry.gitlab.com/vstconsulting/images:node20-tests
   before_script:
     - yarn install --pure-lockfile --mutex network
   script:
@@ -62,10 +62,8 @@ code_style:
   parallel:
     matrix:
       - TOX_ENVS:
-          - flake
-          - mypy
-          - pylint
-          - bandit
+          - flake,bandit
+          - mypy,pylint
 
 js_style:
   <<: *branch_js_tests
@@ -81,9 +79,9 @@ functional_test:
   parallel:
     matrix:
       - TOX_ENVS:
-          - py38-django42-install
+          - py310-django50-install
       - TOX_ENVS:
-          - py312-django42-coverage
+          - py312-django50-coverage
 
 js_tests:
   <<: *branch_js_tests
@@ -117,6 +115,8 @@ release:
       when: on_success
     - if: '$CI_COMMIT_BRANCH =~ /^release_/ && $GIT_ACCESS_USER && $GIT_ACCESS_PASSWORD'
       when: on_success
+    - if: '$CI_COMMIT_BRANCH =~ /^collect-release-/ && $GIT_ACCESS_USER && $GIT_ACCESS_PASSWORD'
+      when: manual
     - when: never
   before_script:
     - url_host=`echo "${CI_REPOSITORY_URL}" | sed -e "s/https:\/\/gitlab-ci-token:.*@//g"`
@@ -126,28 +126,37 @@ release:
   script:
     - bash autorelease.sh
 
-release_pypi:
+release_packages:
   stage: release
-  image: registry.gitlab.com/vstconsulting/images:ubuntu-v3
+  image: registry.gitlab.com/vstconsulting/images:manylinux_2_17
   artifacts:
     reports:
       dotenv: variables.env
   rules:
-    - if: '$CI_COMMIT_TAG && $PYPI_UPLOAD_PASSWORD && $PYPI_UPLOAD_NAME'
+    - if: '$CI_COMMIT_TAG && $PYPI_UPLOAD_PASSWORD && $PYPI_UPLOAD_NAME && $NPM_TOKEN'
       when: on_success
     - when: never
   script:
+    - echo MILESTONE=${CI_COMMIT_TAG%.*} > variables.env
+    # pypi
     - tox -e build
     - twine upload -u ${PYPI_UPLOAD_NAME} -p ${PYPI_UPLOAD_PASSWORD} $(find dist/* -type f -name '*.tar.gz') $(find dist/* -type f -name '*.whl')
-    - echo MILESTONE=${CI_COMMIT_TAG%.*} > variables.env
+    # npm
+    - |
+      VERSION=$(grep "__version__" vstutils/__init__.py | sed -E "s/__version__: str = '(.+)'/\1/" | sed -E "s/(a|b|rc|post)/-\1/")
+      sed -i -E "s/\"version\": \"[^\"]+\"/\"version\": \"$VERSION\"/" package.json
+    - echo NPM_VERSION=$VERSION >> variables.env
+    - yarn build-lib
+    - cd dist
+    - echo '//registry.npmjs.org/:_authToken=${NPM_TOKEN}'>.npmrc
+    - npm publish --access public
   allow_failure: true
 
-
 release_job:
   stage: post_release
   image: registry.gitlab.com/gitlab-org/release-cli:latest
   needs:
-    - job: release_pypi
+    - job: release_packages
       artifacts: true
   rules:
     - if: '$CI_COMMIT_TAG && $PYPI_UPLOAD_PASSWORD && $PYPI_UPLOAD_NAME'
@@ -167,3 +176,6 @@ release_job:
         - name: 'PyPI'
           url: "https://pypi.org/project/vstutils/${CI_COMMIT_TAG}/"
           link_type: 'other'
+        - name: 'npm'
+          url: "https://www.npmjs.com/package/@vstconsulting/vstutils/v/${NPM_VERSION}"
+          link_type: 'other'

.pylintrc

@@ -13,7 +13,7 @@ ignore=CVS,migrations,unittests,tests,settings.py,settings_production.py
 
 # Add files or directories matching the regex patterns to the blacklist. The
 # regex matches against base names, not paths.
-# ignore-patterns=
+ignore-patterns=.*.pyi
 
 # Pickle collected data for later comparisons.
 # persistent=yes
@@ -56,7 +56,7 @@ confidence=
 # --enable=similarities". If you want to run only the classes checker, but have
 # no Warning level messages displayed, use"--disable=all --enable=classes
 # --disable=W"
-disable=unused-private-member,super-with-arguments,duplicate-code,logging-fstring-interpolation,no-else-continue,unused-argument,signature-differs,no-else-return,consider-using-ternary,inconsistent-return-statements,len-as-condition,keyword-arg-before-vararg,expression-not-assigned,broad-except,logging-format-interpolation,model-no-explicit-unicode,too-many-ancestors,redefined-builtin,missing-docstring,line-too-long,suppressed-message,useless-suppression,model-has-unicode,bare-except,too-few-public-methods,fixme,dangerous-default-value,attribute-defined-outside-init,pointless-string-statement,too-many-instance-attributes,arguments-differ,binary-op-exception,bad-classmethod-argument,locally-disabled,file-ignored,multiple-statements,superfluous-parens,isinstance-second-argument-not-valid-type
+disable=cyclic-import,unused-private-member,super-with-arguments,duplicate-code,logging-fstring-interpolation,no-else-continue,unused-argument,signature-differs,no-else-return,consider-using-ternary,inconsistent-return-statements,len-as-condition,keyword-arg-before-vararg,expression-not-assigned,broad-except,logging-format-interpolation,model-no-explicit-unicode,too-many-ancestors,redefined-builtin,missing-docstring,line-too-long,suppressed-message,useless-suppression,model-has-unicode,bare-except,too-few-public-methods,fixme,dangerous-default-value,attribute-defined-outside-init,pointless-string-statement,too-many-instance-attributes,arguments-differ,binary-op-exception,bad-classmethod-argument,locally-disabled,file-ignored,multiple-statements,superfluous-parens,isinstance-second-argument-not-valid-type
 
 
 [REPORTS]

.run/Full.run.xml

@@ -1,10 +1,12 @@
 <component name="ProjectRunConfigurationManager">
   <configuration default="false" name="Full" type="Tox" factoryName="Tox" singleton="false">
     <module name="vst-utils" />
+    <option name="ENV_FILES" value="" />
     <option name="INTERPRETER_OPTIONS" value="" />
     <option name="PARENT_ENVS" value="true" />
     <envs>
       <env name="DJANGO_LOG_LEVEL" value="DEBUG" />
+      <env name="CC" value="ccache gcc" />
     </envs>
     <option name="SDK_HOME" value="$PROJECT_DIR$/env/bin/python" />
     <option name="SDK_NAME" value="Python 3.8 (vst-utils)" />

doc/backend.rst

@@ -102,7 +102,10 @@ Here is main functionality provided for working with ETag's mechanism:
 
 
 .. automodule:: vstutils.api.base
-    :members: CachableHeadMixin,check_request_etag,get_etag_value,EtagDependency
+    :members: CachableHeadMixin,get_etag_value,EtagDependency
+
+.. automodule:: vstutils.utils
+    :members: check_request_etag
 
 Actions
 ~~~~~~~
@@ -160,7 +163,7 @@ The vstutils library offers a convenient request handler class for elegant OOP d
 Middleware is used to process incoming requests and send responses before they reach final destination.
 
 .. automodule:: vstutils.middleware
-    :members: BaseMiddleware
+    :members: BaseMiddleware,AsyncBaseMiddleware
 
 Filter Backends
 ~~~~~~~~~~~~~~~

doc/conf.py

@@ -256,7 +256,7 @@
 
 intersphinx_mapping = {
     'django': (django_version_docs, django_version_docs + '_objects/'),
-    'python': ('https://docs.python.org/3.8', None),
+    'python': ('https://docs.python.org/3.10', None),
 }
 extlinks = {
     'wiki': ('https://en.wikipedia.org/wiki/%s', None),

doc/config.rst

@@ -365,7 +365,6 @@ Developers often switch between these backends based on the context of their wor
 * **password** - Auth password for smtp-server connection. Default: ``""``.
 * **tls** - Enable/disable tls for smtp-server connection. Default: ``False``.
 * **send_confirmation** - Enable/disable confirmation message after registration. Default: ``False``.
-* **authenticate_after_registration** - Enable/disable autologin after registration confirmation. Default: ``False``.
 
 
 .. _web:
@@ -592,6 +591,33 @@ For more detailed guidance on using and implementing web push notifications in V
 Remember, these settings are crucial for the proper functioning and reliability of web push notifications in your application. Ensure that they are configured accurately for optimal performance.
 
 
+OAuth 2 settings
+----------------
+
+Section ``[oauth]``.
+
+For custom OAuth2 server use the following settings:
+
+* **server_url**: URL of OAuth2 server.
+* **server_token_endpoint_path**: Path of OAuth2 server token endpoint (used in swagger schema). If not provided, `/{API_URL}/oauth2/token/` will be used instead.
+
+For custom OAuth2 server use the following settings:
+
+* **server_enable**: Enable or disable OAuth2 server. Default: `True`.
+* **server_issuer**: Issuer for JWT tokens. Must be provided.
+* **server_jwt_key**: JWT key. Octet sequence (used to represent symmetric keys). Must be provided.
+* **server_jwt_alg**: JWT algorithm. Default: `HS256`.
+* **server_class**: Import path to OAuth2 server class. See `authlib docs <https://docs.authlib.org/en/latest/specs/rfc6749.html#authlib.oauth2.rfc6749.AuthorizationServer>`_ for more information. Default: `vstutils.oauth2.authorization_server.AuthorizationServer`.
+* **server_enable_anon_login**: Enable or disable anonymous login using empty strings as username and password. Default: `False`.
+* **server_jwt_extra_claims_provider**: Import path to function that receives user and returns extra claims for JWT token. Default: `None`.
+* **server_allow_insecure**: If enabled then server will allow HTTP requests. Default: `False`.
+* **server_token_expires_in**: Token expiration time in seconds. Duration values can be used, for example `3d2h32m`. Default: `864000`.
+* **server_client_authentication_methods**: List of client authentication methods. Default server supports following values `'client_secret_basic', 'client_secret_post', 'none'`. Default: `['client_secret_basic', 'client_secret_post']`.
+* **server_authorization_endpoint**: Url of OAuth 2 Authorization endpoint. Will appear in output of `/.well-known/oauth-authorization-server` and `/.well-known/openid-configuration` discovery endpoints.
+
+By default one client can be configured using **server_simple_client_id** and **server_simple_client_secret**. Any other clients can be configured using **OAUTH_SERVER_CLIENTS** dictionary in `settings.py` where key is `client_id` and value is `client_secret`.
+
+
 Production web settings
 -----------------------
 
@@ -845,7 +871,7 @@ This section contains additional information for configure additional elements.
    For project without special settings and project levels named ``project`` these variables will start with ``PROJECT_`` prefix.
    There is a list of these variables: ``{ENV_NAME}_ENABLE_ADMIN_PANEL``, ``{ENV_NAME}_ENABLE_REGISTRATION``, ``{ENV_NAME}_MAX_TFA_ATTEMPTS``,
    ``{ENV_NAME}_ETAG_TIMEOUT``, ``{ENV_NAME}_SEND_CONFIRMATION_EMAIL``, ``{ENV_NAME}_SEND_EMAIL_RETRIES``,
-   ``{ENV_NAME}_SEND_EMAIL_RETRY_DELAY``, ``{ENV_NAME}_AUTHENTICATE_AFTER_REGISTRATION``,
+   ``{ENV_NAME}_SEND_EMAIL_RETRY_DELAY``,
    ``{ENV_NAME}_MEDIA_ROOT`` (dir with uploads), ``{ENV_NAME}_GLOBAL_THROTTLE_RATE``,
    and ``{ENV_NAME}_GLOBAL_THROTTLE_ACTIONS``.