Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add slots to parser #375

Merged
merged 58 commits into from
Aug 27, 2024
Merged

feat: Add slots to parser #375

merged 58 commits into from
Aug 27, 2024

Conversation

Ramimashkouk
Copy link
Member

Description

The master was merged into script-parser, and a slot parsing functionality was added.

Checklist

  • I have performed a self-review of the changes

List here tasks to complete in order to mark this PR as ready for review.

To Consider

  • Add tests (if functionality is changed)
  • Update API reference / tutorials / guides
  • Update CONTRIBUTING.md (if devel workflow is changed)
  • Update .ignore files, scripts (such as lint), distribution manifest (if files are added/deleted)
  • Search for references to changed entities in the codebase

RLKRo and others added 30 commits December 27, 2023 20:28
For some reason if you replace `branch=dev` with `branch=master` it doesn't work, so I had to set `dev` as target branch.
And without this param it shows the status of the latest workflow run on any branch.

(cherry picked from commit 1ca2df0)
* Add into  README.md info about target platform

* restructure markdown

---------

Co-authored-by: Roman Zlobin <[email protected]>
(cherry picked from commit 7652451)
* replace multimessage with misc messages

* format
* build(deps): bump the deps group with 16 updates

Bumps the deps group with 16 updates:

| Package | From | To |
| --- | --- | --- |
| [pydantic](https://github.com/pydantic/pydantic) | `2.5.3` | `2.6.0` |
| [nest-asyncio](https://github.com/erdewit/nest_asyncio) | `1.5.8` | `1.6.0` |
| [ydb](https://github.com/ydb-platform/ydb-python-sdk) | `3.7.0` | `3.8.0` |
| [cryptography](https://github.com/pyca/cryptography) | `41.0.7` | `42.0.2` |
| [pytelegrambotapi](https://github.com/eternnoir/pyTelegramBotAPI) | `4.14.1` | `4.15.4` |
| [flake8](https://github.com/pycqa/flake8) | `6.1.0` | `7.0.0` |
| [black](https://github.com/psf/black) | `23.12.1` | `24.1.1` |
| [pytest](https://github.com/pytest-dev/pytest) | `7.4.3` | `8.0.0` |
| [coverage](https://github.com/nedbat/coveragepy) | `7.3.2` | `7.4.1` |
| [jsonschema](https://github.com/python-jsonschema/jsonschema) | `4.20.0` | `4.21.1` |
| [telethon](https://github.com/LonamiWebs/Telethon) | `1.33.1` | `1.34.0` |
| [fastapi](https://github.com/tiangolo/fastapi) | `0.107.0` | `0.109.2` |
| [uvicorn](https://github.com/encode/uvicorn) | `0.25.0` | `0.27.0.post1` |
| [locust](https://github.com/locustio/locust) | `2.20.0` | `2.22.0` |
| [streamlit](https://github.com/streamlit/streamlit) | `1.29.0` | `1.31.0` |
| [sphinxcontrib-apidoc](https://github.com/sphinx-contrib/apidoc) | `0.4.0` | `0.5.0` |


Updates `pydantic` from 2.5.3 to 2.6.0
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](pydantic/pydantic@v2.5.3...v2.6.0)

Updates `nest-asyncio` from 1.5.8 to 1.6.0
- [Release notes](https://github.com/erdewit/nest_asyncio/releases)
- [Commits](erdewit/nest_asyncio@v1.5.8...v1.6.0)

Updates `ydb` from 3.7.0 to 3.8.0
- [Release notes](https://github.com/ydb-platform/ydb-python-sdk/releases)
- [Changelog](https://github.com/ydb-platform/ydb-python-sdk/blob/main/CHANGELOG.md)
- [Commits](ydb-platform/ydb-python-sdk@3.7.0...3.8.0)

Updates `cryptography` from 41.0.7 to 42.0.2
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](pyca/cryptography@41.0.7...42.0.2)

Updates `pytelegrambotapi` from 4.14.1 to 4.15.4
- [Release notes](https://github.com/eternnoir/pyTelegramBotAPI/releases)
- [Commits](https://github.com/eternnoir/pyTelegramBotAPI/commits)

Updates `flake8` from 6.1.0 to 7.0.0
- [Commits](PyCQA/flake8@6.1.0...7.0.0)

Updates `black` from 23.12.1 to 24.1.1
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.12.1...24.1.1)

Updates `pytest` from 7.4.3 to 8.0.0
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@7.4.3...8.0.0)

Updates `coverage` from 7.3.2 to 7.4.1
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](nedbat/coveragepy@7.3.2...7.4.1)

Updates `jsonschema` from 4.20.0 to 4.21.1
- [Release notes](https://github.com/python-jsonschema/jsonschema/releases)
- [Changelog](https://github.com/python-jsonschema/jsonschema/blob/main/CHANGELOG.rst)
- [Commits](python-jsonschema/jsonschema@v4.20.0...v4.21.1)

Updates `telethon` from 1.33.1 to 1.34.0
- [Release notes](https://github.com/LonamiWebs/Telethon/releases)
- [Commits](LonamiWebs/Telethon@v1.33.1...v1.34.0)

Updates `fastapi` from 0.107.0 to 0.109.2
- [Release notes](https://github.com/tiangolo/fastapi/releases)
- [Commits](fastapi/fastapi@0.107.0...0.109.2)

Updates `uvicorn` from 0.25.0 to 0.27.0.post1
- [Release notes](https://github.com/encode/uvicorn/releases)
- [Changelog](https://github.com/encode/uvicorn/blob/master/CHANGELOG.md)
- [Commits](encode/uvicorn@0.25.0...0.27.0.post1)

Updates `locust` from 2.20.0 to 2.22.0
- [Release notes](https://github.com/locustio/locust/releases)
- [Changelog](https://github.com/locustio/locust/blob/master/CHANGELOG.md)
- [Commits](locustio/locust@2.20.0...2.22.0)

Updates `streamlit` from 1.29.0 to 1.31.0
- [Release notes](https://github.com/streamlit/streamlit/releases)
- [Commits](streamlit/streamlit@1.29.0...1.31.0)

Updates `sphinxcontrib-apidoc` from 0.4.0 to 0.5.0
- [Commits](sphinx-contrib/apidoc@0.4.0...0.5.0)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: nest-asyncio
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: ydb
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: cryptography
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: deps
- dependency-name: pytelegrambotapi
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: flake8
  dependency-type: direct:development
  update-type: version-update:semver-major
  dependency-group: deps
- dependency-name: black
  dependency-type: direct:development
  update-type: version-update:semver-major
  dependency-group: deps
- dependency-name: pytest
  dependency-type: direct:development
  update-type: version-update:semver-major
  dependency-group: deps
- dependency-name: coverage
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: jsonschema
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: telethon
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: fastapi
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: uvicorn
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: locust
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: streamlit
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
- dependency-name: sphinxcontrib-apidoc
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: deps
...

Signed-off-by: dependabot[bot] <[email protected]>

* update coverage restriction

* update lock file

* lint

* update poetry lock

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Roman Zlobin <[email protected]>
Removing validation by default caused seeded responses to change
This is done as part of these function's deprecation process.
* Revert "disable validation by default"

This reverts commit 2b5915a.

* remove function calls from validate_script

* lint

* lower coverage to 90
# Description

- Function has_text() added. Checks the 'text' field of last_request(),
if specified text within 'text' field, returns true.
For such a task it's quicker than regexp condition and has less syntax
bloat than exact_match() would have.
This also makes checking telegram callback data easier since it is saved
in the 'text' field: has_text(callback_data).
 - All relevant API reference / tutorials / guides updated.
By that I mean all 'exact_match(Message("Some message"))' was changed
into 'has_text("Some message")'.
 - All relevant references in the codebase updated.
- Tests added, but I'm not sure I did it right. I changed some
exact_match() uses to has_text() there. Well, vim did it, and now I'm
thinking it's maybe preferable, actually.
# Checklist

- [x] I have performed a self-review of the changes
- [x] Check if tests are done right (poetry has no issues, I mean if
exact_match() should return instead of has_text())
# To Consider

- Consider if exact_match() could somehow be used in tandem with
has_text() in the tutorials. It's a part of functionality that new users
can't really see now, since it's rarely used (mostly in telegram
tutorials).

---------

Co-authored-by: Roman Zlobin <[email protected]>
- Removed a duplicate paragraph in CONTRIBUTING.md
Bumps the deps group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [black](https://github.com/psf/black) | `24.2.0` | `24.3.0` |
| [mypy](https://github.com/python/mypy) | `1.8.0` | `1.9.0` |
| [pytest](https://github.com/pytest-dev/pytest) | `8.0.2` | `8.1.1` |
|
[python-on-whales](https://github.com/gabrieldemarmiesse/python-on-whales)
| `0.69.0` | `0.70.0` |
| [uvicorn](https://github.com/encode/uvicorn) | `0.27.1` | `0.28.0` |
| [locust](https://github.com/locustio/locust) | `2.23.1` | `2.24.0` |
| [streamlit](https://github.com/streamlit/streamlit) | `1.31.1` |
`1.32.2` |

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the deps group with 2 updates:
[pytest-cov](https://github.com/pytest-dev/pytest-cov) and
[uvicorn](https://github.com/encode/uvicorn).

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [black](https://github.com/psf/black) from 24.2.0 to 24.3.0.

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# Description

Actor validation made required and also improved.

---------

Co-authored-by: Roman Zlobin <[email protected]>
ZergLev and others added 20 commits June 19, 2024 17:01
# Description

- Made `dff.script.conditions.exact_match` and
`dff.utils.testing.common.check_happy_path` accept `str` along with
`Message`
- Tests added for `exact_match()`, but not for `check_happy_path()`.
- Changed all instances of `exact_match(Message("some text"))` to
`exact_match("some text")`
# Checklist

- [x] I have performed a self-review of the changes

# To Consider

- Update tutorials / guides

---------

Co-authored-by: Roman Zlobin <[email protected]>
# Description

Add `quick_test` command which runs tests deselecting `slow` and
`docker` marked items.

# Checklist

- [x] I have performed a self-review of the changes

# To Consider

- Add tests (if functionality is changed)
- Update API reference / tutorials / guides
- Update CONTRIBUTING.md (if devel workflow is changed)
- Update `.ignore` files, scripts (such as `lint`), distribution
manifest (if files are added/deleted)
- Search for references to changed entities in the codebase

---------

Co-authored-by: Alexander Sergeev <[email protected]>
# Description

Replace framework_states dict with a pydantic model FrameworkData.

This makes it clear which data is stored in the field as well as allows
using pydantic validation to process framework data.

# Checklist

- [x] I have performed a self-review of the changes

# To Consider

- Add tests (if functionality is changed)
- Update API reference / tutorials / guides
- Update CONTRIBUTING.md (if devel workflow is changed)
- Update `.ignore` files, scripts (such as `lint`), distribution
manifest (if files are added/deleted)
- Search for references to changed entities in the codebase
freeze otel col version
# Description

- Telegram interface was rewritten and updated to be used with the most
recent `Interface` and `Attachment` classes.
- Added JSONPickle Serialization tools to allow json-serializing pickle-serializable objects.
- Attachments reworked.

---------

Co-authored-by: Roman Zlobin <[email protected]>
Add slots feature

---------

Co-authored-by: Denis Kuznetsov <[email protected]>
Co-authored-by: ruthenian8 <[email protected]>
Co-authored-by: pseusys <[email protected]>
# Changelog

## General

- Framework renamed from DFF to Chatsky (#368)

## Features

- New `has_text` condition for asserting that specific text is contained
inside the last request (#335)
- Validation stage reworked.
`Pipeline.validation_stage`, `Pipeline.verbose` and `Context.validation`
fields are removed.
  Added new type for annotating labels `dff.script.ConstLabel` (#289)
- Functions `cnd.exact_match` and `check_happy_path` now accept both
`Message` and `str`.
The following are now equivalent: `exact_match(Message("text"))` ==
`exact_match("text")`
  (#337)
- CLIMessengerInterface moved to `messengers.console` (#328)
- Attachments reworked (#328):
- `Session`, `Command`, `Link`, `Button`, `Keyboard` and `Attachments`
classes removed, `Message.commands` field removed.
- Added `CallbackQuery`, `Contact`, `Invoice`, `Poll`, `Animation`,
`Sticker`, `VoiceMessage`, `VideoMessage` and `MediaGroup` classes.
  - Added attachment caching feature.
  - Added `has_callback_query` condition.
  - Removed `title` field from Data Attachments. Use `caption` instead.
- Telegram Messenger interface reworked (#328):
  - Now provides methods for attachment byte download.
- Added support for receiving attachment types as members of
`Message.attachments`. Others may be accessed via
`Message.original_message`.
- Added support for sending new attachment types (`Sticker`, `Poll`,
e.t.c.).
- Added support for extra options such as `disable_notification`,
`message_effect_id` or `caption`.
  - Telegram tutorials rewritten to reflect all the changes.
- Added slots that simplify the process of extracting arbitrary data
from user messages and accessing it later (#36):
  - New user guide and tutorials on slots

## Documentation

- Return PRE_RESPONSE_PROCESSING and PRE_TRANSITION_PROCESSING tutorials
- New telegram interface tutorials (#328)
- Slots user guide and tutorials (#36)

## Contrib

- Added `poe quick_test` to run tests that are not slow and do not
require docker (#344)
- `Context.framework_states` renamed to `Context.framework_data` and
made `Pydantic.BaseModel` (#359)
- Added `MessengerInterfaceWithAttachments` class (#328)
- Added tools for enabling json serialization via pickle (#328)
- Telegram tests made independent from telegram servers (#328)
- `wrap_sync_function_in_async` moved to `utils.devel` (#36)

# Checklist

- [x] I have performed a self-review of the changes
- [x] Rename repos (this; template; demo)
- [ ] Add new logo; change favicons in conf.py (this can be done later)
- [x] Create new PyPi project and update pypi token
- [ ] Create a new release
- [x] Make a new release on the
[final_dff_release](https://github.com/deeppavlov/dialog_flow_framework/tree/final_dff_release)
branch
Pepy markdown badge was outdated. Now it is updated once again.
@Ramimashkouk Ramimashkouk marked this pull request as ready for review July 24, 2024 02:22
@Ramimashkouk Ramimashkouk requested a review from RLKRo July 24, 2024 02:22
@@ -74,6 +74,7 @@ python-telegram-bot = { version = "~=21.3", extras = ["all"], optional = true }
opentelemetry-instrumentation = { version = "*", optional = true }
sqlalchemy = { version = "*", extras = ["asyncio"], optional = true }
opentelemetry-exporter-otlp = { version = ">=1.20.0", optional = true } # log body serialization is required
pyyaml = "^6.0.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was script parsing working without pyyaml before?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weirdly enough, yes

chatsky/pipeline/pipeline/pipeline.py Outdated Show resolved Hide resolved
@RLKRo RLKRo merged commit 0c8f0a0 into feat/script-parser Aug 27, 2024
@RLKRo RLKRo deleted the chore/slots2parser branch August 27, 2024 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants