Skip to content

Commit

Permalink
Merge pull request #717
Browse files Browse the repository at this point in the history 
Feature/extend debug cli
  • Loading branch information
@drazvan
drazvan authored Sep 3, 2024
2 parents ee6633b + 51c0179 commit c99a17f
Show file tree
Hide file tree
Showing 11 changed files with 450 additions and 113 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG-Colang.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
* [#673](https://github.com/NVIDIA/NeMo-Guardrails/pull/673) Add support for new Colang 2 keyword `deactivate`.
* [#703](https://github.com/NVIDIA/NeMo-Guardrails/pull/703) Add bot configuration as variable `$system.config`.
* [#709](https://github.com/NVIDIA/NeMo-Guardrails/pull/709) Add basic support for most OpenAI and LLame 3 models.
* [#712](https://github.com/NVIDIA/NeMo-Guardrails/pull/712) Add interaction loop priority levels for flows.
* [#717](https://github.com/NVIDIA/NeMo-Guardrails/pull/717) Add CLI chat debugging commands.

### Changed

Expand Down
107 changes: 103 additions & 4 deletions docs/colang_2/language_reference/development-and-debugging.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Currently, the Colang story designing and building environment is fairly limited
Integrated Development Environment (IDE)
-----------------------------------------

- We suggest using `Visual Studio Code <https://code.visualstudio.com/>`_ and the Colang highlighting extension (`Github link <../../../vscode_extension>`_) to help with highlighting Colang code.
- You can use the Visual Studio Code launch setting to run a Colang story with nemoguardrails in a python environment by pressing F5 (`launch.json <../../../.vscode/launch.json>`_)
- We suggest using `Visual Studio Code <https://code.visualstudio.com/>`_ and the Colang highlighting extension (`Github link <https://github.com/NVIDIA/NeMo-Guardrails/tree/main/vscode_extension>`_) to help with highlighting Colang code.
- You can use the Visual Studio Code launch setting to run a Colang story with nemoguardrails in a python environment by pressing F5 (`launch.json <https://github.com/NVIDIA/NeMo-Guardrails/blob/main/.vscode/launch.json>`_)
- You can show generated and received events by adding the ``--verbose`` flag when starting nemoguardrails, that will also show all generated LLM prompts and responses
- To see even more details and show the internal run-time logs use ``--debug-level=INFO`` or set it equal to ``DEBUG``

Expand All @@ -31,13 +31,13 @@ To help debugging your Colang flows you can use the print statement ``print <exp
user said something as $flow
print $flow.transcript
.. code-block:: text
.. code-block:: console
> Hi
Hi
Alternatively, use the log statement ``log <expression>`` to append to the logging shown in the verbose mode, which will appear as "Colang debug info: <expression>".
Alternatively, use the log statement ``log <expression>`` to append to the logging shown in the verbose mode, which will appear as "Colang Log <flow_instance_uid> :: <expression>".

Furthermore, the Colang function ``flows_info`` can be used to return more information about a flow instance:

Expand Down Expand Up @@ -71,6 +71,105 @@ Furthermore, the Colang function ``flows_info`` can be used to return more infor
Where ``pretty_str`` converts the returned dictionary object to a nicely formatted string. If no parameter is provided to the function it will return a dictionary containing all the currently active flow instances.

-------------------------
CLI Debugging Commands
-------------------------

The NeMo Guardrail CLI provides a couple of additional debugging commands that always start with the ``!`` character:

.. code-block:: console
> !list-flows
┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID ┃ Flow Name ┃ Loop (Priority | Type | Id) ┃ Flow Instances ┃ Source ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1 │ tracking bot talking state │ 10 │ Named │ 'state_tracking' │ 4d0cb,3ea17,b3b49 │ /colang/v2_x/library/core.co │
│ 2 │ tracking user talking state │ 10 │ Named │ 'state_tracking' │ │ /colang/v2_x/library/core.co │
│ 3 │ tracking visual choice selection state │ 10 │ Named │ 'state_tracking' │ fa0f8,196ad │ /colang/v2_x/library/avatars.co │
│ 4 │ _bot_say │ 0 │ parent │ b6036 │ /colang/v2_x/library/core.co │
│ 5 │ _user_said │ 0 │ parent │ 303c3,565f7,cca80 │ /colang/v2_x/library/core.co │
│ 6 │ _user_said_something_unexpected │ 0 │ parent │ 1b5f7,41f02 │ /colang/v2_x/library/core.co │
│ 7 │ _user_saying │ 0 │ parent │ a65e9 │ /colang/v2_x/library/core.co │
│ 8 │ automating intent detection │ 0 │ parent │ c7ead │ /colang/v2_x/library/llm.co │
│ 9 │ await_flow_by_name │ 0 │ parent │ 5f7ef │ /colang/v2_x/library/core.co │
│ 10 │ bot answer question about france │ 0 │ parent │ │ llm_example_flows.co │
│ 11 │ bot ask │ 0 │ parent │ │ /colang/v2_x/library/core.co │
│ 12 │ bot ask how are you │ 0 │ parent │ │ demo.co │
│ 13 │ bot ask user for age │ 0 │ parent │ │ llm_example_flows.co │
│ 14 │ bot ask user to pick a color │ 0 │ parent │ │ llm_example_flows.co │
│ 15 │ bot asked something │ 0 │ parent │ │ /colang/v2_x/library/core.co │
│ 16 │ bot asks email address │ 0 │ parent │ │ show_case_back_channelling_interaction.co │
│ 17 │ bot asks user how the day was │ 0 │ parent │ │ show_case_back_channelling_interaction.co │
│ 18 │ bot attract user │ 0 │ parent │ │ llm_example_flows.co │
│ 19 │ bot clarified something │ 0 │ parent │ │ /colang/v2_x/library/core.co │
│ 20 │ bot clarify │ 0 │ parent │ │ /colang/v2_x/library/core.co │
│ 21 │ bot count from a number to another number │ 0 │ parent │ │ llm_example_flows.co │
│ 22 │ bot express │ 0 │ parent │ │ /colang/v2_x/library/core.co │
│ 23 │ bot express feeling bad │ 0 │ parent │ │ demo.co │
│ 24 │ bot express feeling well │ 0 │ parent │ │ demo.co │
│ 25 │ bot express greeting │ 0 │ parent │ │ demo.co │
│ 26 │ bot expressed something │ 0 │ parent │ │ /colang/v2_x/library/core.co │
│ 27 │ bot gesture │ 0 │ parent │ a2528 │ /colang/v2_x/library/avatars.co │
│ 28 │ bot gesture with delay │ 0 │ parent │ │ /colang/v2_x/library/avatars.co │
│ 29 │ bot inform │ 0 │ parent │ da186 │ /colang/v2_x/library/core.co │
│ 30 │ bot inform about service │ 0 │ parent │ │ demo.co │
│ 31 │ bot informed something │ 0 │ parent │ │ /colang/v2_x/library/core.co │
│ 32 │ bot make long pause │ 0 │ parent │ │ demo.co │
│ 33 │ bot make short pause │ 0 │ parent │ │ demo.co │
└────┴───────────────────────────────────────────┴───┴───────────────────────────┴───────────────────┴───────────────────────────────────────────┴
.. code-block:: console
> !tree
main
├── notification of undefined flow start `Excuse me, what did you say?` (fdb... ,started)
├── notification of colang errors `Excuse me, what did you say?` (69c... ,started)
├── automating intent detection (c7e... ,started)
│ ├── logging marked user intent flows (d84... ,started)
│ └── logging marked bot intent flows (462... ,started)
├── showcase selector (742... ,started)
│ ├── handling bot talking interruption `inform` `stop|cancel` (dea... ,started)
│ │ └── > user interrupted bot talking `15` `stop|cancel` (109... ,started)
│ │ └── > bot started saying something (561... ,started)
│ ├── > user was silent `15.0` (40f... ,started)
│ │ └── > wait `15.0` `wait_timer_1f499f52-9634-4925-b18a-579fef485d5e` (ae4... ,started)
│ ├── > user picked number guessing game showcase (6bc... ,started)
│ │ ├── > user has selected choice `game` (b88... ,started)
│ │ ├── > user said `I want to play the number guessing game` (448... ,started)
│ │ │ └── > _user_said `I want to play the number guessing game` (303... ,started)
│ │ ├── > user said `Show me the game` (d32... ,started)
│ │ │ └── > _user_said `Show me the game` (565... ,started)
│ │ ├── > user said `showcase A` (3fd... ,started)
│ │ │ └── > _user_said `showcase A` (cca... ,started)
│ │ ├── > user said `First showcase` (013... ,started)
│ │ │ └── > _user_said `First showcase` (9d4... ,started)
│ │ └── > user said `re.compile('(?i)guessing game', re.IGNORECASE)` (af9... ,started)
│ │ └── > _user_said `re.compile('(?i)guessing game', re.IGNORECASE)` (966... ,started)
│ ├── > user picked multimodality showcase (e2d... ,started)
│ │ ├── > user has selected choice `multimodality` (f69... ,started)
│ │ ├── > user said `Show me the multimodality showcase` (9be... ,started)
│ │ │ └── > _user_said `Show me the multimodality showcase` (33f... ,started)
│ │ ├── > user said `multimodality` (dfe... ,started)
│ │ │ └── > _user_said `multimodality` (18f... ,started)
│ │ ├── > user said `showcase B` (aa6... ,started)
│ │ │ └── > _user_said `showcase B` (4c5... ,started)
│ │ ├── > user said `Second showcase` (205... ,started)
│ │ │ └── > _user_said `Second showcase` (92a... ,started)
│ │ └── > user said `re.compile('(?i)multimodality', re.IGNORECASE)` (b10... ,started)
│ │ └── > _user_said `re.compile('(?i)multimodality', re.IGNORECASE)` (d86... ,started)
.. code-block:: colang
:caption: All CLI debugging commands
flows [--all] [--order_by_name] # Shows all (active) flows in a table in order of their interaction loop priority and name
tree # Shows the flow hierarchy tree of all (active) flows
flow [<flow_name>|<flow_instance_uid>] # Show flow or flow instance details
pause # Pause timer event processing such that interaction does not continue on its own
resume # Resume timer event processing, including the ones trigger during paus
restart # Reset interaction and restart the Colang script
-------------
Useful Flows
-------------
Expand Down
6 changes: 4 additions & 2 deletions docs/colang_2/language_reference/more-on-flows.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -297,12 +297,12 @@ So far, any concurrently progressing flows that resulted in different event gene

.. code-block:: colang
@loop("<loop_name>")
@loop([id=]"<loop_name>"[,[priority=]<integer_number>])
flow <name of flow> ...
Hint: To generate a new loop name for each flow call use the loop name "NEW"

By default, any flow without an explicit interaction loop inherits the interaction loop of its parent flow. Let's see now an example of a second interaction loop to design flows that augment the main interaction rather than compete with it:
By default, any flow without an explicit interaction loop inherits the interaction loop of its parent flow and has priority level 0. Let's see now an example of a second interaction loop to design flows that augment the main interaction rather than compete with it:

.. code-block:: colang
:caption: more_on_flows/interaction_loops/main.co
Expand Down Expand Up @@ -356,6 +356,8 @@ The example implements two bot reaction flows that listen to the user saying "Hi
Goodbye
By default, parallel flows in different interaction loops advance in order of their start or activation. This might be an important detail if e.g a global variable is set in one flow and read in another. If the order is wrong, the global variable will not be set yet when read by the other flow. In order to enforce the processing order independent of the start or activation order, you can define the interaction loop priority level using an integer. By default, any interaction loop has priority 0. A higher number defines a higher priority, and lower (negative) number a lower processing priority.


.. _more-on-flows-flow-conflict-resolution-prioritization:

Expand Down
Loading

0 comments on commit c99a17f

Please sign in to comment.