Skip to content

Commit

Permalink
deploy: 7527534
Browse files Browse the repository at this point in the history
  • Loading branch information
ZiTao-Li committed May 15, 2024
1 parent aec240a commit 80e7475
Show file tree
Hide file tree
Showing 14 changed files with 282 additions and 322 deletions.
Binary file modified en/.doctrees/environment.pickle
Binary file not shown.
Binary file modified en/.doctrees/index.doctree
Binary file not shown.
Binary file modified en/.doctrees/tutorial/203-parser.doctree
Binary file not shown.
146 changes: 63 additions & 83 deletions en/_sources/tutorial/203-parser.md.txt
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,12 @@ The main functions of the parser module include:

1. Provide "format instruction", that is, remind LLM where to generate what output, for example

> You should generate python code in a fenced code block as follows
>
> \```python
>
> {your_python_code}
>
> \```
````
You should generate python code in a fenced code block as follows
```python
{your_python_code}
```
````

2. Provide a parse function, which directly parses the text generated by LLM into the target data format,

Expand All @@ -82,9 +81,7 @@ In the following sections, we will introduce the usage of these parsers based on

### String Type

<details>

<summary id="markdowncodeblockparser"> MarkdownCodeBlockParser </summary>
#### MarkdownCodeBlockParser

##### Initialization

Expand All @@ -93,34 +90,32 @@ In the following sections, we will introduce the usage of these parsers based on
```python
from agentscope.parsers import MarkdownCodeBlockParser

parser = MarkdownCodeBlockParser(language_name="python")
parser = MarkdownCodeBlockParser(language_name="python", content_hint="your python code")
```

##### Format Instruction Template

- `MarkdownCodeBlockParser` provides the following format instruction template. When the user calls the `format_instruction` attribute, `{language_name}` will be replaced with the string entered at initialization:

> You should generate {language_name} code in a {language_name} fenced code block as follows:
>
> \```{language_name}
>
> ${your_{language_name}_code}
>
> \```
````
You should generate {language_name} code in a {language_name} fenced code block as follows:
```{language_name}
{content_hint}
```
````

- For the above initialization with `language_name` as `"python"`, when the `format_instruction` attribute is called, the following string will be returned:

```python
print(parser.format_instruction)
```
```python
print(parser.format_instruction)
```

> You should generate python code in a python fenced code block as follows
>
> \```python
>
> ${your_python_code}
>
> \```
````
You should generate python code in a python fenced code block as follows
```python
your python code
```
````

##### Parse Function

Expand All @@ -140,9 +135,9 @@ In the following sections, we will introduce the usage of these parsers based on
print(res.parsed)
````

> print("hello world!")

</details>
```
print("hello world!")
```

### Dictionary Type

Expand Down Expand Up @@ -259,18 +254,16 @@ In AgentScope, we achieve post-processing by calling the `to_content`, `to_memor
> print(parser.to_memory(example_dict)) # {"thought": "abc", "speak": "def"}
> print(parser.to_metadata(example_dict)) # None
> ```
> > def
> >
> > {"thought": "abc", "speak": "def"}
> >
> > None
> ```
> def
> {"thought": "abc", "speak": "def"}
> None
> ```


Next we will introduce two parsers for dictionary type.

<details>

<summary id="markdownjsondictparser"> MarkdownJsonDictParser </summary>
#### MarkdownJsonDictParser

##### Initialization & Format Instruction Template

Expand Down Expand Up @@ -300,19 +293,14 @@ This parameter can be a string or a dictionary. For dictionary, it will be autom

- The corresponding `instruction_format` attribute

> You should respond a json object in a json fenced code block as follows:
>
> \```json
>
> {content_hint}
>
> \```

</details>

<details>
````
You should respond a json object in a json fenced code block as follows:
```json
{content_hint}
```
````

<summary id="multitaggedcontentparser"> MultiTaggedContentParser </summary>
#### MultiTaggedContentParser

`MultiTaggedContentParser` asks LLM to generate specific content within multiple tag pairs. The content from different tag pairs will be parsed into a single Python dictionary. Its usage is similar to `MarkdownJsonDictParser`, but the initialization method is different, and it is more suitable for weak LLMs or complex return content.

Expand Down Expand Up @@ -352,13 +340,12 @@ parser = MultiTaggedContentParser(
print(parser.format_instruction)
```

> Respond with specific tags as outlined below, and the content between [FINISH_DISCUSSION] and [/FINISH_DISCUSSION] MUST be a JSON object:
>
> [THOUGHT]what you thought[/THOUGHT]
>
> [SPEAK]what you speak[/SPEAK]
>
> [FINISH_DISCUSSION]true/false, whether the discussion is finished[/FINISH_DISCUSSION]
```
Respond with specific tags as outlined below, and the content between [FINISH_DISCUSSION] and [/FINISH_DISCUSSION] MUST be a JSON object:
[THOUGHT]what you thought[/THOUGHT]
[SPEAK]what you speak[/SPEAK]
[FINISH_DISCUSSION]true/false, whether the discussion is finished[/FINISH_DISCUSSION]
```

##### Parse Function

Expand All @@ -378,23 +365,17 @@ res_dict = parser.parse(
print(res_dict)
```

> {
>
> "thought": "The others didn't realize I was a werewolf. I should end the discussion soon.",
>
> "speak": "I agree with you.",
>
> "finish_discussion": true
>
> }

</details>
```
{
"thought": "The others didn't realize I was a werewolf. I should end the discussion soon.",
"speak": "I agree with you.",
"finish_discussion": true
}
```

### JSON / Python Object Type

<details>

<summary id="markdownjsonobjectparser"> MarkdownJsonObjectParser </summary>
#### MarkdownJsonObjectParser

`MarkdownJsonObjectParser` also uses the \```json and \``` tags in Markdown, but does not limit the content type. It can be a list, dictionary, number, string, etc., which can be parsed into a Python object via `json.loads`.

Expand All @@ -410,13 +391,12 @@ parser = MarkdownJsonObjectParser(
print(parser.format_instruction)
```

> You should respond a json object in a json fenced code block as follows:
>
> \```json
>
> {a list of numbers}
>
> \```
````
You should respond a json object in a json fenced code block as follows:
```json
{a list of numbers}
```
````

##### Parse Function

Expand All @@ -433,11 +413,11 @@ res = parser.parse(
print(type(res))
print(res)
````
> <class 'list'>
>
> [1, 2, 3, 4, 5]

</details>
```
<class 'list'>
[1, 2, 3, 4, 5]
```

## Typical Use Cases

Expand Down
Binary file modified en/objects.inv
Binary file not shown.
2 changes: 1 addition & 1 deletion en/searchindex.js

Large diffs are not rendered by default.

Loading

0 comments on commit 80e7475

Please sign in to comment.