Skip to content

Commit

Permalink
add model_config_path and update tutorial
Browse files Browse the repository at this point in the history
  • Loading branch information
pan-x-c committed May 10, 2024
1 parent 8f3eaa2 commit 49c5dcf
Show file tree
Hide file tree
Showing 3 changed files with 47 additions and 27 deletions.
15 changes: 9 additions & 6 deletions docs/sphinx_doc/en/source/tutorial/208-distribute.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,16 @@ b = AgentB(
#### Independent Process Mode

In the Independent Process Mode, we need to start the agent server process on the target machine first.
When starting the agent server process, you need to specify a model config file, which contains the models which can be used in the agent server, the IP address and port of the agent server process
For example, start two agent server processes on the two different machines with IP `ip_a` and `ip_b`(called `Machine1` and `Machine2` accrodingly).
You can run the following code on `Machine1`:
You can run the following code on `Machine1`, and make sure you have put your model config file in `model_config_path_a`.

```python
# import some packages

# register models which can be used in the server
agentscope.init(
...
model_configs=model_config_path_a,
)
# Create an agent service process
server = RpcAgentServerLauncher(
Expand All @@ -82,16 +84,17 @@ server.wait_until_terminate()
> For similarity, you can run the following command in your terminal rather than the above code:
>
> ```shell
> as_server --host ip_a --port 12001
> as_server --host ip_a --port 12001 --model-config-path model_config_path_a
> ```
And run the following code on `Machine2`:
And put your model config file accordingly in `model_config_path_b` and run the following code on `Machine2`.
```python
# import some packages
# register models which can be used in the server
agentscope.init(
...
model_configs=model_config_path_b,
)
# Create an agent service process
server = RpcAgentServerLauncher(
Expand All @@ -107,7 +110,7 @@ server.wait_until_terminate()
> Similarly, you can run the following command in your terminal to setup the agent server:
>
> ```shell
> as_server --host ip_b --port 12002
> as_server --host ip_b --port 12002 --model-config-path model_config_path_b
> ```
Then, you can connect to the agent servers from the main process with the following code.
Expand Down
16 changes: 9 additions & 7 deletions docs/sphinx_doc/zh_CN/source/tutorial/208-distribute.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,15 +57,16 @@ b = AgentB(

#### 独立进程模式

在独立进程模式中,需要首先在目标机器上启动智能体服务器进程。
在独立进程模式中,需要首先在目标机器上启动智能体服务器进程,启动时需要提供该服务器能够使用的模型的配置信息,以及服务器的 IP 和端口号
例如想要将两个智能体服务进程部署在 IP 分别为 `ip_a``ip_b` 的机器上(假设这两台机器分别为`Machine1``Machine2`)。
你可以先在 `Machine1` 上运行如下代码:
你可以先在 `Machine1` 上运行如下代码,运行之前请确保已经将模型配置文件放置在 `model_config_path_a` 位置。

```python
# import some packages

# register models which can be used in the server
agentscope.init(
...
model_configs=model_config_path_a,
)
# Create an agent service process
server = RpcAgentServerLauncher(
Expand All @@ -81,16 +82,17 @@ server.wait_until_terminate()
> 为了进一步简化使用,可以在命令行中输入如下指令来代替上述代码:
>
> ```shell
> as_server --host ip_a --port 12001
> as_server --host ip_a --port 12001 --model-config-path model_config_path_a
> ```
之后在 `Machine2` 上运行如下代码
`Machine2` 上运行如下代码,这里同样要确保已经将模型配置文件放置在 `model_config_path_b` 位置。
```python
# import some packages
# register models which can be used in the server
agentscope.init(
...
model_configs=model_config_path_b,
)
# Create an agent service process
server = RpcAgentServerLauncher(
Expand All @@ -106,7 +108,7 @@ server.wait_until_terminate()
> 这里也同样可以用如下指令来代替上面的代码。
>
> ```shell
> as_server --host ip_b --port 12002
> as_server --host ip_b --port 12002 --model-config-path model_config_path_b
> ```
接下来,就可以使用如下代码从主进程中连接这两个智能体服务器进程。
Expand Down
43 changes: 29 additions & 14 deletions src/agentscope/server/launcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,13 @@
except ImportError:
grpc = None

from .servicer import AgentServerServicer
from ..agents.agent import AgentBase
from ..utils.tools import _get_timestamp
import agentscope
from agentscope.server.servicer import AgentServerServicer
from agentscope.agents.agent import AgentBase
from agentscope.utils.tools import _get_timestamp

try:
from ..rpc.rpc_agent_pb2_grpc import (
from agentscope.rpc.rpc_agent_pb2_grpc import (
add_RpcAgentServicer_to_server,
)
except ModuleNotFoundError:
Expand Down Expand Up @@ -390,20 +391,21 @@ def as_server() -> None:
* `--host`: the hostname of the server.
* `--port`: the socket port of the server.
* `--max_pool_size`: max number of task results that the server can
* `--max-pool-size`: max number of task results that the server can
accommodate.
* `--max_timeout_seconds`: max timeout seconds of a task.
* `--local_mode`: whether the started agent server only listens to
* `--max-timeout-seconds`: max timeout seconds of a task.
* `--local-mode`: whether the started agent server only listens to
local requests.
* `--model-config-path`: the path to the model config json file
In most cases, you only need to specify the `--host` and `--port`.
In most cases, you only need to specify the `--host`, `--port` and
`--model-config-path`.
.. code-block:: shell
as_server --host localhost --port 12345
"""
as_server --host localhost --port 12345 --model-config-path config.json
""" # noqa
parser = argparse.ArgumentParser()
parser.add_argument(
"--host",
Expand All @@ -418,24 +420,37 @@ def as_server() -> None:
help="socket port of the server",
)
parser.add_argument(
"--max_pool_size",
"--max-pool-size",
type=int,
default=8192,
help="max number of task results that the server can accommodate",
)
parser.add_argument(
"--max_timeout_seconds",
"--max-timeout-seconds",
type=int,
default=1800,
help="max timeout for task results",
)
parser.add_argument(
"--local_mode",
"--local-mode",
type=bool,
default=False,
help="whether the started agent server only listens to local requests",
)
parser.add_argument(
"--model-config-path",
type=str,
help="path to the model config json file",
)
args = parser.parse_args()
agentscope.init(
project="agent_server",
name=f"server_{args.host}:{args.port}",
runtime_id=_get_timestamp(
"server_{}_{}_%y%m%d-%H%M%S",
).format(args.host, args.port),
model_configs=args.model_config_path,
)
launcher = RpcAgentServerLauncher(
host=args.host,
port=args.port,
Expand Down

0 comments on commit 49c5dcf

Please sign in to comment.