feat: Batch Mode API / Headless #139

dkindlund · 2024-10-30T00:01:14Z

This PR adds a new mode called "Batch Mode" to the computer-use-demo container. If you specify RUN_MODE=batch as an additional environment variable, the code will turn off Streamlit (on port 8080) and turn on the Batch Mode API (on port 8000).

This will allow users to submit asynchronous tasks via this endpoint:

curl -X POST http://localhost:8000/tasks \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Open the Firefox browser and navigate to google.com"
  }'

If the container is NOT running in Google Cloud Run, then this task will return immediately with a task_id. You can then fetch the status and results of the task through the /tasks endpoint, accordingly.

If the container IS running in Google Cloud Run, then this task will switch to synchronous processing. Only when the Agent fully completes, will the final results be returned to the client.

See the updated README.md for more details on these modes (and caveats).

…services.

x5a

thanks for the contribution @dkindlund.

I'm glad you found this project useful. In its current state, this PR doesn't meet a few criteria we're looking for new features:

provider/environment agnostic: this only works with cloud run; the "local" behavior is substantively different than cloud run.
broad applicability: because the batch mode resets between inputs, you can't take multi-turn actions. as such, the utility is fairly limited for multi-turn agent use.

x5a · 2024-10-30T20:31:41Z

computer-use-demo/Dockerfile

@@ -85,7 +86,8 @@ RUN eval "$(pyenv init -)" && \
 ENV PATH="$HOME/.pyenv/shims:$HOME/.pyenv/bin:$PATH"

 RUN python -m pip install --upgrade pip==23.1.2 setuptools==58.0.4 wheel==0.40.0 && \
-    python -m pip config set global.disable-pip-version-check true
+    python -m pip config set global.disable-pip-version-check true && \
+    python -m pip install fastapi uvicorn


this should be covered by the subsequent line, correct?

x5a · 2024-10-30T20:38:00Z

computer-use-demo/computer_use_demo/requirements.txt

+fastapi>=0.104.0
+uvicorn[standard]>=0.24.0


add starlette/pydantic given that they are now explicit imports

x5a · 2024-10-30T20:40:41Z

computer-use-demo/computer_use_demo/batch_api.py

+    status: str
+    created_at: datetime
+    completed_at: Optional[datetime] = None
+    messages: list = []


Suggested change

messages: list = []

messages: list[BetaMessage] = []

x5a · 2024-10-30T20:41:27Z

computer-use-demo/computer_use_demo/batch_api.py

+
+class TaskRequest(BaseModel):
+    prompt: str
+    provider: APIProvider = APIProvider.VERTEX  # Default to VERTEX since we're in Vertex environment


not necessarily true for a general use case.

x5a · 2024-10-30T20:41:50Z

computer-use-demo/computer_use_demo/batch_api.py

+    logger.info("👋 Batch API server shutting down")
+
+# Add environment detection
+IS_CLOUD_RUN = os.getenv('K_SERVICE') is not None  # Cloud Run sets this automatically


this would maybe better be called IS_STATELESS or something more generic to indicate that it's behind a load balancer.

x5a · 2024-10-30T20:44:47Z

computer-use-demo/computer_use_demo/batch_api.py

+                    "pkill firefox-esr || true",
+                    "pkill -f 'libreoffice' || true",
+                    "pkill -f 'gedit' || true",


claude can start arbitrary programs, so this is not sufficient. also see https://github.com/anthropics/anthropic-quickstarts/blob/main/computer-use-demo/computer_use_demo/streamlit.py#L152

x5a · 2024-10-30T20:47:56Z

computer-use-demo/computer_use_demo/batch_api.py

+    def get_model(self) -> str:
+        """Get the model name, using the default if none specified"""
+        if self.model is None:
+            return PROVIDER_TO_DEFAULT_MODEL_NAME[self.provider]
+        return self.model


use model: str = field(default_factory=_get_model) instead to simplify

x5a · 2024-10-30T20:50:46Z

computer-use-demo/image/entrypoint.sh

-echo "✨ Computer Use Demo is ready!"
-echo "➡️  Open http://localhost:8080 in your browser to begin"
+# Check if we should start in batch mode
+if [ "$RUN_MODE" = "batch" ]; then


from my perspective, batch mode and streamlit aren't necessarily mutually exclusive (although if both are supported simulatenously, there should probably be a shared lock)

x5a · 2024-10-30T20:58:02Z

computer-use-demo/computer_use_demo/batch_api.py

+
+    tasks[task_id] = task_data
+
+    if IS_CLOUD_RUN:


I understand why, but if I wasn't familiar with cloud run, I'd find it surprising how different the cloud run implementation is from the local implementation.

dkindlund added 3 commits October 29, 2024 18:24

Added batch mode API support that's compatible with Google Cloud Run …

30ad22b

…services.

Added /health check endpoint and initial documentation for Batch Mode.

ba6ad51

Added more clarifications to the documentation

a00d61a

dkindlund mentioned this pull request Oct 30, 2024

Feature Request: Headless Mode #138

Open

x5a requested changes Oct 30, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Batch Mode API / Headless #139

feat: Batch Mode API / Headless #139

dkindlund commented Oct 30, 2024

x5a left a comment

x5a Oct 30, 2024

x5a Oct 30, 2024

x5a Oct 30, 2024

x5a Oct 30, 2024

x5a Oct 30, 2024

x5a Oct 30, 2024

x5a Oct 30, 2024

x5a Oct 30, 2024

x5a Oct 30, 2024

		fastapi>=0.104.0
		uvicorn[standard]>=0.24.0

feat: Batch Mode API / Headless #139

Are you sure you want to change the base?

feat: Batch Mode API / Headless #139

Conversation

dkindlund commented Oct 30, 2024

x5a left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment