Skip to content

Commit

Permalink
Docs overhaul - Introduction+Quickstart (#221)
Browse files Browse the repository at this point in the history
* Consolidating

* WIP

* typo

* Fixing Cohere

* Added Introduction

* WIP

* Consolidating

* WIP

* typo

* Fixing Cohere

* Added Introduction

* WIP

* Adding syntax highlighting

* Added to env var

* Wrapping up

* Done
  • Loading branch information
HowieG authored May 23, 2024
1 parent 45ac351 commit 27d8311
Show file tree
Hide file tree
Showing 17 changed files with 150 additions and 102 deletions.
Binary file added docs/images/link-to-session.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/overview.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/session-drawer.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/session-summary.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/session-summary.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/session-waterfall.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/session-waterfall.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 1 addition & 6 deletions docs/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,7 @@
"navigation": [
{
"group": "Getting Started",
"pages": [
"v1/introduction",
"v1/installation",
"v1/quickstart",
"v1/examples"
]
"pages": ["v1/introduction", "v1/quickstart", "v1/examples"]
},
{
"group": "Details",
Expand Down
15 changes: 10 additions & 5 deletions docs/v1/details/environment-variables.mdx
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
---
title: "Environment Variables"
description: "Environment Variables that AgentOps will look for"
mode: "wide"
---

### Sample .env file

```.env
```python .env
# API KEY for using AgentOps. Find this in your settings page
AGENTOPS_API_KEY=<YOUR API KEY>
# Secondary API Key for simultaneously recording to a second project

##########################################################
# Only copy the following if you're going to override them

# (OPTIONAL) Secondary API Key for simultaneously recording to a second project
AGENTOPS_PARENT_KEY=<YOUR PARENT KEY>
# The AgentOps API endpoint. Defaults to https://api.agentops.ai
# (OPTIONAL) The AgentOps API endpoint. Defaults to https://api.agentops.ai
AGENTOPS_API_ENDPOINT=https://api.agentops.ai
# Logging level. <DEBUG, INFO, CRITICAL>
# (OPTIONAL) Logging level. <DEBUG, INFO, CRITICAL>
AGENTOPS_LOGGING_LEVEL=INFO
# Whether to opt out of recording environment data. <FALSE, TRUE>
# (OPTIONAL) Whether to opt out of recording environment data. <FALSE, TRUE>
AGENTOPS_ENV_DATA_OPT_OUT=FALSE
```
6 changes: 2 additions & 4 deletions docs/v1/details/tracking-agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,9 @@ description: "Use the `@track_agent()` decorator to tag your agents"

All Events are automatically associated with the agent that originated them. Agents are given a name which is what you will see in the dashboard.

<div
style={{ backgroundColor: "#f0f0f0", padding: "10px", borderRadius: "10px" }}
>
<Frame type="glass" caption="This event is labeled with the name of the Agent that originated it">
<img height="200" src="/images/agent-name.png" />
</div>
</Frame>

The example below tags the agent MyAgent with name `SomeCustomName`.

Expand Down
2 changes: 1 addition & 1 deletion docs/v1/details/tracking-llm-calls.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Still not working? Please let us know! You can find us on [Discord](https://disc
To get started, just follow the quick start guide.

<CardGroup cols={2}>
<Card title="Quick Start" icon="code" href="/v0/quickstart">
<Card title="Quick Start" icon="code" href="/v1/quickstart">
Get started with AgentOps in under 5 minutes
</Card>
</CardGroup>
9 changes: 5 additions & 4 deletions docs/v1/examples.mdx
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
---
title: 'Examples'
description: 'Sample code that you can run!'
mode: "wide"
---

## Explore our example code to see AgentOps in action!

<CardGroup cols={2}>
<Card title="Basic Example" icon="square-code" href="https://github.com/AgentOps-AI/agentops-py/blob/main/examples/openai-gpt.ipynb">
Juypter Notebook with sample code that you can run! Start here!
Jupyter Notebook with sample code that you can run! Start here!
</Card>
<Card title="Multi-Agent Code Example" icon="object-group" href="https://github.com/AgentOps-AI/agentops-py/blob/main/examples/multi_agent_example.ipynb">
Juypter Notebook with a simple multi-agent design
Jupyter Notebook with a simple multi-agent design
</Card>
<Card title="LangChain Example" icon="link" href="https://github.com/AgentOps-AI/agentops-py/blob/main/examples/langchain_examples.ipynb">
Juypter Notebook with a sample LangChain integration
Jupyter Notebook with a sample LangChain integration
</Card>
<Card title="Crew.ai Example" icon="ship" href="/v1/integrations/crewai">
CrewAI multi-agent framework with AgentOps support
Expand All @@ -28,7 +29,7 @@ description: 'Sample code that you can run!'
![thumbnail](https://cdn.loom.com/sessions/thumbnails/0e0d2986f3d644a58d1e186dc81cd8b1-with-play.gif)
</Card>
<Card title="Multi-Agent" icon="link" href="https://www.loom.com/share/2a62fd70efe8423fb495946b09c2d8ac">
Juypter Notebook with a sample LangChain integration
Jupyter Notebook with a sample LangChain integration
![thumbnail](https://cdn.loom.com/sessions/thumbnails/2a62fd70efe8423fb495946b09c2d8ac-with-play.gif)
</Card>
<Card title="Crew.ai" icon="ship" href="https://www.loom.com/share/cfcaaef8d4a14cc7a974843bda1076bf">
Expand Down
29 changes: 0 additions & 29 deletions docs/v1/installation.mdx

This file was deleted.

8 changes: 3 additions & 5 deletions docs/v1/integrations/cohere.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Cohere
description: "AgentOps's support for Cohere(>=5.4.0) is first class like Jack Harlow. This is a living integration, should you need any added functionality please message us on Discord!"
description: "AgentOps's support for Cohere(>=5.4.0) is first class. This is a living integration, should you need any added functionality please message us on Discord!"
---

## Install the SDK
Expand All @@ -16,7 +16,7 @@ poetry add agentops

## Get an API key

Retrieve an API Key from your [settings](https://app.agentops.ai/settings/projects) page.
Retrieve an API Key from the Settings > [Projects & API Keys](https://app.agentops.ai/settings/projects) page.

API keys are tied to individual projects.<br></br>
A Default Project has been created for you, so just click <span style={{ padding: "5px 10px", backgroundColor: "black", color: "white", borderRadius: "5px"}}>Copy API Key</span>
Expand All @@ -32,9 +32,7 @@ A Default Project has been created for you, so just click <span style={{ padding

## Using AgentOps with Cohere

### Requires cohere>=5.4.0

#### Session replays in 3 lines of code
<div style={{ color: "#880000" }}>Requires cohere>=5.4.0</div>

Initialize the AgentOps client and automatically get analytics on every LLM call.

Expand Down
71 changes: 54 additions & 17 deletions docs/v1/introduction.mdx
Original file line number Diff line number Diff line change
@@ -1,23 +1,60 @@
---
title: Introduction
description: "Build your next agent with evals, observability, and replay."
title: "Introduction"
description: "Build your next agent with evals, observability, and replays"
mode: "wide"
---

## Get Started
<Check>[Give us a star](https://github.com/AgentOps-AI/agentops) on GitHub if you find AgentOps helpful (you may be our 1,000th 😊)</Check>

<CardGroup cols={2}>
<Card title="Installing AgentOps" icon="wrench" href="./installation">
Prepare your system for AgentOps
</Card>
<Card title="Quick Start" icon="rocket" href="./quickstart">
Start using AgentOps with just 3 lines of code
</Card>
</CardGroup>
# AgentOps is your new terminal

## Integrate with your agent
<Card title="Terminals suck" icon="trash" iconType="solid" color="#CC0000">
- • Lose track of what your agents did in between executions
- • Parsing through terminal output searching for LLM completions
- • Printing "tool called"
</Card>
<Card title="AgentOps doesn't" icon="face-smile-halo" iconType="duotone" color="#00CC00">
- • Visual dashboard so you can see what your agents did in human-readable format
- • LLM calls are magically recorded - prompt, completion, timestamps for each - with one line of code
- • Agents and their events (including tool calls) are recorded with one more line of code
- • Errors are magically associated to its causal event
- • Record any other events to your session with two more lines of code
- • Tons of other useful data if you're developing with supported agent frameworks: SDK version
</Card>

<CardGroup cols={2}>
<Card title="Example Code" icon="square-code" href="./examples">
Sample code that you can run!
</Card>
</CardGroup>
## The AgentOps Dashboard

With just two lines of code, you can free yourself from the chains of the terminal and instead visualize your agents' behavior
in your AgentOps Dashboard. After setting up AgentOps, each execution of your program is recorded as a session and the above
data is automatically recorded for you.

The examples below were captured with two lines of code.

### Session Drilldown
Here you will find a list of all of your previously recorded sessions and useful data about each such as total execution time.
You also get helpful debugging info such as any SDK versions you were on if you're building on a supported agent framework like Crew or AutoGen.
LLM calls are presented as a familiar chat history view, and charts give you a breakdown of the types of events that were called and how long they took.

<Frame type="glass" caption="Session Summary">
<img height="200" src="/images/session-summary.gif" />
</Frame>

Find any past sessions from your Session Drawer.
<Frame type="glass" caption="Session Drawer">
<img height="250" src="/images/session-drawer.gif" />
</Frame>

Most powerful of all is the Session Waterfall. On the left, a time visualization of all your LLM calls, Action events, Tool calls, and Errors.
On the right, specific details about the event you've selected on the waterfall. For instance the exact prompt and completion for a given LLM call.
Most of which has been automatically recorded for you.

<Frame type="glass" caption="Session Waterfall">
<img height="200" src="/images/session-waterfall.gif" />
</Frame>


### Session Overview
View a meta-analysis of all of your sessions in a single view.
<Frame type="glass" caption="Session Overview">
<img height="200" src="/images/overview.gif" />
</Frame>
105 changes: 74 additions & 31 deletions docs/v1/quickstart.mdx
Original file line number Diff line number Diff line change
@@ -1,28 +1,71 @@
---
title: "Quickstart"
description: "Start using AgentOps with just 3 lines of code"
description: "Start using AgentOps with just 2 lines of code"
---

<CodeGroup>
```python python
import openai
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)

agentops.init(<INSERT YOUR API KEY HERE>)

# Woohoo! You're on your way 🎉

```
</CodeGroup>

When the `openai`, `litellm`, or `cohere` packages have been imported, instantiating the AgentOps client will automatically
instrument them, meaning you will be able to see all of your sessions on the AgentOps
dashboard along with the full LLM chat histories, cost, token counts, etc.


## Decorate Functions
<Steps>
<Step title="Install the SDK">
<CodeGroup>
```bash pip
pip install agentops
```
```bash poetry
poetry add agentops
```
</CodeGroup>
<Check>[Give us a star](https://github.com/AgentOps-AI/agentops) on GitHub while you're at it (you may be our 1,000th 😊)</Check>
</Step>
<Step title="Add 2 lines of code">
<CodeGroup>
```python python
# IMPORTANT: Must be after 'import openai' or 'from crewai import Agent' etc
import agentops

# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
```
</CodeGroup>
<Check>Just these two lines and you have magically tracked LLM calls!</Check>
<Tip>
After the `openai`, `litellm`, or `cohere` packages have been imported, importing + instantiating the AgentOps client
will automatically instrument them, meaning you will be able to see all of your sessions on the AgentOps
DEBUGashboard along with the full LLM chat histories, cost, token counts, etc.
</Tip>

</Step>
<Step title="Get an API key">
Retrieve an API Key from the Settings > [Projects & API Keys](https://app.agentops.ai/settings/projects) page.
<div/>{/* Intentionally blank div for newline */}
Set this API Key in your [environment variables](/v1/details/environment-variables)

<Frame type="glass" caption="Settings > Projects & API Keys">
<img height="200" src="/images/api-keys.png" />
</Frame>

<Info>
API keys are tied to individual projects.<br></br>
A Default Project has been created for you, so just click Copy API Key
</Info>
{/* TODO: Install Tailwind for this stuff */}
</Step>
<Step title="Run your agent">
Execute your program and visit [app.agentops.ai/drilldown](https://app.agentops.ai/drilldown) to observe your Agent! 🕵️

<Tip>After your run, AgentOps prints a clickable url to console linking directly to your session in the Dashboard</Tip>
<div/>{/* Intentionally blank div for newline */}

<Frame type="glass" caption="Clickable link to session">
<img height="200" src="/images/link-to-session.gif" />
</Frame>
</Step>
</Steps>


## More basic functionality

<CardGroup cols={1}>

<Card icon="code" title="Decorate Functions">
You can instrument other functions inside your code with the handy `@record_function`
decorator, which will record an `action_type`, the parameters, and the returns. You
will see these function calls alongside your LLM calls from instantiating the AgentOps client.
Expand All @@ -33,7 +76,8 @@ dashboard along with the full LLM chat histories, cost, token counts, etc.
...
```

## Track Agents
</Card>
<Card icon="robot" title="Track Agents">
If you use specific named agents within your system, you can tie all downstream [Events](/v1/details/events) to a
Named Agent with the `@track_agent` decorator.
```python python
Expand All @@ -43,21 +87,19 @@ dashboard along with the full LLM chat histories, cost, token counts, etc.
...
```

## Ending Your Session

</Card>
<Card icon="stop" title="Ending Your Session">
Finally, you should end your session by calling `.end_session()` with whether your session
was successful or not `(Success|Fail)`. We suggest setting session state depending on how
your agent exits or whether your agent succeeded or not. You can also specify a end state reason,
such as user interrupted, ran to completion, or unhandled exception.
```python python
# End of program
agentops.end_session('Success')
# Woohoo! You're done 🎉
```
</Card>
</CardGroup>

## Check out your dashboard
Run your agent and then visit [`app.agentops.ai`](https://app.agentops.ai).
From the "Session Drill-Down" you should be able to see all of your OpenAI calls!

## Example Code

Expand All @@ -82,7 +124,6 @@ class sample_agent(...):

# End of program
agentops.end_session('Success')
# Woohoo! You're done 🎉
```

<Card
Expand All @@ -93,7 +134,9 @@ agentops.end_session('Success')
Jupyter Notebook with sample code that you can run!
</Card>

That's all you need to get started! Check out the documentation below to see how you can record other events. AgentOps is a lot more powerful this way!
<Check>
That's all you need to get started! Check out the documentation below to see how you can record other events. AgentOps is a lot more powerful this way!
</Check>

## Explore our more advanced functionality!

Expand All @@ -108,4 +151,4 @@ That's all you need to get started! Check out the documentation below to see how
<Card title="Tracking Agents" icon="robot" href="/v1/details/tracking-agents">
Associate Events with specific named agents.
</Card>
</CardGroup>
</CardGroup>

0 comments on commit 27d8311

Please sign in to comment.