Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open Telemetry V2 #581

Closed
wants to merge 44 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
c7c07b0
docs: add current OpenTelemetry design documentation
teocns Dec 11, 2024
aee7b10
docs: add entity mapping example for OpenTelemetry concepts
teocns Dec 13, 2024
e6ce443
`teleemtry` package base with OTELConfig export
teocns Dec 13, 2024
87f8fe2
examples/open-telemetry
teocns Jan 7, 2025
37a5858
save
teocns Jan 7, 2025
263948c
chore: rename logging.py to log_handler.py
teocns Jan 7, 2025
95c9542
docs(log_config): update logging module documentation
teocns Jan 7, 2025
eda1e20
fix converter
teocns Jan 7, 2025
366b9ed
feat(logging): add function to configure custom logger
teocns Jan 7, 2025
aa2823d
feat(telemetry): add event to OpenTelemetry span converter
teocns Jan 7, 2025
98e6bd8
Streamline tests with shareable fixtures and simpler configuration
teocns Jan 7, 2025
fb11e95
alternative implementation with AgentOpsSpanAttributes
teocns Jan 7, 2025
5fd6131
refactor(client, session): optional ClientTelemetry, default retrieva…
teocns Jan 7, 2025
a494851
feat(telemetry): add configure method for OTELManager
teocns Jan 7, 2025
aa7488f
fix(exporter): update endpoint URL for event creation
teocns Jan 7, 2025
871f3ad
save rly quik
teocns Jan 7, 2025
b5c7bda
feat(telemetry): standardize use of AgentOpsAttributes
teocns Jan 7, 2025
fa19ce8
sav
teocns Jan 7, 2025
8a196a3
refactor(event): ErrorEvent to subclass Event
teocns Jan 7, 2025
65f0d04
refactor(session): simplify event handling in spans
teocns Jan 7, 2025
549fe27
session: towards OTEL standardization
teocns Jan 7, 2025
6a6f71c
refactor(session): simplify imports and add locks
teocns Jan 7, 2025
de5554e
remove otel docs from sessino
teocns Jan 7, 2025
c100e1e
boy
teocns Jan 7, 2025
77e2d49
feat(event): add session_id to Event class and methods
teocns Jan 7, 2025
979be2b
feat(telemetry): add force flush method and improve config handling
teocns Jan 7, 2025
39b8350
build: add opentelemetry exporter dependency for grpc
teocns Jan 7, 2025
5dfd96f
Clearer separation between OTELConfig and Configuration
teocns Jan 7, 2025
f989ef4
test: add unit tests for telemetry configuration functionality
teocns Jan 7, 2025
d2e319f
feat: rename otel: OTELConfig to telemetry in configuration
teocns Jan 7, 2025
e4b7bf1
refactor(telemetry): correct config accessor
teocns Jan 7, 2025
2447b80
refactor(client): pass client to ClientTelemetry constructor
teocns Jan 7, 2025
983e36e
chore: rename telemetry test file for better structure
teocns Jan 7, 2025
1689c8b
OTELConfig: add redundant fields from Configuration (remove later)
teocns Jan 7, 2025
935f578
fix converter to pass all tests
teocns Jan 7, 2025
60a4f61
delete docs
teocns Jan 7, 2025
e12231a
feat(telemetry): add error event handling in processor
teocns Jan 7, 2025
d12b869
test: add test for telemetry config with env variable
teocns Jan 7, 2025
3ddbbae
change the way we test instrumentation
teocns Jan 7, 2025
8cde6d9
test: add instrumentation testing utilities and cleanup fixture
teocns Jan 7, 2025
42adc09
change the way ClientTelemetry is initialized
teocns Jan 7, 2025
4fd051a
use `_span_processor` instead of `_otel_exporter`
the-praxs Jan 7, 2025
d9ad03f
Merge branch 'main' into otel/v2
the-praxs Jan 8, 2025
a4389cd
Merge branch 'main' into otel/v2
the-praxs Jan 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions agentops/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
import sys
from typing import Optional, List, Union

from agentops.telemetry.config import OTELConfig

from .client import Client
from .event import Event, ActionEvent, LLMEvent, ToolEvent, ErrorEvent
from .decorators import record_action, track_agent, record_tool, record_function
Expand Down Expand Up @@ -48,6 +50,7 @@ def init(
auto_start_session: Optional[bool] = None,
inherited_session_id: Optional[str] = None,
skip_auto_end_session: Optional[bool] = None,
telemetry: Optional[OTELConfig] = None, # OTEL configuration
) -> Union[Session, None]:
"""
Initializes the AgentOps singleton pattern.
Expand All @@ -69,6 +72,8 @@ def init(
inherited_session_id (optional, str): Init Agentops with an existing Session
skip_auto_end_session (optional, bool): Don't automatically end session based on your framework's decision-making
(i.e. Crew determining when tasks are complete and ending the session)
exporters (List[SpanExporter], optional): Additional OpenTelemetry exporters for sending
telemetry data to external systems.
Attributes:
"""
Client().unsuppress_logs()
Expand All @@ -84,6 +89,7 @@ def init(
if default_tags is None:
default_tags = tags

# Create OTEL config if exporters provided
Client().configure(
api_key=api_key,
parent_key=parent_key,
Expand All @@ -94,6 +100,7 @@ def init(
instrument_llm_calls=instrument_llm_calls,
auto_start_session=auto_start_session,
skip_auto_end_session=skip_auto_end_session,
otel=telemetry, # Pass OTEL config through
)

if inherited_session_id is not None:
Expand Down
53 changes: 31 additions & 22 deletions agentops/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@

from termcolor import colored

from agentops.telemetry.config import OTELConfig

from .config import Configuration
from .event import ErrorEvent, Event
from .host_env import get_host_env
Expand All @@ -28,10 +30,16 @@
from .meta_client import MetaClient
from .session import Session, active_sessions
from .singleton import conditional_singleton
from .telemetry import ClientTelemetry


@conditional_singleton
class Client(metaclass=MetaClient):
"""
This is the AgentOps core Client.
It's the entrypoint to all core functionality.
"""

def __init__(self):
self._pre_init_messages: List[str] = []
self._initialized: bool = False
Expand All @@ -40,6 +48,7 @@ def __init__(self):
self._config = Configuration()
self._pre_init_queue = {"agents": []}
self._host_env = None # Cache host env data
self.telemetry = ClientTelemetry(self)

self.configure(
api_key=os.environ.get("AGENTOPS_API_KEY"),
Expand All @@ -60,6 +69,7 @@ def configure(
auto_start_session: Optional[bool] = None,
skip_auto_end_session: Optional[bool] = None,
env_data_opt_out: Optional[bool] = None,
otel: Optional[OTELConfig] = None,
):
if self.has_sessions:
return logger.warning(
Expand All @@ -78,36 +88,34 @@ def configure(
auto_start_session=auto_start_session,
skip_auto_end_session=skip_auto_end_session,
env_data_opt_out=env_data_opt_out,
telemetry=otel,
)

def initialize(self) -> Union[Session, None]:
if self.is_initialized:
return

self.unsuppress_logs()
if self._config.api_key is None:
return logger.error(
"Could not initialize AgentOps client - API Key is missing."
+ "\n\t Find your API key at https://app.agentops.ai/settings/projects"
)
"""Initialize the client"""
if not self.is_initialized:
self.unsuppress_logs()
if self._config.api_key is None:
return logger.error(
"Could not initialize AgentOps client - API Key is missing."
+ "\n\t Find your API key at https://app.agentops.ai/settings/projects"
)

self._handle_unclean_exits()
self._initialized = True
self._handle_unclean_exits()
self._initialized = True

if self._config.instrument_llm_calls:
self._llm_tracker = LlmTracker(self)
self._llm_tracker.override_api()
if self._config.instrument_llm_calls:
self._llm_tracker = LlmTracker(self)
self._llm_tracker.override_api()

session = None
if self._config.auto_start_session:
session = self.start_session()
# Initialize telemetry with configuration
self.telemetry.initialize(self._config.telemetry)

if session:
for agent_args in self._pre_init_queue["agents"]:
session.create_agent(name=agent_args["name"], agent_id=agent_args["agent_id"])
self._pre_init_queue["agents"] = []
session = None
if self._config.auto_start_session:
session = self.start_session()

return session
return session

def _initialize_autogen_logger(self) -> None:
try:
Expand Down Expand Up @@ -224,6 +232,7 @@ def start_session(
session_tags.update(tags)

session = Session(
client=self.telemetry,
session_id=session_id,
tags=list(session_tags),
host_env=self.host_env,
Expand Down
7 changes: 7 additions & 0 deletions agentops/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from uuid import UUID

from .log_config import logger
from .telemetry.config import OTELConfig


class Configuration:
Expand All @@ -16,6 +17,7 @@ def __init__(self):
self.auto_start_session: bool = True
self.skip_auto_end_session: bool = False
self.env_data_opt_out: bool = False
self.telemetry: OTELConfig = OTELConfig() # Default OTEL configuration

def configure(
self,
Expand All @@ -30,6 +32,7 @@ def configure(
auto_start_session: Optional[bool] = None,
skip_auto_end_session: Optional[bool] = None,
env_data_opt_out: Optional[bool] = None,
telemetry: Optional[OTELConfig] = None, # New parameter
):
if api_key is not None:
try:
Expand Down Expand Up @@ -72,3 +75,7 @@ def configure(

if env_data_opt_out is not None:
self.env_data_opt_out = env_data_opt_out

# OTEL configuration
if telemetry is not None:
self.telemetry = telemetry
23 changes: 17 additions & 6 deletions agentops/event.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class Event:
end_timestamp(str): A timestamp indicating when the event ended. Defaults to the time when this Event was instantiated.
agent_id(UUID, optional): The unique identifier of the agent that triggered the event.
id(UUID): A unique identifier for the event. Defaults to a new UUID.
session_id(UUID, optional): The unique identifier of the session that the event belongs to.

foo(x=1) {
...
Expand All @@ -43,6 +44,7 @@ class Event:
end_timestamp: Optional[str] = None
agent_id: Optional[UUID] = field(default_factory=check_call_stack_for_agent_id)
id: UUID = field(default_factory=uuid4)
session_id: Optional[UUID] = None


@dataclass
Expand Down Expand Up @@ -105,7 +107,7 @@ class ToolEvent(Event):


@dataclass
class ErrorEvent:
class ErrorEvent(Event):
"""
For recording any errors e.g. ones related to agent execution

Expand All @@ -115,21 +117,30 @@ class ErrorEvent:
code(str, optional): A code that can be used to identify the error e.g. 501.
details(str, optional): Detailed information about the error.
logs(str, optional): For detailed information/logging related to the error.
timestamp(str): A timestamp indicating when the error occurred. Defaults to the time when this ErrorEvent was instantiated.

"""

# Inherit common Event fields
event_type: str = field(default=EventType.ERROR.value)

# Error-specific fields
trigger_event: Optional[Event] = None
exception: Optional[BaseException] = None
error_type: Optional[str] = None
code: Optional[str] = None
details: Optional[Union[str, Dict[str, str]]] = None
logs: Optional[str] = field(default_factory=traceback.format_exc)
timestamp: str = field(default_factory=get_ISO_time)

def __post_init__(self):
self.event_type = EventType.ERROR.value
"""Process exception if provided"""
if self.exception:
self.error_type = self.error_type or type(self.exception).__name__
self.details = self.details or str(self.exception)
self.exception = None # removes exception from serialization

# Ensure end timestamp is set
if not self.end_timestamp:
self.end_timestamp = get_ISO_time()

@property
def timestamp(self) -> str:
"""Maintain backward compatibility with old code expecting timestamp"""
return self.init_timestamp
Loading
Loading