Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

separated llm tracker by model #346

Merged
merged 29 commits into from
Aug 20, 2024
Merged

separated llm tracker by model #346

merged 29 commits into from
Aug 20, 2024

Conversation

bboynton97
Copy link
Contributor

Each LLM now has functions in a different file. I will probably continue this to use inheritance and consistent function names

@bboynton97 bboynton97 requested a review from siyangqiu August 13, 2024 22:24
Copy link

gitguardian bot commented Aug 14, 2024

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
12083425 Triggered Generic High Entropy Secret 4bf8ba1 tests/core_manual_tests/agentchat_agentops.ipynb View secret
12083425 Triggered Generic High Entropy Secret 4bf8ba1 tests/core_manual_tests/agentchat_agentops.ipynb View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Copy link

codecov bot commented Aug 14, 2024

Codecov Report

Attention: Patch coverage is 20.38288% with 707 lines in your changes missing coverage. Please review.

Files Patch % Lines
agentops/llms/anthropic.py 12.66% 131 Missing ⚠️
agentops/llms/cohere.py 14.09% 128 Missing ⚠️
agentops/llms/openai.py 13.98% 123 Missing ⚠️
agentops/llms/litellm.py 15.32% 116 Missing ⚠️
agentops/llms/groq.py 16.66% 90 Missing ⚠️
agentops/llms/ollama.py 25.31% 59 Missing ⚠️
agentops/llms/__init__.py 30.66% 51 Missing and 1 partial ⚠️
agentops/llms/instrumented_provider.py 69.23% 8 Missing ⚠️
Flag Coverage Δ
unittests 43.54% <20.38%> (-0.98%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
agentops/client.py 68.55% <100.00%> (ø)
agentops/event.py 92.00% <ø> (ø)
agentops/helpers.py 65.93% <ø> (-6.14%) ⬇️
agentops/singleton.py 100.00% <100.00%> (ø)
agentops/time_travel.py 32.07% <100.00%> (ø)
agentops/llms/instrumented_provider.py 69.23% <69.23%> (ø)
agentops/llms/__init__.py 30.66% <30.66%> (ø)
agentops/llms/ollama.py 25.31% <25.31%> (ø)
agentops/llms/groq.py 16.66% <16.66%> (ø)
agentops/llms/litellm.py 15.32% <15.32%> (ø)
... and 3 more

@bboynton97 bboynton97 requested a review from HowieG August 14, 2024 22:47
agentops/llms/__init__.py Outdated Show resolved Hide resolved
agentops/llms/__init__.py Show resolved Hide resolved
agentops/llms/__init__.py Outdated Show resolved Hide resolved
@siyangqiu
Copy link
Contributor

I ran a core manual test and it worked

the-praxs and others added 6 commits August 19, 2024 15:19
* fix typo

* add anthropic support

* add example file

* fixed linting using black

* add time travel support for anthropic

* linting

* fix kwargs key to get prompt

* fix for extracting tokens from Message.usage

* minor fix for output tokens

* remove anthropic example python file

* fix completions not show in session

* some more fixes and cleanup

* add Message object to pydantic models

* fix typo

* overhaul anthropic code

* linting

* add anthropic example notebook

* linting

* added readme examples

* fix incorrect attribute access for the model content

* add async example

* refactor code

* fix function name

* linting

* add provider tests

---------

Co-authored-by: reibs <[email protected]>
…andler-refactor

# Conflicts:
#	agentops/llms/__init__.py
@bboynton97 bboynton97 merged commit caaacc3 into main Aug 20, 2024
11 of 13 checks passed
@bboynton97 bboynton97 deleted the llm-handler-refactor branch August 20, 2024 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants