Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Anthropic Parallel API Processor #100

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

milistu
Copy link

@milistu milistu commented Sep 27, 2024

Anthropic API Parallel Request Processor

Overview

This project provides a script for making parallel API requests to Anthropic's AI models, optimized for high-volume processing while respecting rate limits.

Key Features

  • Parallel processing of API requests
  • Rate limiting to comply with Anthropic's API guidelines
  • Configurable settings for requests/minute and tokens/minute
  • Optional caching for repeated content
  • Customizable logging levels

Inspiration

It is adapted from OpenAI's API Request Parallel Processor, and modified and improved for Anthropic's API.

Configuration

  • Adjustable rate limits (default: 40 requests/min, 16,000 tokens/min - 80% of Tier 1)
  • Adjustable Logging levels
  • Optional caching for efficiency

Detailed documentation is available in README.

This tool aims to streamline the bulk processing of Anthropic API requests while maintaining efficiency and adhering to usage guidelines. It is perfect for creating/processing large datasets.

Best Regards,
Milutin Studen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant