LLM Gateway

Unified Python SDK and proxy server that calls 100+ LLM providers using the OpenAI API format — drop-in router with cost tracking, fallbacks, and load balancing.

Official OpenAI Python client for the OpenAI API — chat completions, embeddings, images, audio, and the new Responses + Assistants APIs.

Official Anthropic Python client for the Claude API — messages, streaming, tool use, prompt caching, and AWS Bedrock / Vertex AI integration.

Structured outputs from LLMs using Pydantic — patches OpenAI/Anthropic/Cohere/etc. clients to return validated typed objects with retries.