API Reference

This section provides detailed documentation for the BorgLLM API.

Core Classes

BorgLLM Configuration

class borgllm.borgllm.BorgLLM(*args, **kwargs)

Bases: object

property config: LLMConfig | None

Public property to access the configuration.

get(name: str | None = None, approximate_tokens: int | None = None, timeout: float | None = None, allow_await_cooldown: bool = True) → LLMProviderConfig

get_cooldown_duration(provider_name: str) → int

Get the cooldown duration for a specific provider.

Parameters:

provider_name – Name of the provider (may include model, e.g., “openai:gpt-4o”)

Returns:

Cooldown duration in seconds

classmethod get_instance(config_path: str = 'borg.yaml', initial_config_data: Dict | None = None)

Get the singleton BorgLLM instance.

get_timeout_duration(provider_name: str) → float | None

Get the timeout duration for a specific provider.

Parameters:

provider_name – Name of the provider (may include model, e.g., “openai:gpt-4o”)

Returns:

Timeout duration in seconds, or None if no timeout is configured

property providers: Dict[str, LLMProviderConfig]

Public property to access the real providers.

set_cooldown_config(cooldown: int | Dict[str, int])

Set the cooldown configuration for providers.

Parameters:

cooldown – Either a global cooldown duration in seconds, or a dict mapping provider names to specific cooldown durations. The dict can include a “default” key for providers not explicitly listed.

set_default_provider(provider_name: str)

Set the default LLM provider name for this BorgLLM instance.

set_timeout_config(timeout: float | Dict[str, float])

Set the timeout configuration for providers.

Parameters:

timeout – Either a global timeout duration in seconds, or a dict mapping provider names to specific timeout durations. The dict can include a “default” key for providers not explicitly listed.

signal_429(provider_name: str, duration: int | None = None)

Signal that a provider received a 429 error and should be put on cooldown.

Parameters:

• provider_name – Name of the provider that received the 429 error

• duration – Optional specific cooldown duration in seconds. If not provided, uses the configured cooldown duration for this provider.

OpenAI SDK Clients

The primary way to use BorgLLM. These classes are drop-in replacements for the official OpenAI SDK.

OpenAI client integration for BorgLLM.

This module provides BorgOpenAI and BorgAsyncOpenAI clients that are drop-in replacements for openai.OpenAI and openai.AsyncOpenAI, with automatic: - Provider resolution from model IDs (e.g., “openai:gpt-4o”, “google:gemini-2.5-flash”) - Rate limit handling with automatic retry - API key rotation - Virtual provider support - Cooldown management

class borgllm.openai.BorgAsyncOpenAI(config_file: str = 'borg.yaml', initial_config_data: Dict[str, Any] | None = None, overrides: Dict[str, Any] | None = None, cooldown: int | Dict[str, int] | None = None, timeout: float | Dict[str, float] | None = None, max_retries: int = 10, **kwargs)

Bases: object

Drop-in replacement for openai.AsyncOpenAI with BorgLLM integration.

Automatically handles:

• Provider resolution from model IDs (e.g., “openai:gpt-4o”)

• Rate limit detection and retry

• API key rotation

• Virtual provider support

• Cooldown management

Example:

from borgllm import BorgAsyncOpenAI

client = BorgAsyncOpenAI()
response = await client.chat.completions.create(
    model="openai:gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

property chat: BorgAsyncChat

Access the chat completions API.

property responses: BorgAsyncResponses

Access the responses API.

class borgllm.openai.BorgOpenAI(config_file: str = 'borg.yaml', initial_config_data: Dict[str, Any] | None = None, overrides: Dict[str, Any] | None = None, cooldown: int | Dict[str, int] | None = None, timeout: float | Dict[str, float] | None = None, max_retries: int = 10, **kwargs)

Bases: object

Drop-in replacement for openai.OpenAI with BorgLLM integration.

Automatically handles:

• Provider resolution from model IDs (e.g., “openai:gpt-4o”)

• Rate limit detection and retry

• API key rotation

• Virtual provider support

• Cooldown management

Example:

from borgllm import BorgOpenAI

client = BorgOpenAI()
response = client.chat.completions.create(
    model="openai:gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

property chat: BorgChat

Access the chat completions API.

property responses: BorgResponses

Access the responses API.

LangChain Integration (Optional)

Note

Requires pip install borgllm[langchain]

Note

The BorgLLMLangChainClient class is a low-level component typically not manually initialized. Use the create_llm convenience function instead.