Rate limits & budgets

fremai is prepaid: you hold a credit balance, and usage is debited against it on the backend-reported token counts in each response. Budgets and rate limits are enforced at four nested scopes so you can bound both spend and throughput.

Scopes

Budgets are enforced top-down: the tenant cap bounds group budgets, which bound user budgets, with optional per-key sub-limits.

Per-key controls

Each key can be scoped on:

• Model subset — which catalog models the key may call.
• Budget + reset cadence — an absolute spend cap that resets on a schedule.
• Rate limits — requests per minute (RPM) and tokens per minute (TPM).
• Concurrency — maximum parallel requests.
• Expiry / TTL — keys can be time-bounded.
• Cost-center tag — for attribution across keys.

This lets a narrow CI key (few models, low budget, no burst) and a full interactive key (all models, higher concurrency) be distinct profiles off the same user.

Budgets and alerts

• Prepaid credit — when the balance is exhausted or a budget cap is reached, requests return 402 (see errors). Top up in the console.
• Threshold alerts — configurable notifications at usage thresholds so you are warned before a cap is hit.
• Budget-exceeded behaviour — depending on configuration, a scope that hits its cap can block, alert only, or degrade.

Rate and concurrency limiting

When a rate, token, or concurrency limit is exceeded, the API returns 429 with a Retry-After header (seconds). Back off for that interval and retry:

HTTP/1.1 429 Too Many Requests
Retry-After: 5

The standard OpenAI SDKs already implement Retry-After-aware retries; if you roll your own client, honour the header.

Where to configure

Budgets, limits, key scopes, alerts, and team membership are all managed in the console at app.fremai.eu. The console is the system of record.