Multi-Tenant LLM Apps: Per-Customer Keys, Quotas, and Cost Attribution

Multi-tenant support in an LLM product comes down to four questions: who is calling (credentials), how much can they use (quotas), who pays for what (cost attribution), and who can see whose data (isolation). Platforms hand you primitives for the first three. The fourth one, data isolation, is entirely yours: no quota setting or header will stop one tenant's documents from appearing in another tenant's search results. This article maps the platform primitives onto the four questions, spends the most time on the two cost-related failure modes that actually take production down, and ends with the checklist this site applies to tenant work.

Start by scoping your tenant

Before touching dashboards, decide what a tenant is in your product: a paying customer, a team inside one customer, or an environment of your own? The mapping matters because platform objects are for isolating your application instances and environments, not for modeling every customer one-to-one. Keep the tenant-to-instance mapping in your own service layer, and you stay free to move a tenant between instances, merge two small tenants onto one, or shard a large one without reorganizing your platform account.

If you do go one-platform-object-per-tenant, be aware you inherit a ceiling you did not choose. Anthropic documents a default maximum of 100 workspaces per organization (archived workspaces do not count, and the docs note you can contact your account team for more; check the official page for the current limit). That ceiling is fine for environments and major product lines; it is not fine for a signup flow. Which is why the mapping lives in your database, not in your org chart of workspaces.

Credentials: one key per tenant, or per tenant group

The default anti-pattern is one key shared by every tenant: it makes the blast radius of a leak equal to your whole business, makes per-tenant revocation impossible, and makes attribution guesswork. Give each tenant, or each group of tenants with the same trust level, its own credential, scoped as narrowly as the platform allows. The full hygiene playbook, including rotation and what to do when you suspect a leak, is in the key hygiene guide.

On OpenAI, the natural unit is the project API key, and administrators can enforce a maximum key lifetime at the organization or project level. On Anthropic, keys can be scoped to a single workspace: a workspace key can only access resources within that workspace, and when you create additional workspaces you can assign members, service accounts, API keys, and resource limits to each one. Some keys can span multiple workspaces, in which case the workspace is named per request rather than baked into the key.

Whichever platform sits behind your endpoint, verify which context a request actually landed in rather than trusting configuration. Anthropic API responses include an anthropic-workspace-id header, whose value is the workspace ID that the request's API key or access token resolved to, and the documentation notes it appears even when that workspace is the Default Workspace. Log that header per request and you can reconstruct, after the fact, which tenant's context served any given call.

Quotas: two layers, two different control surfaces

Quota work has an organization layer and a workspace layer. As the Anthropic rate limits documentation puts it: "The API enforces service-configured limits at the organization level, but you may also set user-configurable limits for your organization's workspaces." In other words, the platform enforces a floor; you enforce the tenants' individual shares on top of it.

The trap is assuming both numbers live in the same place. They do not, and the documentation says so plainly: "You set a workspace's rate limits and spend limits in different places." Rate limits are configured per workspace, per model, from the workspace selector on the Rate limits page, covering requests per minute, input tokens, or output tokens. Spend limits are configured from the Workspaces page, in the workspace's panel. Configure one and forget the other, and you have either an unenforced budget or an unmetered tenant.

On the OpenAI side, the same two layers exist in a different shape: project rate limits and spend alerts are manageable programmatically through the platform's admin endpoints for projects, and production best practices describes setting spend alerts on the limits page so notifications fire when usage crosses a threshold, with hard spend limits as the enforcement step (read the spend-limits guidance before enabling hard caps in production). What you should not do is quote today's limits as a plan: limits move, and per the standard rule, always confirm against the official documentation current when you deploy.

Two kinds of "you cannot use the API"

This is the section worth the read. When a tenant goes over its allowance, the platform can fail in two very different ways, and a client that cannot tell them apart will either retry forever or give up too early.

The first failure is a rate limit: a 429 that describes which limit was exceeded, and, when present, carries a retry-after header telling you how long to wait. Back off and retry; that is the retry budget discipline this site has covered before.

The second failure is a spend limit. It looks like a rate limit and is not one. The error type is still rate_limit_error, but the response has no retry-after header, and on the Messages API the error.details.error_code field reads enforced_spend_limit_reached. The documentation is explicit about the operational consequence: retrying, including the SDK's automatic retries, fails until access resumes. There is no wait long enough, no exponent large enough; the budget is gone.

Your client code has to branch on this. Anything carrying enforced_spend_limit_reached, or a spend-shaped 429 with no retry-after, should stop the retry loop, surface an actionable alert to the tenant and your team, and let your error triage process decide what happens next. Treating it as a transient rate limit is how one tenant's exhausted budget turns into a wall of pointless requests.

Cost attribution: who actually spent it

Attribution asks a simple question with a fiddly API answer: for each tenant, how much did we spend? On Anthropic, the Usage and Cost Admin API is the tool, with report parameters that line up with tenant accounting: group_by[] to break results down (for example by model), bucket_width to pick day or hour granularity, and filters for specific API keys or workspaces. From the usage and cost documentation: the endpoints support granular, programmatic access to historical usage and cost data for your organization.

The credentials requirement is the part that trips people, and the documentation flags it directly. These endpoints do not accept workspace-scoped keys. You need an Admin API key, an OAuth token with the org:admin scope, or a personal or service account key that is not scoped to a workspace. The page also notes that the Admin API is unavailable for individual accounts. Plan for a dedicated reporting credential that is deliberately not one of your tenant keys, and store it with the highest protection you have.

One attribution nuance worth designing around: usage and cost reports show a null workspace_id for the Default Workspace and for keys that span all workspaces, while a key's scope field distinguishes them. If you plan to bill per workspace, make sure the traffic you care about actually runs under workspace-scoped keys, or your reports will have an unlabeled bucket.

On the OpenAI side, attribution has a request-header story: the API reference's request headers section shows requests carrying OpenAI-Organization and OpenAI-Project headers, and notes that usage from these requests counts as usage for the specified organization and project. If your architecture forwards tenant traffic through shared infrastructure, make sure those headers reflect the tenant's designated project, or everything lands in one bucket. The broader instrumentation discipline, including which fields to record per call, is in the usage tracking guide.

Data isolation: the part the platform does not do

Now the fourth question. Everything above is about money and access; isolation is about content, and no platform can enforce your data model for you. The platform can isolate files, batches, and caches per workspace, but it cannot know that tenant A's support transcripts must never be retrieved for tenant B's query. That boundary lives in your stack:

• Storage partitioning. Every row that contains tenant content carries a tenant identifier, and every query filters by it at the data layer, not the application layer where a forgotten where clause is one refactor away.
• Log partitioning. Logs carry the tenant identifier as a first-class field, so debugging, exports, and retention policies can be scoped per tenant. The field list this site recommends, and the fields to keep out of logs entirely, is in the observability guide.
• Cache key prefixes. Cached completions, embeddings, and retrieved context must be keyed with the tenant identity in the prefix. A shared cache without tenant scoping is a cross-tenant data leak with a performance budget.
• Templates and configuration. Prompt templates, model settings, and feature flags are scoped per tenant, and changing one tenant's configuration cannot bleed into another's.
• Export and delete paths. When a tenant exports or deletes its data, the operation is verified against the tenant boundary, not run by an admin script that "should be fine."

Failure patterns to design against

The same failures recur in multi-tenant reviews, so design them out deliberately:

1. One key shared by every tenant, making revocation and attribution all-or-nothing.
2. Limits configured only at the organization level, so one noisy tenant can consume everyone's headroom.
3. Treating a spend-limit stop as a retryable rate limit, and burning effort against a budget that will not resume on its own schedule.
4. Using workspace-scoped keys to call the usage and cost endpoints, which the documentation says will not work.
5. Isolating platform resources while your own database, cache, and logs stay global.
6. No per-tenant reconciliation: the invoice says one thing, the usage report says another, and nobody can explain the delta.

The checklist

1. Define the tenant boundary, and keep the tenant-to-instance mapping in your own service layer.
2. Issue credentials per tenant or tenant group, scoped as narrowly as the platform allows.
3. Configure quotas at both layers, on both control surfaces, and document where each lives.
4. Branch client behavior on the two failure types: retry budgets for rate limits, alerts for spend stops.
5. Produce per-tenant cost reports with a dedicated, non-tenant reporting credential.
6. Partition your own storage, caches, and indexes by tenant.
7. Stamp every log line with the tenant identifier.
8. Rehearse rotation and revocation for a single tenant without touching the others.
9. Add per-tenant alerting, so the first signal is a notification, not a support ticket.

Four questions, three platform primitives, one job that stays yours. Get the mapping and the failure branching right, and multi-tenancy stops being an architecture problem and becomes a billing detail. Endpoint-level specifics for whatever compatible service you call live in the platform's own documentation, and as always, verify every quoted behavior against the official pages current at the time you build.