Privacy and compliance
Apply data requirements to the actual provider route, not only to the model's name.
Sovereign EU
A route counts as sovereign EU only when the whole chain behind it is European. Tokenlane classifies this strictly:
- The provider is an EU company and operates the service itself, without a parent company outside the EU.
- No company from outside the EU sits in the data path, not even one that only supplies infrastructure such as cloud hosting or a CDN. A single such link, for example a US company subject to the US CLOUD Act, excludes the route from
sovereign_eu. - The class is a verified guarantee per provider and can be set per route. A model maker's location does not establish it.
| Class | Guarantee |
|---|---|
sovereign_eu | An EU company and operation without a non-EU company in ownership or data path |
eu_data_residency | EU processing, without the sovereignty guarantee |
global | No EU jurisdiction restriction |
Not sovereign does not mean outside the EU: an eu_data_residency route still processes in the EU, but a company from outside the EU is part of its chain. Routes with an unknown classification never count as sovereign.
How to enforce it:
- Catalog: the Sovereign EU filter is on by default and shows only sovereign routes.
- Rule: set Jurisdiction (
min_compliance: "sovereign_eu") for the organization in Privacy settings, or on a workspace or key. Requests then use sovereign routes only. There is no request field that sets or relaxes the class.
On each model page, a route's detail panel under Providers shows its compliance class.
Zero data retention
Zero data retention (ZDR) means the provider does not store prompts and responses. A route qualifies only when the provider explicitly establishes it; a missing or contradictory claim counts as no ZDR.
- ZDR concerns contents at the provider, not billing and usage metadata.
- It is separate from what Tokenlane stores: usage metadata always, prompt and completion contents only with the key's
captureopt-in, and withIdempotency-Keythe full response, encrypted, for at most ten minutes (see below). Provider ZDR does not enable or disable gateway capture.
| Where | How to require ZDR |
|---|---|
| Catalog | Zero data retention filter |
| Organization, workspace or key | Zero data retention rule (require_zdr), for the organization in Privacy settings |
| Request | provider.zdr: true |
Availability is evaluated for each request. If no route qualifies, the request fails with 404 no_endpoint, naming the failed filter and the issue per route; the gateway never falls back to a route without ZDR. On each model page, the Privacy column under Providers shows whether a route stores request data.
Preserve stricter policies
Organization, workspace, key and request constraints are intersected. A request cannot relax a stricter existing requirement.
Control stored contents
What is stored depends on what you opt into:
- Usage metadata is always recorded for accounting and analysis. Disabling capture does not remove it or replace the upstream provider's data policy.
- Prompt and completion contents in the GenerationStore require the key's explicit
captureopt-in, appropriate permissions and retention. - Idempotent responses: sending
Idempotency-Keyexplicitly opts that request into encrypted storage of its full response in Valkey for at most ten minutes. This permits exact replay within the same API key identity, without another charge; it grants no capture permission and exposes no generation content endpoint. See the idempotency storage and privacy guide. - Optional server tools can send data to third parties and require separate permission.
curl
curl --request POST \
--url 'https://api.tokenlane.de/v1/chat/completions' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
"model": "deepseek/deepseek-v4-flash-0731",
"messages": [
{
"role": "user",
"content": "Hello"
}
],
"max_tokens": 400
}'
node
import OpenAI from 'openai';
const client = new OpenAI({ baseURL: "https://api.tokenlane.de/v1", apiKey: process.env.TOKENLANE_API_KEY });
const response = await client.chat.completions.create({
"model": "deepseek/deepseek-v4-flash-0731",
"messages": [
{
"role": "user",
"content": "Hello"
}
],
"max_tokens": 400
}, {});
console.log(response);
python
import os
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.tokenlane.de/v1", api_key=os.environ["TOKENLANE_API_KEY"])
body = json.loads(r'''{
"model": "deepseek/deepseek-v4-flash-0731",
"messages": [
{
"role": "user",
"content": "Hello"
}
],
"max_tokens": 400
}''')
response = client.chat.completions.create(
model=body["model"],
messages=body["messages"],
max_tokens=body["max_tokens"]
)
print(response)
vercel
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';
const tokenlane = createOpenAICompatible({ name: 'tokenlane', baseURL: "https://api.tokenlane.de/v1", apiKey: process.env.TOKENLANE_API_KEY, includeUsage: true });
const result = await generateText({
model: tokenlane("deepseek/deepseek-v4-flash-0731"),
messages: [
{
"role": "user",
"content": "Hello"
}
],
maxOutputTokens: 400,
});
console.log(result.text);
Use a minimal request and avoid submitting unnecessary personal or confidential data. The example does not enable capture or grant additional permissions.
Route labels describe configured technical guarantees, not a certification of an application's legal compliance.
Organization rules and content capture
Privacy settings use the same rule fields and origins as workspace and key settings. Only organization owners and administrators may save these rules.
- Processing countries, allowed providers, jurisdiction guarantees and provider ZDR are separate constraints.
- The catalog preview does not know price tiers or provider time windows; the gateway decides for each request.
Content capture
- A key's configured capture preference is retained when an organization or workspace rule forbids capture. The console shows the effective result and the blocking level.
- A current prohibition also blocks reading previously captured contents.
- Provider ZDR does not itself enable or disable gateway capture.
- Idempotency replay remains the separate, encrypted request opt-in described above.
- The key editor represents older key-level capture blocks with the single content-capture setting and removes that legacy block on save.
Maximum key lifetime
- Measured from creation, including management keys.
- Setting or tightening it shortens existing expiries, so older keys may expire immediately.
- New keys and policy edits require an expiry within the limit.
- Relaxing or removing the rule never extends an existing expiry.
Export metadata to an owned receiver
Observability settings configure up to five organization-owned destinations. Each has an OTLP/HTTP trace URL with its complete path (or a Langfuse project) plus an optional write-only Authorization header (Bearer or Basic, as required by the receiver), and a scope: all receipts, or only those of up to 50 chosen workspaces or keys. Each destination has its own credentials, headers, consent, retries, counters and status, including its last 20 delivery attempts with outcome and error class. A different scope or URL starts a new period: counters restart and older pending receipts are not sent. Only organization owners and admins manage destinations.
The receiver must accept OTLP/HTTP; proprietary ingestion APIs are not interchangeable with OTLP. New destinations send protobuf (application/x-protobuf); destinations created before this setting keep JSON, and the encoding can be switched per destination. If the receiver answers HTTP 415, receipts stay pending until you switch the encoding. The 512 KB request limit applies to the bytes sent. Test sends an empty OTLP request to the saved destination without enabling export.
Each new usage receipt in a destination's scope becomes one span following the OpenTelemetry GenAI client conventions. Delivery is at least once; a receiver that does not answer is retried for up to 24 hours (honouring Retry-After), then the receipt counts as expired. Prompts and responses are exported only with the content export described below; raw errors and arbitrary upstream fields are never exported. Public HTTPS destinations only; private addresses and redirects are refused.
Extra headers
Receivers with their own key or tenant header take up to four extra headers from a fixed list: x-honeycomb-team, x-honeycomb-dataset, dd-api-key, x-axiom-dataset and x-scope-orgid. Values are printable ASCII up to 4,096 characters, are encrypted like the Authorization header and are never returned or audited; the settings and the audit show only the names. Saving a list replaces all saved headers, an empty list deletes them. Changing the URL needs new headers or an empty list. Authorization stays its own field; Host, Content-*, Cookie, hop-by-hop and Proxy-* headers cannot be set. Langfuse destinations take no extra headers.
| Receiver | Trace URL | Encoding | Headers | Source |
|---|---|---|---|---|
| Honeycomb | https://api.eu1.honeycomb.io/v1/traces (EU), https://api.honeycomb.io/v1/traces (US) | protobuf or JSON | x-honeycomb-team (API key); x-honeycomb-dataset only for Honeycomb Classic | Honeycomb docs |
| Datadog | OTLP trace intake of your Datadog site | protobuf or JSON | dd-api-key | Datadog docs; the exact URL per site is unverified |
| Axiom | https://<your edge domain>/v1/traces | protobuf | Authorization Bearer <token>, x-axiom-dataset | Axiom docs |
| Grafana Tempo (multi-tenant) | Your Tempo OTLP/HTTP URL ending in /v1/traces | protobuf or JSON | x-scope-orgid (tenant) | Tempo docs |
| Grafana Cloud | https://otlp-gateway-<region>.grafana.net/otlp/v1/traces | protobuf or JSON | Authorization Basic <instance:token> | unverified |
Content export
With Content export switched on and the current consent confirmed, spans additionally carry gen_ai.input.messages and gen_ai.output.messages in the OpenTelemetry GenAI message form (role, parts, and finish_reason on the output), encoded as structured values, not JSON strings. Langfuse shows them as the generation's input and output.
- Only content Tokenlane already stores is sent: requests of keys with the
captureopt-in, readable in the console under the same rules. No content is stored for the export. Aforbid_capturerule on the organization, workspace or key that applies at export time sends no content, also for receipts captured earlier. - PII redaction: redaction protects only against the model provider. Tokenlane stores the original text, so for a request that required PII redaction (by a rule or the PII plugin) it sends no content, only the metadata and
gateway.content.withheld=pii_redaction. This also applies when the redaction failed or the request continued without it. All other stored content is the customer's original text. - System messages stay in
gen_ai.input.messagesbecause the chat API carries them inline; the Messages API is translated into this form before it is stored, so itssystemis there too. Tool calls becometool_callparts, tool resultstool_call_responseparts. Parts without a counterpart in the convention are left out and setgateway.content.truncated. - Images, audio and files are never sent as data; a placeholder part keeps only the modality and a declared MIME type.
- At most 64 KB of content per span: at most 256 input messages remain (a leading system message and the newest ones), text parts are cut at 16,384 characters, then the oldest input messages are dropped. Cut or already truncated content carries
gateway.content.truncated. Stored error texts, provider errors and reasoning are never sent. - Consent covers one receiver with its credentials: a new URL, destination kind, Authorization, Langfuse key or region, new extra headers or a new consent text need fresh consent. Keeping the switch on without consenting again returns an error (400,
contentConsentVersion); leaving the switch out of the request turns content export off. A changed consent text stops content export at once; metadata keeps flowing. - After switching off or revoking, at most one page already in flight (up to 50 spans, 512 KB) can still be delivered.
- A receiver outside the EU takes the content outside the EU. Content already delivered stays at the receiver after the export is switched off.
Exported span attributes
The span is named {operation} {requested model} and has kind CLIENT. Attributes appear only when they are known:
gen_ai.operation.name:chatfor chat completions, Responses and Messages,embeddings, or the gateway's own namesrerank,ocr,pii,decision,transcriptionandspeech. The convention allows own values where no fixed value fits. Without a recorded API type the attribute is omitted.gen_ai.provider.namecarries the convention's well-known value where one exists (for examplemistral_ai), otherwise the Tokenlane provider ID;gateway.provider_idalways carries the Tokenlane provider ID. Changed in November 2026: earlier spans carried the Tokenlane provider ID ingen_ai.provider.name.gen_ai.request.model,gen_ai.response.model,gen_ai.response.id(gen-…),gen_ai.response.finish_reasons(string array),gen_ai.request.streamandgen_ai.response.time_to_first_chunk(seconds, streaming only).gen_ai.usage.input_tokens,gen_ai.usage.output_tokens,gen_ai.usage.cache_read.input_tokensandgen_ai.usage.reasoning.output_tokensas integers; embeddings carry no output tokens.error.typewith span status ERROR for failed requests;gateway.cancelledfor cancelled ones.- From your own request:
gen_ai.conversation.id(session),user.id(theuserfield),gateway.app.title,gateway.trace.<name>for trace fields andgateway.metadata.<key>for metadata, at most 16 pairs. Your trace and metadata values are kept up to 512 characters, the request limit; other text is shortened to 256 characters. Keys with characters other than letters, digits,_,.and-are not exported. gateway.cost.amountas an exact decimal string withgateway.cost.currencyEUR, plusgateway.key.idandgateway.workspace.id.
Purchase prices, provider attempts, exchange-rate details, routing internals and timing phases are never exported.
Caller trace: If your application sends a valid W3C traceparent (OpenTelemetry SDKs with HTTP instrumentation do this automatically) and Join the calling trace is on (the default for OTLP destinations), the span carries your trace ID, your calling span as parentSpanId and your tracestate. The span ID is always the low 64 bits of the generation ID from x-generation-id. Without a valid header, or with the switch off, each receipt is its own trace. An invalid header never causes an error.
Langfuse: Langfuse destinations start with Join the calling trace off, so each generation is the root of its own Langfuse trace and Langfuse shows it completely; switch it on only if your application sends its own spans to the same Langfuse project. Existing Langfuse destinations were switched off once. Spans to Langfuse additionally carry langfuse.observation.type (generation, embedding for embeddings, otherwise span), langfuse.observation.model.name, langfuse.observation.completion_start_time (streaming only), langfuse.observation.level (ERROR with langfuse.observation.status_message, WARNING when cancelled), langfuse.trace.name (your trace.trace_name, otherwise the span name), langfuse.user.id, langfuse.session.id, langfuse.trace.tags (only when trace.tags is a list of strings), langfuse.trace.metadata.<key> for your metadata and langfuse.environment from metadata.environment (lower case, at most 40 characters). Generic OTLP destinations never receive langfuse.* keys.
Change: gateway.cost.pico_eur has been replaced by gateway.cost.amount and gateway.cost.currency, without a transition period.
Delivery and retries
Delivery runs outside inference in batches of at most 50 receipts.
- Full acceptance: The receiver accepted every span. A warning with zero rejected spans is also a success.
- Partial acceptance: The HTTP 200 reply is final. This batch is not sent again; later receipts continue.
- Temporary failure: Network failures and HTTP 429/502/503/504 are retried for up to 24 hours after receipt arrival. Exponential backoff includes jitter and honors Retry-After. Longer receiver delays are respected; receipts that have expired meanwhile are then settled without another send.
- Permanent rejection: Other error responses are not retried and do not block later receipts.
Enabling export or changing the URL starts a new export period without historical backfill. Disabling stops future deliveries. Saving takes over a running export at once: at most the batch already in flight is still delivered. A test does not start while another test or export is running; retry afterward.
Delivery totals
Totals distinguish accepted, rejected and unconfirmed receipts from the displayed accounting start. Expired receipts (24 hours without delivery) and oversized spans that alone exceed the 512 KB request limit are counted separately; larger batches are split before sending. Invalid or oversized replies remain unconfirmed. A receipt from which Tokenlane cannot build a span is not sent and is counted separately as a local error, not as a rejection by the receiver; when no receipt of a full batch can be built, the receipts stay pending and the export retries. The settings also show the pending backlog (counted up to 10,000, or unknown if the count does not finish in time) and the oldest pending receipt. Receipts are also counted as expired when the destination stays unreachable or blocked for longer, for example with an unreadable credential. A disabled destination does not expire receipts, and receipts older than 6 days drop out of the export without being counted. A test changes neither the totals nor the destination's status, retry timing or attempt history.
A partial reply supplies counts, not identities. The entire batch is therefore settled, but individual receipts are not labelled accepted.
Receipt settlement, totals and scheduling commit together under the destination revision and lease. An expired owner cannot overwrite a newer pass.
An acknowledgement lost before that commit can lead to a repeated stable trace/span identity. Receivers should deduplicate by that identity; this is not exactly-once delivery. Delivery is at least once: saving the settings while a page is in flight can send that page once more, because the superseded pass may not record its result.
Credentials
Credentials are encrypted at rest and are never returned or audited. Operators configure OBSERVABILITY_KEYS as a comma-separated list of kid:base64-32-bytes and select OBSERVABILITY_ACTIVE_KEY in the web container's secrets/web.env (infra/mvp/compose.yml, env_file). To rotate, retain A in the ring, add B and activate B on all web replicas. New credentials use B; existing A and legacy credentials remain readable and are re-encrypted in batches of at most 50 per worker pass, including disabled destinations. Wait until all replicas use B and observability_credentials_stale is 0 before removing A. Keep AUTH_SECRET unchanged until legacy credentials have migrated. Without OBSERVABILITY_KEYS, the existing AUTH_SECRET-based encryption remains in use. The tenant destination does not change the operator's gateway collector.
Data sources and licenses
Client-IP country estimates use IP to Country Lite by DB-IP, licensed under CC BY 4.0. We retain only the country derived by the gateway's configured IP resolver; estimates may be inaccurate and do not establish a provider's processing location. The source link is available in the country value's help and in the imprint.