Direct PII redaction
Detect and mask personal information with a normal routed PII model.
When to use it
Use the direct redaction endpoint when you need masked text and detection spans without an inference response. To scan chat input automatically, use the PII redaction plugin.
Detection processes the original text at the selected provider and is not proof of anonymization.
Minimal example
This request uses a synthetic reserved email address and requests email detection only.
curl
curl --request POST \
--url 'https://api.tokenlane.de/v1/pii/redact' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
"model": "grunden/gliner-multi-pii",
"text": "Contact lena@example.invalid",
"entities": [
"email"
],
"threshold": 0.5
}'
node
const response = await fetch("https://api.tokenlane.de/v1/pii/redact", {
method: "POST",
headers: {
"Authorization": "Bearer " + process.env.TOKENLANE_API_KEY,
"Content-Type": "application/json",
},
body: `{
"model": "grunden/gliner-multi-pii",
"text": "Contact lena@example.invalid",
"entities": [
"email"
],
"threshold": 0.5
}`,
});
console.log(await response.text());
python
import os
import urllib.request
body = (r'''{
"model": "grunden/gliner-multi-pii",
"text": "Contact lena@example.invalid",
"entities": [
"email"
],
"threshold": 0.5
}''').encode("utf-8")
request = urllib.request.Request(
"https://api.tokenlane.de/v1/pii/redact",
method="POST",
headers={
"Authorization": "Bearer " + os.environ["TOKENLANE_API_KEY"],
"Content-Type": "application/json",
},
data=body,
)
with urllib.request.urlopen(request) as response:
print(response.read().decode("utf-8"))
vercel
// This operation uses HTTP directly.
const response = await fetch("https://api.tokenlane.de/v1/pii/redact", {
method: "POST",
headers: {
"Authorization": "Bearer " + process.env.TOKENLANE_API_KEY,
"Content-Type": "application/json",
},
body: `{
"model": "grunden/gliner-multi-pii",
"text": "Contact lena@example.invalid",
"entities": [
"email"
],
"threshold": 0.5
}`,
});
console.log(await response.text());
Request format
Send JSON to POST /v1/pii/redact with text and a current PII model. The response contains masked text and spans.
| Field | Value |
|---|---|
text | The text to scan |
model | A PII model; omitted, the operator's configured default applies |
entities | Optional; narrows the requested categories |
threshold | Optional; controls detection confidence |
Without entities, all eight categories are requested: person, address, email, phone, iban, identifier, date_of_birth and organization. The native confidence threshold defaults to 0.5 and accepts values from zero to one. Lowering it accepts more reported spans, but cannot recover a span the detector did not report.
The response field masked contains transformed text. entities contains validated category and source-position metadata, not original quotes or a restoration map. With no accepted spans, masked is unchanged. A successful empty detection is not proof that the input contains no personal information.
The direct endpoint does not restore values. In chat, restoration defaults to true; restore: false disables gateway replacement-map restoration in JSON and SSE, not detection or arbitrary PII generation in the answer. See diagnosing the plugin data path before attributing an original value in an answer to restoration.
Choosing a model
PII models have ordinary provider routes and fallback. Provider, model, country, jurisdiction and price policies also apply inside plugins.
Detection always receives the raw personal information; masking does not protect it from the detection provider.
| Specialized PII model | Chat model (LLM) | |
|---|---|---|
| Pricing | character tariff, no extra prompt and completion tokens | actual token prices |
| Coverage | the detector's categories | may cover different languages and detection patterns |
| Calls | one detection | structured instructions, sometimes several sequential calls |
Quality and latency depend on the model and are not guaranteed. Both approaches can miss personal information.
Using a chat model
As a gateway extension, model can select a normal chat model when the operator has explicitly configured a pii.llm task with plaintext approval. A chat route alone is insufficient: JSON Schema output or forced tools, preserved execution controls and bounded context/output tokens are required.
The direct endpoint then accepts:
{
"model": "<approved-chat-model>",
"text": "Contact lena@example.invalid",
"entities": ["email"]
}
The plugin selects it with:
{ "id": "pii-redact", "model": "<approved-chat-model>", "restore": false }
Choose IDs from the actually configured catalog; these placeholders are not available model names.
LLM bounds
The versioned task contains the prompt, schema, window size, overlap, call limit, output tokens and response byte limit. Operators may configure their own versioned task data within the validated bounds. The bundled revision uses:
| Bound | Value |
|---|---|
| Window | 8,192 Unicode scalar values |
| Overlap | 512 |
| Windows | at most 16 |
| Output tokens | 8,192 |
| Span output | 128 KiB |
| Whole input | 65,536 UTF-8 bytes |
How LLM locations are applied
- The LLM returns only
label,start,endandtext. - Only exact source quotes at valid Unicode scalar positions with allowed categories authorize a replacement.
- Invalid entries are discarded and counted; an entirely invalid nonempty result is an error.
- Valid overlapping spans are conservatively united and duplicate detections are masked once.
- At most 256 reported spans are accepted globally, including duplicates from overlapping windows; exceeding the bound is an error, not silent truncation.
- Existing collision-safe placeholders change the text, not the LLM.
- LLM locations do not provide calibrated confidence;
thresholdremains the native detector threshold. - An LLM task cannot enable
fail_open.
OCR
OCR providers receive original document contents, just as PII detectors receive original text; this does not change PII integration or its privacy boundary. Model selection, limits and token billing for document extraction are in PDF parser and OCR.
Costs
The character tariff is independent of token prices. Its unit counts Unicode scalar values in the submitted text, not UTF-8 bytes.
The receipt's preprocessing entries record unit: "character", quantity, model, provider and EUR cost; original provider currency and the exchange-rate snapshot remain available when applicable. Unknown charges are not represented as free work.
LLM billing
For an LLM, unit: "character" remains the scanned workload, including rescanned overlaps. It is not the price basis.
execution: "llm_span_extract",task_version,rejected_countandtoken_usagewith actual input, output and cache tokens identify token billing.token_usage.costsretains original, customer and purchase costs by price component.- The parent request reserves money and tokens before sending and settles together with its plugin.
- Paid invalid responses and fallback attempts remain in receipts; missing confirmed usage is not free detection.
Common errors
Unknown models, no eligible provider, unsupported entity options and unavailable detectors have distinct errors. Check error.metadata.issues and remedy_hint.
Required redaction must fail closed when no permitted detector is available, including inside plugins.
Best practice
- Send only information the application is allowed to process.
- Keep text within the published bounds.
- Do not log input text or detection spans.
- Treat masked output as sensitive unless the application has established a stronger privacy guarantee.
For AI agents
Use the direct PII endpoint reference and current PII catalog. Select explicit model IDs for reproducible routing. Do not call chat completions as a substitute or disable required redaction after a provider failure.