# Rerank
Order candidate documents by relevance to a query: request format, limits, costs and errors.

## When to use it
Reranking is the second step of retrieval. A vector or keyword search returns, say, 100 candidates quickly; a rerank model reads the query together with each candidate and orders them much more precisely. Then you pass the best few to the chat model. Rerank does not search your database and does not generate an answer. For the first step see [Embeddings](https://tokenlane.de/docs/guides/embeddings.md?lang=en).

## Minimal example
A query and two documents; `top_n` limits the result to the best hit.

### curl

```curl
curl --request POST \
  --url 'https://api.tokenlane.de/v1/rerank' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "baai/bge-reranker-v2-m3",
  "query": "European capital",
  "documents": [
    "Paris is in France.",
    "A recipe for soup."
  ],
  "top_n": 1
}'
```

### node

```node
const response = await fetch("https://api.tokenlane.de/v1/rerank", {
  method: "POST",
  headers: {
    "Authorization": "Bearer " + process.env.TOKENLANE_API_KEY,
    "Content-Type": "application/json",
  },
  body: `{
  "model": "baai/bge-reranker-v2-m3",
  "query": "European capital",
  "documents": [
    "Paris is in France.",
    "A recipe for soup."
  ],
  "top_n": 1
}`,
});
console.log(await response.text());
```

### python

```python
import os
import urllib.request

body = (r'''{
  "model": "baai/bge-reranker-v2-m3",
  "query": "European capital",
  "documents": [
    "Paris is in France.",
    "A recipe for soup."
  ],
  "top_n": 1
}''').encode("utf-8")

request = urllib.request.Request(
    "https://api.tokenlane.de/v1/rerank",
    method="POST",
    headers={
        "Authorization": "Bearer " + os.environ["TOKENLANE_API_KEY"],
        "Content-Type": "application/json",
    },
    data=body,
)

with urllib.request.urlopen(request) as response:
    print(response.read().decode("utf-8"))
```

### vercel

```vercel
// This operation uses HTTP directly.
const response = await fetch("https://api.tokenlane.de/v1/rerank", {
  method: "POST",
  headers: {
    "Authorization": "Bearer " + process.env.TOKENLANE_API_KEY,
    "Content-Type": "application/json",
  },
  body: `{
  "model": "baai/bge-reranker-v2-m3",
  "query": "European capital",
  "documents": [
    "Paris is in France.",
    "A recipe for soup."
  ],
  "top_n": 1
}`,
});
console.log(await response.text());
```

Each result has the `index` of the document in your list, a `relevance_score` and the `document` you sent. The list is sorted best first.

## Request format
| Field | Value |
|---|---|
| `model` | Required. A reranking model. A chat model fails with 404 `no_endpoint`. |
| `query` | Required. Text, at most 65,536 UTF-8 bytes, not blank. |
| `documents` | Required. 1 to 1,000 items. Each is a string or an object with a non-empty `text` string, at most 65,536 UTF-8 bytes. Query and documents together at most 4 MiB of text. |
| `top_n` | Integer from 1 up to the number of documents. Default: all documents. |
| `return_documents` | Boolean. Accepted for compatibility and has no effect: every result carries `document`, whether the field is omitted, `true` or `false`. `null` or another type fails with 400. |
| `provider` | Routing, as for chat. |

### What the provider receives and what comes back
The response always contains your documents, as the OpenRouter rerank format requires a `document` in every result. There is no option for index and score only.

| Part of your request | Sent to the provider | Returned to you |
|---|---|---|
| `query` | Yes, unchanged. | No. |
| Text of each document | Yes, all documents in your order, also with `top_n`. | Yes, in `results[].document.text`, for the returned results only. |
| Other fields of a document object (your ID, title, nested metadata) | No. | Yes, unchanged in `results[].document`. |
| `top_n` | No. The gateway ranks all documents and then keeps the best `top_n`. | Only that many results. |
| `return_documents` | No. | No effect. |
| `model` | The provider's model ID for the route. | The public model ID in `model`. |

A string document comes back as an object with `text`. An object comes back with all its fields. Put your own ID into a document object, and you do not have to map `index` back:

```json
"documents": [{ "text": "Madrid is in Spain.", "id": "doc-9", "source": "wiki" }]
```

The matching result:

```json
{ "document": { "id": "doc-9", "source": "wiki", "text": "Madrid is in Spain." }, "index": 0, "relevance_score": 0.91 }
```

Everything in `document` is your own input; nothing else is added. If you pass the results on to a browser, a log or another service, remove `document` or the fields that must not travel with it.

Response fields:

| Field | Meaning |
|---|---|
| `results[].index` | Zero-based position in your `documents`. |
| `results[].relevance_score` | A number from 0 to 1. Sorted descending; equal scores keep your order. |
| `results[].document` | Always. The document you sent: a string as `{ "text": … }`, an object with all its fields. |
| `usage` | `prompt_tokens`, `total_tokens`, `estimated`, `cost` (EUR). |
| `id`, `model`, `provider` | Generation ID (also in `x-generation-id`), the model and the provider that answered. |

The gateway checks the provider's ranking: scores outside 0 to 1, a missing or repeated index or a malformed answer become 502 `provider_invalid_response`, and the next provider is tried.

Each pair of query and document must fit the model's context (4 bytes of text per token are estimated). With a context of 8,192 tokens that is about 32 KB per pair; a longer pair fails with 400 `context_length_exceeded`.

## Choosing a model
Choose a model that serves the rerank endpoint. In the [catalog](https://tokenlane.de/models) it takes text as input, shows a price per request (`pricing.request`) and lists no sampling parameters. Compare the context length (the longest query-plus-document pair) and the price. A chat model is not a reranker, and a reranker cannot chat.

## Costs
Reranking has no output. The price depends on the route: a fixed price per request (`pricing.request`), a price per input token, or both. The charge is in `usage.cost` (EUR), split in `cost_details`. If a provider reports no token usage, the token count is an estimate (`usage.estimated: true`), allowed only for routes with a fixed request price; the cost is exact.

With a fixed price per request, one search over 100 candidates costs one request price, and 1,000 searches cost 1,000 request prices, however many documents each one has. Current prices are on each model page in the [catalog](https://tokenlane.de/models).

On a fixed-price route, send all candidates in one request instead of several small ones. The reservation before the call is the request price plus the cost of the estimated input tokens at the route's prompt price.

## Common errors
| Status, type | Cause and fix |
|---|---|
| 400 `invalid_parameter` | Blank `query`, no or more than 1,000 `documents`, an object without `text`, text over 65,536 bytes, `top_n` below 1 or above the document count, a non-boolean `return_documents`. **Fix:** Fix each path in `issues`. |
| 400 `context_length_exceeded` | A query-document pair is longer than the model's context. **Fix:** Shorten the documents. |
| 404 `no_endpoint` | The model is not a reranker. **Fix:** Choose a model from `remedy_hint`. |
| 502 `provider_invalid_response` | The provider returned an invalid ranking. **Fix:** Retry; another provider is tried automatically. |
| 402 `insufficient_credits`, 429 | Credits or limits. **Fix:** As for chat. |

## Best practice
- Retrieve first, rerank second: 50 to 200 candidates from the vector search, then `top_n` of 3 to 10 for the prompt.
- Keep documents to one idea each, a few hundred tokens. Long pages dilute the score and use context.
- A `relevance_score` ranks documents for one query. It is not a probability and differs between models; compare scores only within one response, and pick any cut-off by testing on your own data.
- Every result carries its document text, so the response grows with the documents. With long documents, keep `top_n` small; `return_documents: false` does not shorten the response.
- Remove duplicates before you rerank; they take the places of better documents.
- Pass the documents to the chat model in the new order, with their IDs, so that answers can cite sources.

## For AI agents
- Endpoint: [`POST /rerank`](https://tokenlane.de/docs/gateway/post-L3JlcmFuaw.md?lang=en).
- Required: `model` (a reranking model from `GET /v1/models`), `query` (string), `documents` (1 to 1000 strings or objects with `text`).
- Optional: `top_n` (integer from 1 to the document count, applied by the gateway), `return_documents` (boolean, no effect), `provider`.
- Response: `results[]` sorted by `relevance_score` (0 to 1, descending) with `index` (position in `documents`) and `document` (always: your object unchanged, or `{ "text": … }` for a string); `usage.cost` (EUR).
- Data flow: only `query` and the document texts reach the provider; object fields, `top_n` and `return_documents` stay in the gateway.
- Limits: 1000 documents; 65,536 UTF-8 bytes per text; 4 MiB in total; each query-document pair within the model's context; no streaming.
- Errors: 400 `invalid_parameter` and `context_length_exceeded`, 404 `no_endpoint`, 502 `provider_invalid_response`, 402, 429.
