Embeddings
Turn text into vectors for search, retrieval and clustering: request format, limits, costs and errors.
When to use it
Semantic search, retrieval for question answering, near-duplicate detection, clustering, and features for classifiers. An embedding is a vector; texts with similar meaning have vectors that point the same way. Embeddings do not generate text. To order a candidate list by relevance use Rerank. Embeddings are never streamed.
Minimal example
One short text, vectors as numbers. data[0].embedding is the vector.
curl
curl --request POST \
--url 'https://api.tokenlane.de/v1/embeddings' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
"model": "baai/bge-m3",
"input": "A short sentence to compare.",
"encoding_format": "float"
}'
node
import OpenAI from 'openai';
const client = new OpenAI({ baseURL: "https://api.tokenlane.de/v1", apiKey: process.env.TOKENLANE_API_KEY });
const response = await client.embeddings.create({
"model": "baai/bge-m3",
"input": "A short sentence to compare.",
"encoding_format": "float"
}, {});
console.log(response);
python
import os
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.tokenlane.de/v1", api_key=os.environ["TOKENLANE_API_KEY"])
body = json.loads(r'''{
"model": "baai/bge-m3",
"input": "A short sentence to compare.",
"encoding_format": "float"
}''')
response = client.embeddings.create(
model=body["model"],
input=body["input"],
encoding_format=body["encoding_format"]
)
print(response)
vercel
// This operation uses HTTP directly.
const response = await fetch("https://api.tokenlane.de/v1/embeddings", {
method: "POST",
headers: {
"Authorization": "Bearer " + process.env.TOKENLANE_API_KEY,
"Content-Type": "application/json",
},
body: `{
"model": "baai/bge-m3",
"input": "A short sentence to compare.",
"encoding_format": "float"
}`,
});
console.log(await response.text());
Request format
| Field | Value |
|---|---|
model | Required. An embedding model. A chat model fails with 404 no_endpoint. |
input | Required. One text; an array of texts (one vector each, in order); token IDs of one text or of several; or multimodal inputs. 1 to 2048 inputs. Texts must not be empty. All items of an array must be of the same kind. |
encoding_format | float (default): arrays of numbers. base64: the vector as little-endian 32-bit floats in base64, often more compact than a JSON number array. Works with every embedding model: if a provider returns numbers, the gateway encodes them. |
dimensions | Integer from 1, for models that can shorten their vectors. Others ignore it and name it in x-ignored-params; with provider.require_parameters: true such routes are skipped, and 404 no_endpoint follows if none is left. |
input_type | For models that treat queries and documents differently, for example search_query or search_document. Passed on as sent. |
provider, session_id, user | Routing, grouping and end-user ID, as for chat. |
Batch: "input": ["first text", "second text"] returns two entries in data, each with its index. Token IDs: "input": [101, 2054, 102] is one input, "input": [[101, 2054], [101, 2129]] are two. Multimodal, for models with media input:
"input": [{ "content": [
{ "type": "text", "text": "A cat" },
{ "type": "image_url", "image_url": { "url": "https://example.com/cat.jpg" } }
] }]
Parts are text, image_url (URL or data URL), input_audio, input_video and input_file (each with data as data URL or base64, and format). The model must list the media type in architecture.input_modalities; otherwise 404 no_endpoint.
Decode a base64 vector in Node:
const bytes = Buffer.from(entry.embedding, 'base64');
const vector = new Float32Array(bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength));
In Python: numpy.frombuffer(base64.b64decode(s), dtype='<f4').
Limits:
| Limit | Value | When exceeded |
|---|---|---|
| Inputs per request | 2048 | 400 invalid_parameter; split the inputs |
| Length of one input | the model's context; the check estimates 4 bytes of text per token | 400 context_length_exceeded |
| Request body | 50 MiB | 413 payload_too_large |
| Provider response and merged batch response | 32 MiB each | 502 provider_invalid_response; reduce the batch size |
A provider can count more tokens than the estimate, for German text or code, and reject an input the check let pass; that is a non-retryable 400. The gateway splits a batch that is larger than a provider accepts per call.
Choosing a model
Filter the catalog for output Embeddings. Compare:
context_length: the longest input.pricing.prompt: EUR per input token.supported_parameters:dimensionsfor shorter vectors,input_type,encoding_format.architecture.input_modalities: more thantextmeans multimodal input.
The vector length depends on the model; read it from the first response (data[0].embedding.length) and store it with the index. A route can differ from the others in context length or price; the model page lists them per provider.
Costs
Only input tokens are billed: usage.prompt_tokens times the prompt price, reported as usage.cost in EUR. There is no output charge. If a provider reports no token count, cost is absent, not zero.
Example: 1,000 chunks of 500 tokens are 500,000 input tokens, billed at the model's prompt price; current prices are on each model page in the catalog. A new model needs a new index, so budget the same token volume again.
The reservation is the estimated token count of the batch, so large batches need credit for all of it.
Common errors
| Status, type | Cause and fix |
|---|---|
400 invalid_parameter | Missing or empty input, an empty string, mixed item kinds, a negative token ID, more than 2048 inputs, dimensions below 1, encoding_format other than float or base64. Fix: Fix each path in issues. |
400 context_length_exceeded | An input is longer than the model's context. Fix: Chunk the text. |
404 no_endpoint | The model is not an embedding model, media input is not supported, or dimensions with require_parameters has no route. Fix: Choose a model from remedy_hint. |
402 insufficient_credits | The estimated tokens exceed the key limit or the credits. Fix: Send smaller batches or top up credits. |
429 rate_limited | A token or request limit of the key or provider. Fix: Wait Retry-After, then send in smaller steps. |
Best practice
- Chunk by structure (paragraph or section) to about 200 to 800 tokens, with a small overlap. One vector per whole document blurs the meaning.
- Batch up to 2048 inputs per request instead of one call per text.
- Use the same model and
dimensionsfor the documents and queries of one index. Where the model distinguishes input roles, setinput_typetosearch_documentfor stored passages andsearch_queryfor searches; follow that model's supported values. Store the model ID, dimensions and document input type with the index. - Compare vectors with cosine similarity. Normalize them once when you store them.
- Cache the vector by a hash of the text: the same input and model give the same vector.
- Search the index first, then reorder the best 50 to 200 hits with Rerank and give only the top few to the chat model.
- Use
base64to reduce the vector payload where it helps. It does not bypass the provider-response limit: if the provider still returns a large number array, reduce the batch size. - Embed no secrets: vectors can leak the text they came from.
For AI agents
- Endpoint: [
POST /embeddings](https://tokenlane.de/docs/gateway/post-L2VtYmVkZGluZ3M.md?lang=en). - Required:
model(an embedding model fromGET /v1/models),input(string, array of strings, token IDs, or multimodal objects withcontent). - Optional:
encoding_format(floatorbase64),dimensions(integer from 1),input_type,provider. - Response:
data[i].embeddinganddata[i].index,usage.prompt_tokens,usage.cost(EUR),model. - Limits: 2048 inputs; each input within the model's context (4 bytes of text per token estimated); body 50 MiB; no streaming.
- Errors: 400
invalid_parameterandcontext_length_exceeded, 404no_endpoint, 402insufficient_credits, 429. Readissuesbefore you retry.