Embeddings

Turn text into vectors for search, retrieval and clustering: request format, limits, costs and errors.

When to use it

Semantic search, retrieval for question answering, near-duplicate detection, clustering, and features for classifiers. An embedding is a vector; texts with similar meaning have vectors that point the same way. Embeddings do not generate text. To order a candidate list by relevance use Rerank. Embeddings are never streamed.

Minimal example

One short text, vectors as numbers. data[0].embedding is the vector.

curl

curl --request POST \
  --url 'https://api.tokenlane.de/v1/embeddings' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "baai/bge-m3",
  "input": "A short sentence to compare.",
  "encoding_format": "float"
}'

node

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: "https://api.tokenlane.de/v1", apiKey: process.env.TOKENLANE_API_KEY });
const response = await client.embeddings.create({
  "model": "baai/bge-m3",
  "input": "A short sentence to compare.",
  "encoding_format": "float"
}, {});
console.log(response);

python

import os
import json
from openai import OpenAI

client = OpenAI(base_url="https://api.tokenlane.de/v1", api_key=os.environ["TOKENLANE_API_KEY"])
body = json.loads(r'''{
  "model": "baai/bge-m3",
  "input": "A short sentence to compare.",
  "encoding_format": "float"
}''')
response = client.embeddings.create(
    model=body["model"],
    input=body["input"],
    encoding_format=body["encoding_format"]
)
print(response)

vercel

// This operation uses HTTP directly.
const response = await fetch("https://api.tokenlane.de/v1/embeddings", {
  method: "POST",
  headers: {
    "Authorization": "Bearer " + process.env.TOKENLANE_API_KEY,
    "Content-Type": "application/json",
  },
  body: `{
  "model": "baai/bge-m3",
  "input": "A short sentence to compare.",
  "encoding_format": "float"
}`,
});
console.log(await response.text());

Request format

FieldValue
modelRequired. An embedding model. A chat model fails with 404 no_endpoint.
inputRequired. One text; an array of texts (one vector each, in order); token IDs of one text or of several; or multimodal inputs. 1 to 2048 inputs. Texts must not be empty. All items of an array must be of the same kind.
encoding_formatfloat (default): arrays of numbers. base64: the vector as little-endian 32-bit floats in base64, often more compact than a JSON number array. Works with every embedding model: if a provider returns numbers, the gateway encodes them.
dimensionsInteger from 1, for models that can shorten their vectors. Others ignore it and name it in x-ignored-params; with provider.require_parameters: true such routes are skipped, and 404 no_endpoint follows if none is left.
input_typeFor models that treat queries and documents differently, for example search_query or search_document. Passed on as sent.
provider, session_id, userRouting, grouping and end-user ID, as for chat.

Batch: "input": ["first text", "second text"] returns two entries in data, each with its index. Token IDs: "input": [101, 2054, 102] is one input, "input": [[101, 2054], [101, 2129]] are two. Multimodal, for models with media input:

"input": [{ "content": [
  { "type": "text", "text": "A cat" },
  { "type": "image_url", "image_url": { "url": "https://example.com/cat.jpg" } }
] }]

Parts are text, image_url (URL or data URL), input_audio, input_video and input_file (each with data as data URL or base64, and format). The model must list the media type in architecture.input_modalities; otherwise 404 no_endpoint.

Decode a base64 vector in Node:

const bytes = Buffer.from(entry.embedding, 'base64');
const vector = new Float32Array(bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength));

In Python: numpy.frombuffer(base64.b64decode(s), dtype='<f4').

Limits:

LimitValueWhen exceeded
Inputs per request2048400 invalid_parameter; split the inputs
Length of one inputthe model's context; the check estimates 4 bytes of text per token400 context_length_exceeded
Request body50 MiB413 payload_too_large
Provider response and merged batch response32 MiB each502 provider_invalid_response; reduce the batch size

A provider can count more tokens than the estimate, for German text or code, and reject an input the check let pass; that is a non-retryable 400. The gateway splits a batch that is larger than a provider accepts per call.

Choosing a model

Filter the catalog for output Embeddings. Compare:

  • context_length: the longest input.
  • pricing.prompt: EUR per input token.
  • supported_parameters: dimensions for shorter vectors, input_type, encoding_format.
  • architecture.input_modalities: more than text means multimodal input.

The vector length depends on the model; read it from the first response (data[0].embedding.length) and store it with the index. A route can differ from the others in context length or price; the model page lists them per provider.

Costs

Only input tokens are billed: usage.prompt_tokens times the prompt price, reported as usage.cost in EUR. There is no output charge. If a provider reports no token count, cost is absent, not zero.

Example: 1,000 chunks of 500 tokens are 500,000 input tokens, billed at the model's prompt price; current prices are on each model page in the catalog. A new model needs a new index, so budget the same token volume again.

The reservation is the estimated token count of the batch, so large batches need credit for all of it.

Common errors

Status, typeCause and fix
400 invalid_parameterMissing or empty input, an empty string, mixed item kinds, a negative token ID, more than 2048 inputs, dimensions below 1, encoding_format other than float or base64. Fix: Fix each path in issues.
400 context_length_exceededAn input is longer than the model's context. Fix: Chunk the text.
404 no_endpointThe model is not an embedding model, media input is not supported, or dimensions with require_parameters has no route. Fix: Choose a model from remedy_hint.
402 insufficient_creditsThe estimated tokens exceed the key limit or the credits. Fix: Send smaller batches or top up credits.
429 rate_limitedA token or request limit of the key or provider. Fix: Wait Retry-After, then send in smaller steps.

Best practice

  • Chunk by structure (paragraph or section) to about 200 to 800 tokens, with a small overlap. One vector per whole document blurs the meaning.
  • Batch up to 2048 inputs per request instead of one call per text.
  • Use the same model and dimensions for the documents and queries of one index. Where the model distinguishes input roles, set input_type to search_document for stored passages and search_query for searches; follow that model's supported values. Store the model ID, dimensions and document input type with the index.
  • Compare vectors with cosine similarity. Normalize them once when you store them.
  • Cache the vector by a hash of the text: the same input and model give the same vector.
  • Search the index first, then reorder the best 50 to 200 hits with Rerank and give only the top few to the chat model.
  • Use base64 to reduce the vector payload where it helps. It does not bypass the provider-response limit: if the provider still returns a large number array, reduce the batch size.
  • Embed no secrets: vectors can leak the text they came from.

For AI agents

  • Endpoint: [POST /embeddings](https://tokenlane.de/docs/gateway/post-L2VtYmVkZGluZ3M.md?lang=en).
  • Required: model (an embedding model from GET /v1/models), input (string, array of strings, token IDs, or multimodal objects with content).
  • Optional: encoding_format (float or base64), dimensions (integer from 1), input_type, provider.
  • Response: data[i].embedding and data[i].index, usage.prompt_tokens, usage.cost (EUR), model.
  • Limits: 2048 inputs; each input within the model's context (4 bytes of text per token estimated); body 50 MiB; no streaming.
  • Errors: 400 invalid_parameter and context_length_exceeded, 404 no_endpoint, 402 insufficient_credits, 429. Read issues before you retry.