Sentence Transformers v6.0 introduces a fourth model type, MultiVectorEncoder, to support ColBERT-style late interaction retrieval. The new model class loads PyLate checkpoints, Stanford-NLP ColBERT checkpoints and ColPali-family visual retrieval models through the same API used for dense, sparse and reranker models.
How multi-vector models work
Instead of compressing an entire text into a single vector, a multi-vector model emits one vector per token. The scoring operator, MaxSim, compares each query token to every document token and sums the per-token maxima: MaxSim(Q, D) = sum_{Qi in Q} max_{Dj in D} (Qi · Dj). Token embeddings are L2-normalized, so each dot product is a cosine similarity in [-1, 1], and the total score lies in [-num_query_tokens, num_query_tokens].
The release notes highlight two retrieval behaviors: token-level matches are preserved (helpful for exact identifiers or multi-requirement queries) and contextualized tokens can align across synonyms or paraphrases. The example given is that a query token “live” can match document token “inhabit” with high similarity in practice.
Costs, formats and examples
The documented trade-off is index size. Encoding 4,874 Natural Questions passages with lightonai/LateOn produced 608,414 token vectors (average 124.8 per passage). Reported storage figures for that sample were: MiniLM dense (384-d) 7.5 MB, gte-modernbert-base (768-d) 15.0 MB, and LateOn multi-vector (128-d) 311.5 MB. The authors note compression options: the same 608,414 vectors take about 92 MB as a fast-plaid index, and a 4096-d dense model like Qwen3-Embedding-8B would occupy roughly 80 MB for the same data.
Multi-vector checkpoints carry configuration knobs (query/document prefixes, length caps, skiplists). Calls are asymmetric: encode_query() and encode_document() return per-input 2D tensors (num_tokens × embedding_dim). An example shows query_embeddings[0].shape == (10, 128) and document_embeddings shapes like (10, 128) or (19, 128).
The release also shows inference examples: model.similarity produced tensor([[10.7942, 11.1104, 10.9743, 11.0811]]) when comparing a query about the “Red Planet” to several candidate passages, with the expected ranking. On MLDR, the authors report mLateOn scoring 77.92 versus mDenseOn’s 51.59.
Install with: pip install -U sentence-transformers. For visual retrieval, include image extras: pip install -U "sentence-transformers[image]". Sentence Transformers v6.0 requires transformers v5.x, torch 2.2+ and huggingface-hub v1.x. Example load: from sentence_transformers import MultiVectorEncoder.
model = MultiVectorEncoder("lightonai/LateOn")
Original source: Hugging Face Blog