FastEmbed - Lightweight Local Embeddings Cheatsheet
FastEmbed (by Qdrant) is a fast, lightweight library for generating text embeddings locally. It runs models via ONNX Runtime (no heavy PyTorch dependency), which makes it small, quick to install, and efficient on CPU. Beyond dense embeddings it supports sparse (BM25, SPLADE), late-interaction (ColBERT), and reranking models — everything you need to build hybrid dense+sparse search for RAG. It integrates tightly with the Qdrant vector database.
Installation
| Method | Command |
|---|
| pip | pip install fastembed |
| GPU (CUDA) | pip install fastembed-gpu |
| Verify | python -c "from fastembed import TextEmbedding; print('ok')" |
Dense Embeddings
from fastembed import TextEmbedding
model = TextEmbedding("BAAI/bge-small-en-v1.5") # downloads once
docs = ["FastEmbed runs on ONNX.", "It is CPU-friendly."]
vectors = list(model.embed(docs)) # generator of numpy arrays
print(len(vectors), vectors[0].shape)
| Call | Description |
|---|
TextEmbedding(model_name) | Load a dense embedding model |
.embed(list) | Embed documents (batched, lazy) |
.query_embed(str) | Embed a query (some models differ q/d) |
.passage_embed(list) | Embed passages |
TextEmbedding.list_supported_models() | See available models |
Model Types
| Type | Class | Use |
|---|
| Dense | TextEmbedding | Semantic similarity |
| Sparse (BM25/SPLADE) | SparseTextEmbedding | Keyword/exact-term matching |
| Late interaction (ColBERT) | LateInteractionTextEmbedding | High-precision reranking retrieval |
| Reranking | TextCrossEncoder | Reorder candidates |
| Image | ImageEmbedding | Multimodal embeddings |
Sparse Embeddings (for Hybrid Search)
from fastembed import SparseTextEmbedding
sparse = SparseTextEmbedding("Qdrant/bm25")
sv = list(sparse.embed(["exact term ABC-123 matters"]))
print(sv[0].indices, sv[0].values) # sparse: token indices + weights
Sparse vectors capture exact terms (identifiers, numbers) that dense embeddings smooth over — the key to strong hybrid search.
Reranking
from fastembed.rerank.cross_encoder import TextCrossEncoder
reranker = TextCrossEncoder("Xenova/ms-marco-MiniLM-L-6-v2")
scores = reranker.rerank("what is fastembed?",
["FastEmbed is an embedding lib.", "Unrelated text."])
Hybrid Search with Qdrant
from fastembed import TextEmbedding, SparseTextEmbedding
# Compute both dense and sparse vectors for each chunk, upsert to Qdrant,
# then query with both and let Qdrant fuse (RRF) the results.
dense = TextEmbedding("BAAI/bge-small-en-v1.5")
sparse = SparseTextEmbedding("Qdrant/bm25")
| Step | Tool |
|---|
| Dense vectors | TextEmbedding |
| Sparse vectors | SparseTextEmbedding (BM25) |
| Store + fuse | Qdrant native hybrid query |
| Optional rerank | TextCrossEncoder |
Why FastEmbed (Design)
| Property | Benefit |
|---|
| ONNX Runtime | No PyTorch; small install, fast on CPU |
| Quantized models | Lower memory/latency |
| Lazy generators | Stream embeddings, low memory |
| Local | No API calls; data stays private |
Common Workflows
# Batch-embed a corpus for a vector DB (CPU-friendly)
from fastembed import TextEmbedding
model = TextEmbedding("BAAI/bge-base-en-v1.5")
embeddings = list(model.embed(corpus, batch_size=64))
# Build hybrid retrieval: dense + BM25 sparse, fused in Qdrant
FastEmbed vs Alternatives
| Aspect | FastEmbed | sentence-transformers | OpenAI embeddings API |
|---|
| Runtime | ONNX (no torch) | PyTorch | Hosted API |
| Sparse/ColBERT | Built-in | Extra libs | No |
| Local/private | Yes | Yes | No |
| Best for | Lightweight local + hybrid | Full torch ecosystem | Managed simplicity |
Resources
