LlamaIndex Framework Cheat Sheet

Overview

LlamaIndex is a powerful data framework designed to connect large language models (LLMs) with external data sources, enabling the creation of sophisticated retrieval-augmented generation (RAG) applications. Developed to address the challenge of LLMs' limited knowledge cutoffs, LlamaIndex provides a comprehensive toolkit for ingesting, structuring, and accessing private or domain-specific data that would otherwise be inaccessible to foundation models.

What sets LlamaIndex apart is its focus on data connectivity and knowledge management. The framework excels at transforming raw data from diverse sources into structured, queryable knowledge that LLMs can effectively leverage. With its modular architecture, LlamaIndex offers developers the flexibility to customize each component of the RAG pipeline while providing sensible defaults for quick implementation.

LlamaIndex has emerged as the go-to solution for building knowledge-intensive applications, from question-answering systems and chatbots to document summarization tools and semantic search engines. Its rich ecosystem of integrations with vector databases, embedding models, and LLM providers makes it adaptable to a wide range of use cases and deployment environments.

Installation and Setup

Basic Installation

# Install core LlamaIndex
pip install llama-index

# Install with specific integrations
pip install llama-index-embeddings-openai  # OpenAI embeddings
pip install llama-index-llms-openai        # OpenAI LLMs
pip install llama-index-readers-file       # File readers
pip install llama-index-vector-stores-chroma  # Chroma vector store

# Install all core packages
pip install llama-index-core[all]

# Install development version
pip install git+https://github.com/jerryjliu/llama_index.git

Environment Configuration

import os
import logging
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

# Set up API keys
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

# Global settings configuration
Settings.llm = OpenAI(model="gpt-4", temperature=0.1)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-ada-002")
Settings.chunk_size = 1024
Settings.chunk_overlap = 20

Project Structure

llamaindex_project/
├── data/
│   ├── documents/
│   └── processed/
├── indices/
│   ├── vector_index/
│   └── summary_index/
├── readers/
│   ├── __init__.py
│   └── custom_readers.py
├── retrievers/
│   ├── __init__.py
│   └── custom_retrievers.py
├── query_engines/
│   ├── __init__.py
│   └── custom_engines.py
├── prompts/
│   ├── __init__.py
│   └── custom_prompts.py
├── config/
│   ├── __init__.py
│   └── settings.py
└── main.py

Data Loading and Processing

Document Loading

from llama_index.core import SimpleDirectoryReader
from llama_index.readers.file import PDFReader, DocxReader
from llama_index.readers.web import SimpleWebPageReader

# Load documents from a directory
documents = SimpleDirectoryReader("./data/documents").load_data()

# Load specific file types
pdf_reader = PDFReader()
pdf_documents = pdf_reader.load_data(file_path="./data/documents/report.pdf")

docx_reader = DocxReader()
docx_documents = docx_reader.load_data(file_path="./data/documents/memo.docx")

# Load web pages
web_documents = SimpleWebPageReader().load_data(
    ["https://example.com/page1", "https://example.com/page2"]
)

# Combine documents from multiple sources
all_documents = pdf_documents + docx_documents + web_documents

Custom Document Loading

from llama_index.core import Document
from typing import List
import json

def load_custom_json_data(file_path: str) -> List[Document]:
    """Load custom JSON data into LlamaIndex documents."""
    with open(file_path, "r") as f:
        data = json.load(f)
    
    documents = []
    for item in data:
        # Create document with text content and metadata
        doc = Document(
            text=item["content"],
            metadata={
                "title": item.get("title", ""),
                "author": item.get("author", ""),
                "date": item.get("date", ""),
                "category": item.get("category", ""),
                "source": file_path
            }
        )
        documents.append(doc)
    
    return documents

# Use custom loader
custom_documents = load_custom_json_data("./data/documents/custom_data.json")

Text Splitting

from llama_index.core.node_parser import SentenceSplitter, TokenTextSplitter

# Sentence-based splitter
sentence_splitter = SentenceSplitter(
    chunk_size=1024,
    chunk_overlap=20,
    paragraph_separator="\n\n",
    secondary_chunking_regex="[^,.;。]+[,.;。]?",
)
sentence_nodes = sentence_splitter.get_nodes_from_documents(documents)

# Token-based splitter
token_splitter = TokenTextSplitter(
    chunk_size=512,
    chunk_overlap=50,
    separator=" ",
)
token_nodes = token_splitter.get_nodes_from_documents(documents)

# Custom splitting logic
from llama_index.core.node_parser import SimpleNodeParser
from llama_index.core.schema import TextNode

class CustomNodeParser(SimpleNodeParser):
    def get_nodes_from_documents(self, documents):
        nodes = super().get_nodes_from_documents(documents)
        # Add custom processing for nodes
        for node in nodes:
            # Add custom metadata or transform node text
            node.metadata["processed"] = True
            node.text = node.text.replace("old_term", "new_term")
        return nodes

custom_parser = CustomNodeParser(
    chunk_size=1024,
    chunk_overlap=20,
)
custom_nodes = custom_parser.get_nodes_from_documents(documents)

Text Transformation

from llama_index.core.ingestion import IngestionPipeline
from llama_index.core.node_parser import SentenceSplitter
from llama_index.embeddings.openai import OpenAIEmbedding

# Define custom text transformer
def clean_text_transformer(nodes):
    for node in nodes:
        # Clean text: remove extra whitespace, normalize quotes, etc.
        node.text = " ".join(node.text.split())
        node.text = node.text.replace(""", "\"").replace(""", "\"")
    return nodes

# Create ingestion pipeline
pipeline = IngestionPipeline(
    transformations=[
        SentenceSplitter(chunk_size=1024, chunk_overlap=20),
        clean_text_transformer,
        OpenAIEmbedding(model="text-embedding-ada-002"),
    ]
)

# Process documents through pipeline
nodes = pipeline.run(documents=documents)

Index Creation and Management

Vector Index

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores.chroma import ChromaVectorStore
import chromadb

# Create a basic in-memory vector index
documents = SimpleDirectoryReader("./data/documents").load_data()
vector_index = VectorStoreIndex.from_documents(documents)

# Create persistent vector index with Chroma
chroma_client = chromadb.PersistentClient("./chroma_db")
chroma_collection = chroma_client.create_collection("my_collection")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
persistent_index = VectorStoreIndex.from_documents(
    documents,
    vector_store=vector_store
)

# Save and load index
vector_index.storage_context.persist("./storage")

from llama_index.core import StorageContext, load_index_from_storage
storage_context = StorageContext.from_defaults(persist_dir="./storage")
loaded_index = load_index_from_storage(storage_context)

Summary Index

from llama_index.core import SummaryIndex

# Create summary index
summary_index = SummaryIndex.from_documents(documents)

# Query with summary index
query_engine = summary_index.as_query_engine()
response = query_engine.query("Summarize the key points in these documents.")

Knowledge Graph Index

from llama_index.core import KnowledgeGraphIndex
from llama_index.graph_stores.neo4j import Neo4jGraphStore

# Create knowledge graph index
kg_index = KnowledgeGraphIndex.from_documents(documents)

# Create with Neo4j backend
neo4j_graph_store = Neo4jGraphStore(
    username="neo4j",
    password="password",
    url="bolt://localhost:7687",
    database="neo4j"
)
neo4j_kg_index = KnowledgeGraphIndex.from_documents(
    documents,
    graph_store=neo4j_graph_store
)

# Query knowledge graph
kg_query_engine = kg_index.as_query_engine()
response = kg_query_engine.query(
    "What is the relationship between entity A and entity B?"
)

Hybrid Index

from llama_index.core import VectorStoreIndex, SummaryIndex
from llama_index.core.query_engine import RouterQueryEngine
from llama_index.core.selectors import LLMSingleSelector

# Create multiple indices
vector_index = VectorStoreIndex.from_documents(documents)
summary_index = SummaryIndex.from_documents(documents)

# Create query engines
vector_query_engine = vector_index.as_query_engine()
summary_query_engine = summary_index.as_query_engine()

# Define routing function
def route_query(query_str):
    if "summarize" in query_str.lower() or "overview" in query_str.lower():
        return "summary"
    else:
        return "vector"

# Create router query engine
query_engine_dict = {
    "vector": vector_query_engine,
    "summary": summary_query_engine
}

router_query_engine = RouterQueryEngine(
    selector=LLMSingleSelector.from_defaults(),
    query_engine_dict=query_engine_dict
)

# Query with automatic routing
response = router_query_engine.query("Give me a summary of the documents.")

Querying and Retrieval

Basic Querying

from llama_index.core import VectorStoreIndex

# Create index and query engine
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

# Simple query
response = query_engine.query("What is the main topic discussed in the documents?")
print(response)

# Access source nodes and metadata
for source_node in response.source_nodes:
    print(f"Source text: {source_node.node.text[:100]}...")
    print(f"Metadata: {source_node.node.metadata}")
    print(f"Score: {source_node.score}")
    print("---")

Advanced Query Configuration

from llama_index.core import VectorStoreIndex
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core.retrievers import VectorIndexRetriever

# Create index
index = VectorStoreIndex.from_documents(documents)

# Configure retriever
retriever = VectorIndexRetriever(
    index=index,
    similarity_top_k=5,  # Number of results to retrieve
    vector_store_query_mode="hybrid"  # hybrid, sparse, or dense
)

# Configure query engine
query_engine = RetrieverQueryEngine.from_args(
    retriever=retriever,
    response_mode="compact",  # compact, tree_summarize, refine, or simple
    node_postprocessors=[],  # Optional postprocessors for retrieved nodes
    llm=None  # Optional custom LLM
)

# Execute query
response = query_engine.query("What are the key challenges mentioned in the documents?")

Streaming Responses

from llama_index.core import VectorStoreIndex

# Create index
index = VectorStoreIndex.from_documents(documents)

# Create streaming query engine
query_engine = index.as_query_engine(streaming=True)

# Stream response
streaming_response = query_engine.query("Explain the concept of RAG in detail.")

# Process streaming response
for token in streaming_response.response_gen:
    print(token, end="", flush=True)

Filtering and Metadata Queries

from llama_index.core import VectorStoreIndex, StorageContext, load_index_from_storage
from llama_index.core.vector_stores import MetadataFilters, FilterOperator, MetadataFilter

# Load index
storage_context = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage_context)

# Create metadata filters
filters = MetadataFilters(
    filters=[
        MetadataFilter(key="category", value="technical", operator=FilterOperator.EQ),
        MetadataFilter(key="date", value="2023-01-01", operator=FilterOperator.GTE)
    ]
)

# Create retriever with filters
retriever = index.as_retriever(
    similarity_top_k=5,
    filters=filters
)

# Create query engine with filtered retriever
query_engine = index.as_query_engine(
    retriever=retriever
)

# Execute filtered query
response = query_engine.query("What technical advancements were made after January 2023?")

Multi-Modal Querying

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
from llama_index.multi_modal_llms.openai import OpenAIMultiModal
from llama_index.core.schema import ImageDocument

# Load text and image documents
text_documents = SimpleDirectoryReader("./data/text").load_data()
image_paths = ["./data/images/image1.jpg", "./data/images/image2.png"]
image_documents = [ImageDocument(image_path=path) for path in image_paths]

# Combine documents
all_documents = text_documents + image_documents

# Create multi-modal index
multi_modal_llm = OpenAIMultiModal(model="gpt-4-vision-preview")
index = VectorStoreIndex.from_documents(
    all_documents,
    multi_modal_llm=multi_modal_llm
)

# Query with multi-modal context
query_engine = index.as_query_engine()
response = query_engine.query("Describe what's in the images and how it relates to the text.")

Advanced Features

Custom Retrievers

from llama_index.core.retrievers import BaseRetriever
from llama_index.core.schema import NodeWithScore, QueryBundle
from typing import List

class CustomRetriever(BaseRetriever):
    """Custom retriever implementation."""
    
    def __init__(self, index, top_k=5):
        """Initialize with index."""
        self.index = index
        self.top_k = top_k
    
    def _retrieve(self, query_bundle: QueryBundle) -> List[NodeWithScore]:
        """Custom retrieval logic."""
        # Implement custom retrieval strategy
        vector_retriever = self.index.as_retriever(similarity_top_k=self.top_k)
        vector_nodes = vector_retriever.retrieve(query_bundle)
        
        # Add custom logic, e.g., re-ranking
        reranked_nodes = self._rerank_nodes(vector_nodes, query_bundle)
        
        return reranked_nodes
    
    def _rerank_nodes(self, nodes: List[NodeWithScore], query_bundle: QueryBundle) -> List[NodeWithScore]:
        """Rerank nodes based on custom criteria."""
        # Example: Boost nodes with certain metadata
        for node in nodes:
            if "important" in node.node.metadata.get("tags", []):
                node.score += 0.2
        
        # Sort by updated scores
        return sorted(nodes, key=lambda x: x.score, reverse=True)

# Use custom retriever
custom_retriever = CustomRetriever(index, top_k=10)
query_engine = RetrieverQueryEngine.from_args(retriever=custom_retriever)
response = query_engine.query("What are the most important concepts?")

Custom Response Synthesis

from llama_index.core.response_synthesizers import BaseSynthesizer
from llama_index.core.schema import NodeWithScore, QueryBundle, Response

class CustomResponseSynthesizer(BaseSynthesizer):
    """Custom response synthesizer."""
    
    def __init__(self, llm=None):
        """Initialize with optional LLM."""
        super().__init__(llm=llm)
    
    def synthesize(
        self,
        query_bundle: QueryBundle,
        nodes: List[NodeWithScore],
        **kwargs
    ) -> Response:
        """Synthesize response from nodes."""
        # Extract relevant text from nodes
        context_texts = [node.node.text for node in nodes]
        context_str = "\n\n".join(context_texts)
        
        # Create custom prompt
        prompt_template = (
            "Based on the following context information, answer the query.\n"
            "Context:\n{context_str}\n\n"
            "Query: {query_str}\n\n"
            "Answer: "
        )
        
        prompt = prompt_template.format(
            context_str=context_str,
            query_str=query_bundle.query_str
        )
        
        # Generate response using LLM
        llm_response = self._llm.complete(prompt)
        
        # Create response object
        response = Response(
            response=llm_response.text,
            source_nodes=nodes,
            metadata={
                "prompt": prompt,
                "node_count": len(nodes)
            }
        )
        
        return response

# Use custom response synthesizer
from llama_index.core.query_engine import RetrieverQueryEngine

custom_synthesizer = CustomResponseSynthesizer()
query_engine = RetrieverQueryEngine.from_args(
    retriever=index.as_retriever(),
    response_synthesizer=custom_synthesizer
)

response = query_engine.query("Explain the key concepts in the documents.")

Custom Prompts

from llama_index.core import PromptTemplate
from llama_index.core.prompts import PromptType

# Define custom text QA prompt
text_qa_template = PromptTemplate(
    """You are an expert assistant. Answer the question based on the provided context.
    
    Context:
    {context_str}
    
    Question:
    {query_str}
    
    Answer the question with a detailed explanation. If the answer cannot be found in the context, 
    state "I don't have enough information to answer this question." and suggest what additional 
    information would be needed.
    
    Answer:""",
    prompt_type=PromptType.QUESTION_ANSWER
)

# Define custom refine prompt
refine_template = PromptTemplate(
    """You are an expert assistant. Refine the existing answer based on new context.
    
    Existing Answer:
    {existing_answer}
    
    New Context:
    {context_msg}
    
    Question:
    {query_str}
    
    Refine the existing answer to improve it. If the new context doesn't provide relevant information, 
    keep the existing answer as is.
    
    Refined Answer:""",
    prompt_type=PromptType.REFINE
)

# Use custom prompts with query engine
from llama_index.core import VectorStoreIndex

query_engine = index.as_query_engine(
    text_qa_template=text_qa_template,
    refine_template=refine_template,
    response_mode="refine"
)

response = query_engine.query("What are the main applications of LlamaIndex?")

Evaluation

from llama_index.core.evaluation import (
    FaithfulnessEvaluator,
    RelevancyEvaluator,
    CorrectnessEvaluator,
    BatchEvalRunner
)

# Create evaluators
faithfulness_evaluator = FaithfulnessEvaluator()
relevancy_evaluator = RelevancyEvaluator()
correctness_evaluator = CorrectnessEvaluator()

# Create query engine
query_engine = index.as_query_engine()

# Define evaluation questions and ground truth
eval_questions = [
    "What is retrieval-augmented generation?",
    "How does LlamaIndex handle document ingestion?",
    "What are the main components of a RAG pipeline?"
]

ground_truths = [
    "Retrieval-augmented generation (RAG) is a technique that enhances LLMs by retrieving external knowledge.",
    "LlamaIndex handles document ingestion through document loaders, text splitters, and embedding generation.",
    "The main components of a RAG pipeline include data ingestion, indexing, retrieval, and response generation."
]

# Run batch evaluation
batch_runner = BatchEvalRunner(
    evaluators=[faithfulness_evaluator, relevancy_evaluator, correctness_evaluator],
    workers=2
)

eval_results = batch_runner.evaluate_queries(
    query_engine=query_engine,
    queries=eval_questions,
    ground_truths=ground_truths
)

# Analyze results
for i, result in enumerate(eval_results):
    print(f"Question: {eval_questions[i]}")
    print(f"Faithfulness: {result[0].score}")
    print(f"Relevancy: {result[1].score}")
    print(f"Correctness: {result[2].score}")
    print("---")

Integrations

Vector Store Integrations

# Chroma
from llama_index.vector_stores.chroma import ChromaVectorStore
import chromadb

chroma_client = chromadb.PersistentClient("./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("my_collection")
chroma_vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

# Pinecone
from llama_index.vector_stores.pinecone import PineconeVectorStore
import pinecone

pinecone.init(api_key="your-api-key", environment="your-environment")
pinecone_index = pinecone.Index("your-index-name")
pinecone_vector_store = PineconeVectorStore(pinecone_index=pinecone_index)

# Weaviate
from llama_index.vector_stores.weaviate import WeaviateVectorStore
import weaviate

weaviate_client = weaviate.Client("http://localhost:8080")
weaviate_vector_store = WeaviateVectorStore(
    weaviate_client=weaviate_client,
    index_name="LlamaIndex"
)

# Create index with vector store
from llama_index.core import VectorStoreIndex

index = VectorStoreIndex.from_documents(
    documents,
    vector_store=chroma_vector_store  # Or any other vector store
)

LLM Integrations

# OpenAI
from llama_index.llms.openai import OpenAI

openai_llm = OpenAI(model="gpt-4", temperature=0.1)

# Anthropic
from llama_index.llms.anthropic import Anthropic

anthropic_llm = Anthropic(model="claude-3-sonnet-20240229", temperature=0.2)

# Hugging Face
from llama_index.llms.huggingface import HuggingFaceLLM

huggingface_llm = HuggingFaceLLM(
    model_name="mistralai/Mistral-7B-Instruct-v0.2",
    tokenizer_name="mistralai/Mistral-7B-Instruct-v0.2",
    context_window=4096,
    max_new_tokens=256,
    generate_kwargs={"temperature": 0.7, "do_sample": True}
)

# Set as default LLM
from llama_index.core import Settings

Settings.llm = openai_llm

Embedding Integrations

# OpenAI Embeddings
from llama_index.embeddings.openai import OpenAIEmbedding

openai_embed_model = OpenAIEmbedding(
    model="text-embedding-ada-002",
    embed_batch_size=100
)

# Hugging Face Embeddings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

huggingface_embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-large-en-v1.5"
)

# Set as default embedding model
from llama_index.core import Settings

Settings.embed_model = openai_embed_model

Chat Engine Integrations

from llama_index.core import VectorStoreIndex
from llama_index.core.chat_engine import ContextChatEngine, CondenseQuestionChatEngine

# Create index
index = VectorStoreIndex.from_documents(documents)

# Simple context chat engine
context_chat_engine = index.as_chat_engine(
    chat_mode="context",
    system_prompt="You are a helpful assistant that answers questions based on the provided context."
)

# Condense question chat engine (for better handling of chat history)
condense_chat_engine = index.as_chat_engine(
    chat_mode="condense_question",
    system_prompt="You are a helpful assistant that answers questions based on the provided context."
)

# Chat with history
from llama_index.core.schema import ChatMessage

messages = [
    ChatMessage(role="user", content="What is LlamaIndex?"),
    ChatMessage(role="assistant", content="LlamaIndex is a data framework for LLM applications."),
    ChatMessage(role="user", content="What are its main features?")
]

response = condense_chat_engine.chat(
    message="What are its main features?",
    chat_history=messages[:-1]  # Exclude the last message
)

Production Deployment

Caching and Optimization

from llama_index.core import Settings, VectorStoreIndex
from llama_index.core.callbacks import CallbackManager, LlamaDebugHandler
import diskcache

# Set up caching
cache = diskcache.Cache("./cache")

def memoize_embed(fn):
    """Memoize embedding function."""
    def wrapper(texts):
        results = []
        uncached_texts = []
        uncached_indices = []
        
        # Check cache for each text
        for i, text in enumerate(texts):
            cache_key = f"embed_{hash(text)}"
            if cache_key in cache:
                results.append(cache[cache_key])
            else:
                uncached_texts.append(text)
                uncached_indices.append(i)
        
        # Compute embeddings for uncached texts
        if uncached_texts:
            uncached_embeddings = fn(uncached_texts)
            
            # Store in cache and results
            for i, embedding in zip(uncached_indices, uncached_embeddings):
                cache_key = f"embed_{hash(texts[i])}"
                cache[cache_key] = embedding
                results.append(embedding)
        
        return results
    
    return wrapper

# Apply caching to embedding model
from llama_index.embeddings.openai import OpenAIEmbedding

embed_model = OpenAIEmbedding()
embed_model.get_text_embedding_batch = memoize_embed(embed_model.get_text_embedding_batch)

# Set up debug handler
debug_handler = LlamaDebugHandler()
callback_manager = CallbackManager([debug_handler])

# Configure settings with optimizations
Settings.embed_model = embed_model
Settings.callback_manager = callback_manager
Settings.chunk_size = 512  # Smaller chunks for efficiency
Settings.chunk_overlap = 50

API Deployment with FastAPI

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import List, Optional
from llama_index.core import VectorStoreIndex, StorageContext, load_index_from_storage

app = FastAPI()

# Load index at startup
@app.on_event("startup")
async def startup_event():
    global query_engine
    try:
        storage_context = StorageContext.from_defaults(persist_dir="./storage")
        index = load_index_from_storage(storage_context)
        query_engine = index.as_query_engine()
    except Exception as e:
        print(f"Error loading index: {e}")
        raise

# Define request/response models
class QueryRequest(BaseModel):
    query: str
    filters: Optional[dict] = None
    top_k: Optional[int] = 5

class SourceNode(BaseModel):
    text: str
    metadata: dict
    score: float

class QueryResponse(BaseModel):
    answer: str
    sources: List[SourceNode]

# Query endpoint
@app.post("/query", response_model=QueryResponse)
async def query(request: QueryRequest):
    try:
        # Apply filters if provided
        if request.filters:
            from llama_index.core.vector_stores import MetadataFilters, MetadataFilter
            
            filters = []
            for key, value in request.filters.items():
                filters.append(MetadataFilter(key=key, value=value))
            
            metadata_filters = MetadataFilters(filters=filters)
            retriever = query_engine.index.as_retriever(
                similarity_top_k=request.top_k,
                filters=metadata_filters
            )
            custom_query_engine = query_engine.index.as_query_engine(
                retriever=retriever
            )
            response = custom_query_engine.query(request.query)
        else:
            response = query_engine.query(request.query)
        
        # Format source nodes
        sources = []
        for node in response.source_nodes:
            sources.append(
                SourceNode(
                    text=node.node.text,
                    metadata=node.node.metadata,
                    score=node.score
                )
            )
        
        return QueryResponse(
            answer=str(response),
            sources=sources
        )
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Error processing query: {str(e)}")

# Chat endpoint
class ChatRequest(BaseModel):
    message: str
    chat_history: Optional[List[dict]] = None

@app.post("/chat")
async def chat(request: ChatRequest):
    try:
        from llama_index.core.schema import ChatMessage
        
        # Convert chat history to ChatMessage objects
        chat_history = []
        if request.chat_history:
            for msg in request.chat_history:
                chat_history.append(
                    ChatMessage(role=msg["role"], content=msg["content"])
                )
        
        # Create chat engine if not exists
        if not hasattr(app.state, "chat_engine"):
            app.state.chat_engine = query_engine.index.as_chat_engine(
                chat_mode="condense_question"
            )
        
        # Get response
        response = app.state.chat_engine.chat(
            message=request.message,
            chat_history=chat_history
        )
        
        return {
            "response": response.response,
            "sources": [
                {
                    "text": node.node.text,
                    "metadata": node.node.metadata,
                    "score": node.score
                }
                for node in response.source_nodes
            ] if hasattr(response, "source_nodes") else []
        }
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Error processing chat: {str(e)}")

Docker Deployment

# Dockerfile for LlamaIndex application
FROM python:3.10-slim

WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Create directories for data and storage
RUN mkdir -p ./data/documents ./storage ./cache

# Set environment variables
ENV PYTHONPATH=/app
ENV OPENAI_API_KEY=${OPENAI_API_KEY}

# Expose port
EXPOSE 8000

# Health check
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:8000/health || exit 1

# Start application
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

# docker-compose.yml
version: '3.8'

services:
  llamaindex-app:
    build: .
    ports:
      - "8000:8000"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    volumes:
      - ./data:/app/data
      - ./storage:/app/storage
      - ./cache:/app/cache
    restart: unless-stopped

  chroma-db:
    image: ghcr.io/chroma-core/chroma:latest
    ports:
      - "8001:8000"
    volumes:
      - ./chroma_data:/chroma/chroma
    environment:
      - CHROMA_DB_IMPL=duckdb+parquet
      - CHROMA_PERSIST_DIRECTORY=/chroma/chroma
    restart: unless-stopped

Best Practices and Patterns

Document Processing

• Chunk Size Optimization: Adjust chunk size based on content type and query patterns
• Metadata Enrichment: Add rich metadata to documents for better filtering and retrieval
• Preprocessing: Clean and normalize text before indexing
• Hierarchical Chunking: Use parent-child relationships for better context preservation

Retrieval Strategies

• Hybrid Search: Combine vector and keyword search for better results
• Reranking: Apply post-retrieval reranking to improve relevance
• Metadata Filtering: Use metadata to narrow down search space
• Multi-Index Retrieval: Query multiple indices for comprehensive results

Response Generation

• Response Modes: Choose appropriate response modes (compact, refine, tree_summarize)
• Custom Prompts: Tailor prompts for specific use cases
• Source Attribution: Include source information in responses
• Streaming: Use streaming for better user experience with long responses

Performance Optimization

• Caching: Implement caching for embeddings and LLM responses
• Batch Processing: Process documents in batches
• Async Operations: Use async APIs for non-blocking operations
• Index Pruning: Regularly clean and optimize indices

Monitoring and Evaluation

• Logging: Implement comprehensive logging
• Evaluation Metrics: Track relevance, faithfulness, and correctness
• User Feedback: Collect and incorporate user feedback
• A/B Testing: Compare different configurations

Troubleshooting

Common Issues

Poor Retrieval Quality

• Cause: Inappropriate chunk size, poor embeddings, or insufficient context
• Solution: Adjust chunk size, try different embedding models, or implement reranking

High Latency

• Cause: Large indices, complex queries, or inefficient retrieval
• Solution: Implement caching, optimize chunk size, or use more efficient vector stores

Memory Issues

• Cause: Loading too many documents or embeddings into memory
• Solution: Use disk-based vector stores, process documents in batches, or implement streaming

Hallucinations

• Cause: Insufficient context, poor retrieval, or LLM limitations
• Solution: Improve retrieval quality, adjust prompts, or implement fact-checking

---

This comprehensive LlamaIndex cheat sheet provides everything needed to build sophisticated RAG applications. From basic setup to advanced production deployment patterns, use these examples and best practices to create powerful, knowledge-intensive AI applications with LlamaIndex's flexible framework.