Ollama

Ollama is a tool for running large language models locally on your machine, providing privacy, control, and offline access to AI models like Llama, Mistral, and CodeLlama.

Installation & Setup

Command	Description
curl -fsSL https://ollama.ai/install.sh | sh	Install Ollama on Linux/macOS
brew install ollama	Install via Homebrew (macOS)
ollama --version	Check installed version
ollama serve	Start Ollama server
ollama ps	List running models
ollama list	List installed models

Model Management

Command	Description
ollama pull llama3.1	Download Llama 3.1 model
ollama pull mistral	Download Mistral model
ollama pull codellama	Download CodeLlama model
ollama pull gemma:7b	Download specific model size
ollama show llama3.1	Show model information
ollama rm mistral	Remove model

Popular Models

General Purpose Models

Command	Description
ollama pull llama3.1:8b	Llama 3.1 8B parameters
ollama pull llama3.1:70b	Llama 3.1 70B parameters
ollama pull mistral:7b	Mistral 7B model
ollama pull mixtral:8x7b	Mixtral 8x7B mixture of experts
ollama pull gemma:7b	Google Gemma 7B
ollama pull phi3:mini	Microsoft Phi-3 Mini

Code-Specialized Models

Command	Description
ollama pull codellama:7b	CodeLlama 7B for coding
ollama pull codellama:13b	CodeLlama 13B for coding
ollama pull codegemma:7b	CodeGemma for code generation
ollama pull deepseek-coder:6.7b	DeepSeek Coder model
ollama pull starcoder2:7b	StarCoder2 for code

Specialized Models

Command	Description
ollama pull llava:7b	LLaVA multimodal model
ollama pull nomic-embed-text	Text embedding model
ollama pull all-minilm	Sentence embedding model
ollama pull mxbai-embed-large	Large embedding model

Running Models

Command	Description
ollama run llama3.1	Start interactive chat with Llama 3.1
ollama run mistral "Hello, how are you?"	Single prompt to Mistral
ollama run codellama "Write a Python function"	Code generation with CodeLlama
ollama run llava "Describe this image" --image photo.jpg	Multimodal with image

Chat Interface

Command	Description
ollama run llama3.1	Start interactive chat
/bye	Exit chat session
/clear	Clear chat history
/save chat.txt	Save chat to file
/load chat.txt	Load chat from file
/multiline	Enable multiline input

API Usage

REST API

Command	Description
curl http://localhost:11434/api/generate -d '{"model":"llama3.1","prompt":"Hello"}'	Generate text via API
curl http://localhost:11434/api/chat -d '{"model":"llama3.1","messages":[{"role":"user","content":"Hello"}]}'	Chat via API
curl http://localhost:11434/api/tags	List models via API
curl http://localhost:11434/api/show -d '{"name":"llama3.1"}'	Show model info via API

Streaming Responses

Command	Description
curl http://localhost:11434/api/generate -d '{"model":"llama3.1","prompt":"Hello","stream":true}'	Stream response
curl http://localhost:11434/api/chat -d '{"model":"llama3.1","messages":[{"role":"user","content":"Hello"}],"stream":true}'	Stream chat

Model Configuration

Temperature and Parameters

Command	Description
ollama run llama3.1 --temperature 0.7	Set temperature
ollama run llama3.1 --top-p 0.9	Set top-p sampling
ollama run llama3.1 --top-k 40	Set top-k sampling
ollama run llama3.1 --repeat-penalty 1.1	Set repeat penalty
ollama run llama3.1 --seed 42	Set random seed

Context and Memory

Command	Description
ollama run llama3.1 --ctx-size 4096	Set context window size
ollama run llama3.1 --batch-size 512	Set batch size
ollama run llama3.1 --threads 8	Set number of threads

Custom Models

Creating Modelfiles

Command	Description
ollama create mymodel -f Modelfile	Create custom model
ollama create mymodel -f Modelfile --quantize q4_0	Create with quantization

Modelfile Examples

# Basic Modelfile
FROM llama3.1
PARAMETER temperature 0.8
PARAMETER top_p 0.9
SYSTEM "You are a helpful coding assistant."

# Advanced Modelfile
FROM codellama:7b
PARAMETER temperature 0.2
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1
SYSTEM """You are an expert programmer. Always provide:
1. Clean, well-commented code
2. Explanation of the solution
3. Best practices and optimizations"""

Integration Examples

Python Integration

import requests
import json

def chat_with_ollama(prompt, model="llama3.1"):
    url = "http://localhost:11434/api/generate"
    data = {
        "model": model,
        "prompt": prompt,
        "stream": False
    }
    response = requests.post(url, json=data)
    return response.json()["response"]

# Usage
result = chat_with_ollama("Explain quantum computing")
print(result)

JavaScript Integration

async function chatWithOllama(prompt, model = "llama3.1") {
    const response = await fetch("http://localhost:11434/api/generate", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({
            model: model,
            prompt: prompt,
            stream: false
        })
    });
    const data = await response.json();
    return data.response;
}

// Usage
chatWithOllama("Write a JavaScript function").then(console.log);

Bash Integration

#!/bin/bash
ollama_chat() {
    local prompt="$1"
    local model="${2:-llama3.1}"
    curl -s http://localhost:11434/api/generate \
        -d "{\"model\":\"$model\",\"prompt\":\"$prompt\",\"stream\":false}" \
        | jq -r '.response'
}

# Usage
ollama_chat "Explain Docker containers"

Performance Optimization

Command	Description
ollama run llama3.1 --gpu-layers 32	Use GPU acceleration
ollama run llama3.1 --memory-limit 8GB	Set memory limit
ollama run llama3.1 --cpu-threads 8	Set CPU threads
ollama run llama3.1 --batch-size 1024	Optimize batch size

Environment Variables

Variable	Description
OLLAMA_HOST	Set server host (default: 127.0.0.1:11434)
OLLAMA_MODELS	Set models directory
OLLAMA_NUM_PARALLEL	Number of parallel requests
OLLAMA_MAX_LOADED_MODELS	Max models in memory
OLLAMA_FLASH_ATTENTION	Enable flash attention
OLLAMA_GPU_OVERHEAD	GPU memory overhead

Docker Usage

Command	Description
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama	Run Ollama in Docker
docker exec -it ollama ollama run llama3.1	Run model in container
docker exec -it ollama ollama pull mistral	Pull model in container

Docker Compose

version: '3.8'
services:
  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0:11434
volumes:
  ollama:

Monitoring & Debugging

Command	Description
ollama logs	View Ollama logs
ollama ps	Show running models and memory usage
curl http://localhost:11434/api/version	Check API version
curl http://localhost:11434/api/tags	List available models

Model Quantization

Command	Description
ollama create mymodel -f Modelfile --quantize q4_0	4-bit quantization
ollama create mymodel -f Modelfile --quantize q5_0	5-bit quantization
ollama create mymodel -f Modelfile --quantize q8_0	8-bit quantization
ollama create mymodel -f Modelfile --quantize f16	16-bit float

Embedding Models

Command	Description
ollama pull nomic-embed-text	Pull text embedding model
curl http://localhost:11434/api/embeddings -d '{"model":"nomic-embed-text","prompt":"Hello world"}'	Generate embeddings

Troubleshooting

Command	Description
ollama --help	Show help information
ollama serve --help	Show server options
ps aux | grep ollama	Check if Ollama is running
lsof -i :11434	Check port usage
ollama rm --all	Remove all models

Best Practices

• Choose model size based on available RAM (7B ≈ 4GB, 13B ≈ 8GB, 70B ≈ 40GB)
• Use GPU acceleration when available for better performance
• Implement proper error handling in API integrations
• Monitor memory usage when running multiple models
• Use quantized models for resource-constrained environments
• Cache frequently used models locally
• Set appropriate context sizes for your use case
• Use streaming for long responses to improve user experience
• Implement rate limiting for production API usage
• Regular model updates for improved performance and capabilities

Common Use Cases

Code Generation

ollama run codellama "Create a REST API in Python using FastAPI"

Text Analysis

ollama run llama3.1 "Analyze the sentiment of this text: 'I love this product!'"

Creative Writing

ollama run mistral "Write a short story about time travel"

Data Processing

ollama run llama3.1 "Convert this JSON to CSV format: {...}"