Danswer (Onyx) Cheat Sheet

Overview

Danswer (rebranded as Onyx in 2024) is an open-source enterprise AI search and assistant platform. Unlike general-purpose RAG frameworks, Danswer focuses on connecting to and indexing existing company knowledge sources — Slack, Confluence, Notion, Google Drive, GitHub, Jira, Zendesk, and 30+ more — without requiring manual document uploads. Connectors run on configurable schedules to keep the knowledge base current.

The platform layers enterprise features on top of RAG: personas (specialized AI assistants with custom prompts and document access), role-based access control (RBAC) with user/group permissions, credential management for OAuth and API key connectors, a Slack bot that answers queries in-channel, and detailed usage analytics. It is designed to be the single AI interface that queries across all company knowledge simultaneously.

Danswer is deployed via Docker Compose with Postgres, Vespa (vector + keyword search engine), Redis, and optional Celery workers. It supports OpenAI, Anthropic, Azure OpenAI, Bedrock, and local models via vLLM or Ollama as LLM backends. The web interface provides full administration including connector management, user onboarding, and search analytics.

Installation

Docker Compose (Standard Deployment)

# Clone repository
git clone https://github.com/danswer-ai/danswer.git
cd danswer/deployment/docker_compose

# Copy and configure environment
cp .env.bak .env

# Edit .env with required settings
nano .env

# Start all services
docker compose -f docker-compose.dev.yml up -d

# Or production (with nginx + SSL)
docker compose -f docker-compose.prod.yml up -d

# Verify all services started
docker compose ps

# View logs
docker compose logs -f api_server
docker compose logs -f background

Key Environment Variables

# .env configuration

# LLM Provider (choose one)
GEN_AI_MODEL_PROVIDER=openai         # openai | anthropic | azure | bedrock | ollama
GEN_AI_MODEL_VERSION=gpt-4o-mini
GEN_AI_API_KEY=sk-...

# Anthropic
GEN_AI_MODEL_PROVIDER=anthropic
GEN_AI_MODEL_VERSION=claude-3-5-haiku-20241022
GEN_AI_API_KEY=sk-ant-...

# Azure OpenAI
GEN_AI_MODEL_PROVIDER=azure
AZURE_DEPLOYMENT_ID=gpt-4o
AZURE_API_BASE=https://your-instance.openai.azure.com/
AZURE_API_VERSION=2024-02-01
GEN_AI_API_KEY=your-azure-key

# Embedding model
DOCUMENT_ENCODER_MODEL=intfloat/e5-base-v2   # Local HuggingFace model

# Authentication
AUTH_TYPE=disabled              # disabled | basic | google_oauth | saml | oidc
SECRET_KEY=your-random-secret-here

# Google OAuth
AUTH_TYPE=google_oauth
GOOGLE_OAUTH_CLIENT_ID=...
GOOGLE_OAUTH_CLIENT_SECRET=...

# Email (for invites)
SMTP_SERVER=smtp.gmail.com
SMTP_PORT=587
EMAIL_FROM=noreply@yourcompany.com
SMTP_USER=...
SMTP_PASS=...

# Slack bot
DANSWER_BOT_SLACK_APP_TOKEN=xapp-...
DANSWER_BOT_SLACK_BOT_TOKEN=xoxb-...

Resource Requirements

Minimum:    4 CPU cores, 16 GB RAM, 50 GB disk
Production: 8+ CPU cores, 32+ GB RAM, 200+ GB disk
Services:   api_server, background, web_server, vespa, postgres, redis, nginx
GPU:        Optional — improves local embedding speed

Configuration

Initial Setup

# After starting services, access web interface
open http://localhost:3000

# First user to register becomes admin
# Navigate to Admin → LLM → Configure your LLM provider
# Navigate to Admin → Embedding → Select embedding model
# Navigate to Admin → Connectors → Add data sources

# API is available at
open http://localhost:8080/docs   # FastAPI interactive docs

# Verify API
curl http://localhost:8080/health

API Authentication

import requests

BASE = "http://localhost:8080"

# Basic auth (if AUTH_TYPE=basic)
session = requests.Session()
resp = session.post(f"{BASE}/auth/login", json={
    "username": "admin@company.com",
    "password": "your-password"
})
# Session cookie maintained automatically

# API token (create in web UI under Profile → API Tokens)
API_TOKEN = "your-api-token"
headers = {"Authorization": f"Bearer {API_TOKEN}"}

Core Commands/API

Endpoint	Method	Description
/health	GET	Health check
/auth/login	POST	Authenticate user
/auth/logout	POST	Logout
/connector	GET	List all connectors
/connector	POST	Create a new connector
/connector/{id}	DELETE	Delete connector
/connector/{id}/indexing-status	GET	Get connector sync status
/connector/run-once/{id}	POST	Trigger immediate sync
/credential	GET	List credentials
/credential	POST	Create a credential
/connector/{id}/credential/{cred_id}	PUT	Link credential to connector
/document-set	GET	List document sets
/document-set	POST	Create document set
/persona	GET	List all personas
/persona	POST	Create a persona
/chat/create-chat-session	POST	Start a new chat session
/chat/send-message	POST	Send a chat message
/search	POST	Direct document search
/user	GET	List all users
/user/me	GET	Get current user
/manage/admin/user	GET	Admin: list users
/manage/admin/user	PATCH	Admin: update user role
/query/answer-with-quote	POST	Get answer with source quotes

Advanced Usage

Connector Management

import requests

BASE    = "http://localhost:8080"
HEADERS = {"Authorization": "Bearer your-api-token"}

# List all connectors
connectors = requests.get(f"{BASE}/connector", headers=HEADERS).json()
for c in connectors:
    print(f"{c['id']}: {c['name']} ({c['source']}) — last sync: {c.get('last_successful_index_time', 'never')}")

# Create a Web connector (crawl a website)
def create_web_connector(name: str, base_url: str) -> dict:
    resp = requests.post(f"{BASE}/connector", headers=HEADERS, json={
        "name":           name,
        "source":         "web",
        "input_type":     "pull",
        "connector_specific_config": {
            "base_url":   base_url,
            "web_connector_type": "recursive"
        },
        "refresh_freq":   86400,   # Sync every 24 hours
        "disabled":       False
    })
    return resp.json()

# Create a GitHub connector
def create_github_connector(name: str, repo_owner: str, repo_name: str,
                             include_prs: bool = True,
                             include_issues: bool = True) -> dict:
    resp = requests.post(f"{BASE}/connector", headers=HEADERS, json={
        "name":   name,
        "source": "github",
        "input_type": "poll",
        "connector_specific_config": {
            "repo_owner":      repo_owner,
            "repo_name":       repo_name,
            "include_prs":     include_prs,
            "include_issues":  include_issues
        },
        "refresh_freq": 3600   # Hourly
    })
    return resp.json()

# Create credential for GitHub
def create_github_credential(github_token: str) -> dict:
    resp = requests.post(f"{BASE}/credential", headers=HEADERS, json={
        "credential_json":     {"github_access_token": github_token},
        "admin_public":        True,
        "source":              "github",
        "name":                "GitHub Token"
    })
    return resp.json()

# Link credential to connector
def link_credential(connector_id: int, credential_id: int):
    requests.put(
        f"{BASE}/connector/{connector_id}/credential/{credential_id}",
        headers=HEADERS,
        json={"is_admin_connector": True}
    )

# Trigger immediate sync
def sync_now(connector_id: int):
    requests.post(f"{BASE}/connector/run-once/{connector_id}", headers=HEADERS)

# Check sync status
def get_sync_status(connector_id: int) -> dict:
    resp = requests.get(
        f"{BASE}/connector/{connector_id}/indexing-status",
        headers=HEADERS
    )
    return resp.json()

Connector Examples by Source

CONNECTOR_CONFIGS = {
    "confluence": {
        "source": "confluence",
        "connector_specific_config": {
            "wiki_base":  "https://yourcompany.atlassian.net/wiki",
            "space":      "ENG",
            "is_cloud":   True
        },
        "credential": {"confluence_access_token": "...", "confluence_username": "..."}
    },
    "google_drive": {
        "source": "google_drive",
        "connector_specific_config": {
            "folder_paths":  ["Engineering Docs", "Product Specs"],
            "include_shared": True
        },
        "credential": {"google_drive_service_account_key": {...}}  # Service account JSON
    },
    "slack": {
        "source": "slack",
        "connector_specific_config": {
            "workspace":    "your-workspace",
            "channels":     ["#engineering", "#product", "#support"],
            "channel_regex_enabled": False
        },
        "credential": {"slack_bot_token": "xoxb-..."}
    },
    "jira": {
        "source": "jira",
        "connector_specific_config": {
            "jira_base":  "https://yourcompany.atlassian.net",
            "projects":   ["ENG", "INFRA"]
        },
        "credential": {"jira_user_email": "...", "jira_api_token": "..."}
    },
    "notion": {
        "source": "notion",
        "connector_specific_config": {"root_page_id": None},
        "credential": {"notion_integration_token": "secret_..."}
    },
    "zendesk": {
        "source": "zendesk",
        "connector_specific_config": {"subdomain": "yourcompany"},
        "credential": {"zendesk_token": "...", "zendesk_email": "..."}
    }
}

Personas and Document Sets

# Create a Document Set (curated collection for a persona)
def create_document_set(name: str, connector_ids: list[int]) -> dict:
    resp = requests.post(f"{BASE}/document-set", headers=HEADERS, json={
        "name":             name,
        "description":      f"Documents for {name}",
        "cc_pair_ids":      connector_ids,  # connector-credential pair IDs
        "is_public":        True
    })
    return resp.json()

# Create a Persona (specialized AI assistant)
def create_persona(name: str, system_prompt: str,
                   document_set_ids: list[int],
                   llm_model: str = None) -> dict:
    resp = requests.post(f"{BASE}/persona", headers=HEADERS, json={
        "name":             name,
        "description":      f"AI assistant for {name}",
        "system_prompt":    system_prompt,
        "task_prompt":      "Answer based only on the provided context.",
        "document_sets":    document_set_ids,
        "is_public":        True,
        "llm_model_version_override": llm_model   # Override default LLM
    })
    return resp.json()

# Example: Engineering assistant
eng_persona = create_persona(
    name="Engineering Assistant",
    system_prompt=(
        "You are an expert software engineering assistant with deep knowledge "
        "of our internal systems, APIs, and architecture. Answer questions "
        "based on our engineering documentation, GitHub repos, and Jira tickets."
    ),
    document_set_ids=[github_set_id, confluence_set_id],
    llm_model="gpt-4o"   # Use smarter model for technical questions
)

Chat and Search API

# Create chat session
def start_session(persona_id: int = None) -> str:
    payload = {}
    if persona_id:
        payload["persona_id"] = persona_id
    resp = requests.post(
        f"{BASE}/chat/create-chat-session",
        headers=HEADERS,
        json=payload
    )
    return resp.json()["chat_session_id"]

# Send message (streaming)
import json

def chat_stream(session_id: str, message: str):
    """Generator that yields answer chunks."""
    resp = requests.post(
        f"{BASE}/chat/send-message",
        headers=HEADERS,
        json={
            "chat_session_id":     session_id,
            "message":             message,
            "search_doc_ids":      None,
            "retrieval_options":   {"run_search": "auto"},
            "prompt_id":           None,
            "query_override":      None
        },
        stream=True
    )
    for line in resp.iter_lines():
        if line:
            data = json.loads(line)
            if data.get("answer_piece"):
                yield data["answer_piece"]

# Direct document search (no LLM)
def search_documents(query: str, top_k: int = 10, persona_id: int = None) -> list:
    payload = {
        "query":       query,
        "filters":     {},
        "enable_auto_detect_filters": True,
        "search_type": "hybrid",       # semantic | keyword | hybrid
        "offset":      0,
        "limit":       top_k
    }
    if persona_id:
        payload["persona_id"] = persona_id

    resp = requests.post(f"{BASE}/search", headers=HEADERS, json=payload)
    return resp.json()["top_sections"]

# Usage
session = start_session(persona_id=eng_persona["id"])
answer = "".join(chat_stream(session, "How do we handle database migrations?"))
print(answer)

docs = search_documents("CI/CD pipeline configuration", top_k=5)
for doc in docs:
    print(f"[{doc['score']:.3f}] {doc['document_id']}: {doc['content'][:100]}")

User and RBAC Management

# List all users
users = requests.get(f"{BASE}/manage/admin/user", headers=HEADERS).json()

# Update user role
def set_role(user_email: str, role: str):
    """role: 'basic' | 'admin' | 'global_curator' | 'curator'"""
    requests.patch(f"{BASE}/manage/admin/user", headers=HEADERS, json={
        "user_email": user_email,
        "role":       role
    })

# Deactivate user
def deactivate_user(user_email: str):
    requests.patch(f"{BASE}/manage/admin/user", headers=HEADERS, json={
        "user_email": user_email,
        "is_active":  False
    })

# Create user group (Enterprise feature)
def create_user_group(name: str, user_ids: list[int],
                      cc_pair_ids: list[int]) -> dict:
    resp = requests.post(f"{BASE}/manage/admin/user-group", headers=HEADERS, json={
        "name":       name,
        "user_ids":   user_ids,
        "cc_pair_ids": cc_pair_ids
    })
    return resp.json()

Common Workflows

Slack Bot Setup

# 1. Create a Slack App at api.slack.com/apps
# 2. Enable Socket Mode and Event Subscriptions
# 3. Add Bot Token Scopes: app_mentions:read, channels:history,
#    chat:write, groups:history, im:history, mpim:history
# 4. Install app to workspace
# 5. Set tokens in .env:
echo "DANSWER_BOT_SLACK_APP_TOKEN=xapp-..." >> .env
echo "DANSWER_BOT_SLACK_BOT_TOKEN=xoxb-..." >> .env

# Restart background worker
docker compose restart background

# In Slack, mention the bot:
# @DanswerBot How do we set up a new microservice?

Connector Sync Monitoring

def monitor_connectors(interval_seconds: int = 300):
    """Log connector health every N seconds."""
    import time
    while True:
        connectors = requests.get(f"{BASE}/connector", headers=HEADERS).json()
        for c in connectors:
            status = get_sync_status(c["id"])
            last_attempt = status.get("last_index_attempt_status", "unknown")
            doc_count    = status.get("docs_indexed", 0)
            print(f"{c['name']:40} status={last_attempt:10} docs={doc_count}")
        time.sleep(interval_seconds)

Backup and Restore

# Backup Postgres database
docker exec danswer-db-1 pg_dump -U postgres danswer > danswer_backup_$(date +%Y%m%d).sql

# Backup Vespa data (indexes)
docker exec danswer-index-1 vespa-visit --everything > vespa_backup.jsonl

# Backup volumes
docker compose down
tar -czf danswer_volumes_$(date +%Y%m%d).tar.gz \
  $(docker volume ls -q | grep danswer)

# Restore Postgres
docker exec -i danswer-db-1 psql -U postgres danswer < danswer_backup.sql

Tips and Best Practices

Tip	Details
Start with high-value connectors	Index Confluence and Slack first — they typically contain the most institutional knowledge
Use document sets for persona scoping	Narrow a persona’s access to relevant connectors to improve precision and reduce noise
Set refresh_freq based on data volatility	Slack/Jira: hourly (3600); Confluence/Notion: daily (86400); GitHub: 30 minutes (1800)
Enable hybrid search	Default hybrid mode outperforms pure semantic or keyword search for most enterprise queries
Monitor Vespa disk usage	Vespa index grows quickly with many connectors; plan for 2-5x raw document size
Use RBAC groups for departments	Create user groups per team and restrict connector access to reduce cross-team data leakage
Test with direct search before personas	Use /search API to verify document indexing quality before routing through an LLM persona
Set conservative LLM temperature	For factual enterprise Q&A, temperature=0.1-0.2 reduces hallucination in GPT-4o/Claude
Grant Slack bot read-only scopes only	channels:history and groups:history are sufficient; avoid write scopes for safety
Update via git pull + docker compose pull	Pull latest images and restart; database migrations run automatically on startup