Sidekiq Cheat Sheet

Overview

Sidekiq is a full-featured background job processing framework for Ruby that uses threads to handle many jobs simultaneously in the same process. It is backed by Redis for job storage and provides reliable, efficient background processing with a clean API and powerful web UI for monitoring.

Sidekiq supports job retries with exponential backoff, scheduled jobs, job batches, rate limiting, unique jobs, and middleware for cross-cutting concerns. It integrates seamlessly with Ruby on Rails and can process thousands of jobs per second with minimal memory overhead compared to process-based alternatives.

Installation

Gemfile

# Gemfile
gem 'sidekiq'

# Optional additions
gem 'sidekiq-scheduler'   # cron-like scheduling
gem 'sidekiq-unique-jobs'  # job uniqueness
gem 'sidekiq-cron'         # recurring jobs

bundle install

Rails Setup

# config/application.rb
config.active_job.queue_adapter = :sidekiq

# config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0') }
end

Sidekiq.configure_client do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0') }
end

Job Definition

Basic Worker

# app/workers/hard_worker.rb
class HardWorker
  include Sidekiq::Worker

  sidekiq_options queue: 'default', retry: 5, backtrace: true

  def perform(name, count)
    # Do work
    puts "Processing #{name} with count #{count}"
  end
end

ActiveJob Worker

# app/jobs/process_order_job.rb
class ProcessOrderJob < ApplicationJob
  queue_as :high_priority
  retry_on StandardError, wait: :polynomially_longer, attempts: 5
  discard_on ActiveJob::DeserializationError

  def perform(order_id)
    order = Order.find(order_id)
    order.process!
  end
end

Worker Options

class MyWorker
  include Sidekiq::Worker

  sidekiq_options(
    queue: 'critical',       # queue name
    retry: 10,               # max retries (default 25)
    backtrace: true,         # log full backtrace
    dead: true,              # move to dead queue after retries
    lock: :until_executed,   # uniqueness (with sidekiq-unique-jobs)
    tags: ['billing'],       # tags for filtering in UI
  )

  sidekiq_retry_in do |count, exception|
    # Custom retry interval
    10 * (count + 1)  # 10, 20, 30 seconds...
  end

  sidekiq_retries_exhausted do |msg, exception|
    # Called when all retries fail
    Bugsnag.notify(exception)
  end

  def perform(user_id)
    # work
  end
end

Enqueueing Jobs

# Perform asynchronously
HardWorker.perform_async('Bob', 5)

# Perform in the future
HardWorker.perform_in(5.minutes, 'Bob', 5)
HardWorker.perform_at(Time.now + 1.hour, 'Bob', 5)

# Using ActiveJob
ProcessOrderJob.perform_later(order.id)
ProcessOrderJob.set(wait: 5.minutes).perform_later(order.id)
ProcessOrderJob.set(queue: :low).perform_later(order.id)

# Bulk enqueue
args = users.map { |u| [u.id] }
Sidekiq::Client.push_bulk('class' => HardWorker, 'args' => args)

Worker Commands

Command	Description
`bundle exec sidekiq`	Start with defaults
`bundle exec sidekiq -q critical,3 -q default,1`	Start with weighted queues
`bundle exec sidekiq -c 25`	Set concurrency to 25 threads
`bundle exec sidekiq -C config/sidekiq.yml`	Use config file
`bundle exec sidekiq -e production`	Set environment

Configuration

Config File (`config/sidekiq.yml`)

---
:concurrency: 10
:timeout: 25
:queues:
  - [critical, 3]
  - [high, 2]
  - [default, 1]
  - [low, 1]

:limits:
  critical: 5

production:
  :concurrency: 25

staging:
  :concurrency: 10

Scheduled / Recurring Jobs

# config/sidekiq_scheduler.yml (with sidekiq-scheduler)
cleanup_job:
  cron: '0 */6 * * *'    # every 6 hours
  class: CleanupWorker
  queue: maintenance

daily_report:
  cron: '0 8 * * *'      # daily at 8am
  class: DailyReportWorker
  args: ['summary']

heartbeat:
  every: '30s'
  class: HeartbeatWorker

Middleware

# Custom server middleware
class LoggingMiddleware
  def call(worker, job, queue)
    start = Time.now
    yield
    duration = Time.now - start
    Rails.logger.info "#{worker.class} completed in #{duration}s"
  end
end

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add LoggingMiddleware
  end
end

Web UI

# config/routes.rb
require 'sidekiq/web'

Rails.application.routes.draw do
  # With authentication (production)
  authenticate :user, ->(u) { u.admin? } do
    mount Sidekiq::Web => '/sidekiq'
  end
end

# Standalone Rack app
# config.ru
require 'sidekiq/web'

Sidekiq::Web.use(Rack::Auth::Basic) do |user, password|
  [user, password] == ['admin', ENV['SIDEKIQ_PASSWORD']]
end

run Sidekiq::Web

Advanced Usage

Batches (Sidekiq Pro)

batch = Sidekiq::Batch.new
batch.description = "Import users batch"
batch.on(:success, BatchCallback, account_id: account.id)
batch.on(:complete, BatchCallback, account_id: account.id)

batch.jobs do
  users.each { |u| ImportWorker.perform_async(u.id) }
end

Job Filtering and Management

# Find and manage jobs in queues
queue = Sidekiq::Queue.new('default')
queue.size                          # number of jobs
queue.clear                         # delete all jobs
queue.each { |job| job.delete }     # iterate

# Scheduled jobs
ss = Sidekiq::ScheduledSet.new
ss.size
ss.each { |job| job.reschedule(Time.now) }

# Retry set
rs = Sidekiq::RetrySet.new
rs.size
rs.each { |job| job.retry }        # retry now
rs.clear                           # clear all retries

# Dead set
ds = Sidekiq::DeadSet.new
ds.each { |job| job.retry }

Rate Limiting

class ApiWorker
  include Sidekiq::Worker

  LIMITER = Sidekiq::Limiter.concurrent(
    'api-calls', 5, wait_timeout: 10
  )

  def perform(user_id)
    LIMITER.within_limit do
      ApiClient.fetch(user_id)
    end
  end
end

Monitoring

# Get real-time stats
stats = Sidekiq::Stats.new
stats.processed         # total processed
stats.failed            # total failed
stats.enqueued          # currently enqueued
stats.scheduled_size    # scheduled jobs
stats.retry_size        # jobs in retry
stats.dead_size         # dead jobs
stats.processes_size    # running processes
stats.workers_size      # active workers

# Health check endpoint (Sidekiq 7+)
curl http://localhost:7433/

Troubleshooting

Issue	Solution
Jobs not processing	Check Redis connection; ensure Sidekiq process is running
Jobs stuck in retry	Check error in web UI; fix the bug and clear retries
Memory growing	Set `MALLOC_ARENA_MAX=2`; check for memory leaks in workers
Redis connection issues	Increase pool size; verify Redis URL; check connection limits
Jobs running twice	Ensure unique job processing; make jobs idempotent
Slow queue processing	Increase concurrency; add more Sidekiq processes; optimize job code
Dead jobs accumulating	Review `sidekiq_retries_exhausted` callback; fix recurring errors
Deployment issues	Use `Sidekiq.configure_server` to handle connection correctly; use `USR1` signal for quiet