Strace Commands

strace is a diagnostic, debugging and instructional userspace utility that intercepts and records system calls and signals received by a process. It’s essential for understanding how programs interact with the operating system.

Installation

Linux/Ubuntu

sudo apt update
sudo apt install strace

macOS (alternative: dtrace)

brew install strace

Basic Usage

Trace a Program

# Trace simple command
strace ls

# Trace with arguments
strace ls -la /tmp

# Trace program execution
strace ./program

# Trace program with output to file
strace -o output.txt ls

Attach to Running Process

# Find process ID
ps aux | grep program

# Attach strace to PID
strace -p 1234

# Attach without stopping process
strace -p 1234 -f

# Detach after N seconds
timeout 10 strace -p 1234

Output Control

Display Options

# Verbose output (shows all syscalls)
strace -v program

# Very verbose (also shows all arguments)
strace -vv program

# Quiet mode (minimal output)
strace -q program

# Redirect output to file
strace -o trace.txt program

# Append to file instead of overwrite
strace -e trace -o trace.txt -a1 program

# Print to stdout even with -o
strace -e trace program 2>&1 | tee trace.txt

Formatting

# Align output columns
strace -a 40 program

# Show argument count
strace program 2>&1 | head

# Abbreviate long arguments
strace -s 100 program

# Show full argument strings
strace -s 4096 program

# Relative time between calls
strace -r program

# Absolute time for each call
strace -t program

# Time with microseconds
strace -tt program

# Time since first call (relative)
strace -ttt program

Filtering System Calls

Trace Specific Syscalls

# Trace only open/close
strace -e open,close program

# Trace file operations
strace -e openat,read,write,close program

# Trace network calls
strace -e socket,connect,sendto,recvfrom program

# Trace process operations
strace -e fork,execve,clone program

# Trace signal handling
strace -e signal program

# Trace memory operations
strace -e mmap,munmap,brk program

# Trace file metadata
strace -e stat,fstat,lstat program

# Group related syscalls (file)
strace -e trace=file program

# Group related syscalls (network)
strace -e trace=network program

# Group related syscalls (process)
strace -e trace=process program

# Group related syscalls (memory)
strace -e trace=memory program

# All syscalls except specific ones
strace -e trace=!futex program

Advanced Filtering

Multiple Conditions

# Trace syscalls with specific return value
strace -e open,read -e retval=0 program

# Trace only failed syscalls
strace -e trace=file -e retval=-1 program

# Trace calls with specific status
strace -e status=none program

# Trace syscalls with errors only
strace -e signal program

# Filter by syscall range
strace -e %read,%write,%mem program

Call Count and Statistics

# Print summary statistics
strace -c program

# Count syscalls by type
strace -c -e trace=file program

# Summary sorted by calls
strace -c -s 0 program

# Show time spent in each syscall
strace -c program

# Show call frequency
strace -c -e trace=network program

Process Control

Tracing Multiple Processes

# Follow child processes
strace -f program

# Follow fork/clone
strace -f -e trace=fork program

# Trace all threads
strace -f program

# PID prefix for multi-process
strace -f -o output.txt program

# Print which process made call
strace -ff -o output.txt program

# Separate output per process
strace -ff program

Performance Analysis

# Show time in system calls
strace -c program

# Show cumulative time
strace -c program

# Wall-clock time statistics
strace -c program

# Count calls to specific syscall
strace -e trace=open -c program

# Per-thread statistics
strace -f -c program

# Sort by total time
strace -c program | sort -rn -k2

Real-World Examples

Debugging Common Issues

# Find where program is hanging
strace -e trace=all program
# Look for blocked syscalls (without return)

# Debug file not found errors
strace -e openat,open program 2>&1 | grep -E 'ENOENT|\.so'

# Track configuration file loading
strace -e trace=file program -c /path/to/config

# Debug library loading issues
strace -e trace=execve,open,openat program

# Find file access patterns
strace -e trace=file -o trace.txt program
cat trace.txt | grep -E 'open|read|write'

# Debug network connection issues
strace -e trace=network -e verbose=connect program

# Track environment variable usage
strace -e trace=process program

# Find memory leaks
strace -e trace=memory program

# Debug slow startup
strace -c program

Monitoring Applications

# Monitor running service
strace -p $(pgrep apache2) -e trace=network

# Monitor MySQL queries timing
strace -p $(pgrep mysqld) -c

# Track file access patterns
strace -f -e trace=file /usr/bin/application

# Network activity monitoring
strace -f -e trace=network -e verbose=connect program

# System call frequency
strace -f -c program 2>&1 | tail -20

Security Analysis

# Find file operations
strace -e trace=file program

# Find network connections
strace -e trace=network program

# Find process execution
strace -e trace=process program

# Monitor privilege changes
strace -e trace=process program

# Find environment access
strace -e getenv program

Output Parsing and Analysis

# Count failed syscalls
strace -o trace.txt program
grep -E 'ENOENT|EPERM|EACCES' trace.txt | wc -l

# Find failed system calls
strace program 2>&1 | grep -E '\-1|ENOENT|EPERM'

# Show only successful opens
strace -e open,openat program 2>&1 | grep -v '\-1'

# Extract file paths
strace -e trace=file program 2>&1 | grep -oP '"\K[^"]*'

# Monitor specific function calls
strace -e trace=execve program

# Check signal handling
strace -e signal program 2>&1 | grep SIG

# Count syscall types
strace -c program | tail -n +3 | awk '{print $1}' | sort | uniq -c

Performance and Profiling

# Minimal overhead capture
strace -q -e trace=!madvise,futex,epoll_wait program

# Reduce output overhead
strace -q -e trace=file -s 32 program

# Profile function timing
strace -T program

# Wall-clock time per call
strace -t program

# System call timing analysis
strace -T program 2>&1 | awk '{print $NF}' | sort -rn

# Create report of slow calls
strace -T program 2>&1 | awk '$NF > 0.001' | head -20

Troubleshooting Tips

# Find library loading issues
strace -e trace=file program 2>&1 | grep \.so

# Identify permission issues
strace program 2>&1 | grep -i perm

# Debug segmentation faults
strace program

# Find resource exhaustion
strace -c program | grep -E 'mmap|brk|sbrk'

# Monitor file descriptor usage
strace -e trace=fd program

# Check initialization order
strace -e trace=process program

Best Practices

Use filters to reduce output volume and improve readability
Capture output to file for large traces
Use -c for quick statistical overview
Combine with other tools (grep, awk) for analysis
Be aware that strace adds overhead to execution
Use -f carefully as it can generate large output with multi-threaded programs
Clean up trace files after analysis (they can be large)
Use meaningful output file names with timestamps
Document what you’re investigating in trace file names
Consider using perf for CPU-level profiling alongside strace

Last updated: 2026-03-30