1337skills.com

Appearance

Stenographer - Full Packet Capture System ​

Stenographer is a full-packet-capture utility for buffering packets to disk for intrusion detection and incident response purposes. It provides a simple, powerful interface for storing and retrieving network packets at scale.

Installation ​

Ubuntu/Debian ​

bash
# Install dependencies
sudo apt update
sudo apt install build-essential git libpcap-dev libsnappy-dev libleveldb-dev

# Install Go (if not already installed)
wget https://go.dev/dl/go1.21.0.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.21.0.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin

# Clone and build Stenographer
git clone https://github.com/google/stenographer.git
cd stenographer
go build ./...

# Install binaries
sudo cp stenographer stenoread stenocurl /usr/local/bin/
sudo chmod +x /usr/local/bin/steno*

CentOS/RHEL/Fedora ​

bash
# Install dependencies
sudo yum groupinstall "Development Tools"
sudo yum install git libpcap-devel snappy-devel leveldb-devel

# Install Go
sudo yum install golang

# Clone and build
git clone https://github.com/google/stenographer.git
cd stenographer
go build ./...

# Install binaries
sudo cp stenographer stenoread stenocurl /usr/local/bin/

Docker Installation ​

bash
# Build Docker image
git clone https://github.com/google/stenographer.git
cd stenographer
docker build -t stenographer .

# Run container
docker run -d --name stenographer \
  --net=host \
  --privileged \
  -v /opt/stenographer:/opt/stenographer \
  stenographer

Configuration ​

Basic Configuration ​

bash
# Create configuration directory
sudo mkdir -p /etc/stenographer
sudo mkdir -p /var/lib/stenographer

# Create basic configuration file
sudo tee /etc/stenographer/config << 'EOF'
{
  "Threads": [
    {
      "PacketsDirectory": "/var/lib/stenographer/packets",
      "IndexDirectory": "/var/lib/stenographer/index",
      "MaxDirectoryFiles": 30000,
      "DiskFreePercentage": 10
    }
  ],
  "StenotypePath": "/usr/local/bin/stenotype",
  "Interface": "eth0",
  "Port": 1234,
  "Host": "127.0.0.1",
  "Flags": [],
  "CertPath": "/etc/stenographer/certs"
}
EOF

Advanced Configuration ​

json
{
  "Threads": [
    {
      "PacketsDirectory": "/data/stenographer/packets0",
      "IndexDirectory": "/data/stenographer/index0",
      "MaxDirectoryFiles": 50000,
      "DiskFreePercentage": 5
    },
    {
      "PacketsDirectory": "/data/stenographer/packets1",
      "IndexDirectory": "/data/stenographer/index1",
      "MaxDirectoryFiles": 50000,
      "DiskFreePercentage": 5
    }
  ],
  "StenotypePath": "/usr/local/bin/stenotype",
  "Interface": "eth0",
  "Port": 1234,
  "Host": "0.0.0.0",
  "Flags": [
    "--threads=2",
    "--fanout_type=FANOUT_HASH"
  ],
  "CertPath": "/etc/stenographer/certs",
  "RateLimit": "1000MB",
  "MaxAge": "7d"
}

Certificate Setup ​

bash
# Create certificate directory
sudo mkdir -p /etc/stenographer/certs

# Generate certificates
cd /etc/stenographer/certs
sudo openssl genrsa -out stenographer.key 2048
sudo openssl req -new -x509 -key stenographer.key -out stenographer.crt -days 365 \
  -subj "/C=US/ST=State/L=City/O=Organization/CN=stenographer"

# Set permissions
sudo chown -R stenographer:stenographer /etc/stenographer/certs
sudo chmod 600 /etc/stenographer/certs/*

Basic Usage ​

Starting Stenographer ​

bash
# Start stenographer daemon
sudo stenographer --config /etc/stenographer/config

# Start as systemd service
sudo systemctl start stenographer
sudo systemctl enable stenographer

# Check status
sudo systemctl status stenographer

Basic Packet Retrieval ​

bash
# Query packets by time range
stenocurl -s "2023-01-01T00:00:00Z" -e "2023-01-01T01:00:00Z"

# Query packets by host
stenocurl -q "host 192.168.1.100"

# Query packets by port
stenocurl -q "port 80"

# Query packets by protocol
stenocurl -q "tcp"

# Combine queries
stenocurl -q "host 192.168.1.100 and port 80"

Output Formats ​

bash
# Output to PCAP file
stenocurl -q "host 192.168.1.100" -w output.pcap

# Output to stdout
stenocurl -q "port 443" | tcpdump -r -

# Count packets only
stenocurl -q "tcp" -c

# Limit number of packets
stenocurl -q "udp" -n 1000

Query Language ​

Basic Filters ​

bash
# Host filters
stenocurl -q "host 10.0.0.1"
stenocurl -q "src host 10.0.0.1"
stenocurl -q "dst host 10.0.0.1"

# Network filters
stenocurl -q "net 192.168.1.0/24"
stenocurl -q "src net 10.0.0.0/8"
stenocurl -q "dst net 172.16.0.0/12"

# Port filters
stenocurl -q "port 80"
stenocurl -q "src port 1234"
stenocurl -q "dst port 443"
stenocurl -q "portrange 1000-2000"

Protocol Filters ​

bash
# Protocol types
stenocurl -q "tcp"
stenocurl -q "udp"
stenocurl -q "icmp"
stenocurl -q "ip"
stenocurl -q "ip6"

# Application protocols
stenocurl -q "http"
stenocurl -q "https"
stenocurl -q "dns"
stenocurl -q "ssh"
stenocurl -q "ftp"

Advanced Queries ​

bash
# Logical operators
stenocurl -q "host 10.0.0.1 and port 80"
stenocurl -q "tcp or udp"
stenocurl -q "not port 22"
stenocurl -q "(host 10.0.0.1 or host 10.0.0.2) and port 443"

# Packet size filters
stenocurl -q "greater 1500"
stenocurl -q "less 64"
stenocurl -q "len = 1518"

# TCP flags
stenocurl -q "tcp[tcpflags] & tcp-syn != 0"
stenocurl -q "tcp[tcpflags] & tcp-ack != 0"
stenocurl -q "tcp[tcpflags] & tcp-rst != 0"

Time-based Queries ​

bash
# Specific time range
stenocurl -s "2023-01-01T10:00:00Z" -e "2023-01-01T11:00:00Z" -q "tcp"

# Last hour
stenocurl -s "$(date -d '1 hour ago' -Iseconds)" -q "dns"

# Last 24 hours
stenocurl -s "$(date -d '1 day ago' -Iseconds)" -q "http"

# Combine time and filters
stenocurl -s "2023-01-01T00:00:00Z" -e "2023-01-01T23:59:59Z" -q "host 192.168.1.100 and tcp"

Advanced Features ​

Multi-threaded Capture ​

json
{
  "Threads": [
    {
      "PacketsDirectory": "/fast-disk/packets0",
      "IndexDirectory": "/fast-disk/index0",
      "MaxDirectoryFiles": 100000,
      "DiskFreePercentage": 5
    },
    {
      "PacketsDirectory": "/fast-disk/packets1",
      "IndexDirectory": "/fast-disk/index1",
      "MaxDirectoryFiles": 100000,
      "DiskFreePercentage": 5
    }
  ],
  "Flags": [
    "--threads=4",
    "--fanout_type=FANOUT_HASH",
    "--blocks=8192",
    "--blocksize=2097152"
  ]
}

Performance Tuning ​

bash
# Optimize network interface
sudo ethtool -G eth0 rx 4096 tx 4096
sudo ethtool -K eth0 gro off lro off tso off gso off

# Set CPU affinity
sudo taskset -c 0,1 stenographer --config /etc/stenographer/config

# Increase buffer sizes
echo 'net.core.rmem_max = 134217728' | sudo tee -a /etc/sysctl.conf
echo 'net.core.rmem_default = 134217728' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Storage Management ​

bash
# Monitor disk usage
stenoread --stats

# Clean old packets
stenoread --clean --before "2023-01-01T00:00:00Z"

# Verify packet integrity
stenoread --verify

# Export statistics
stenoread --stats --json > stenographer_stats.json

Integration with Security Tools ​

Zeek Integration ​

bash
# Configure Zeek to use Stenographer
# In local.zeek:
@load policy/misc/capture-loss

redef Pcap::snaplen = 65535;
redef Pcap::bufsize = 128;

# Extract packets for Zeek analysis
stenocurl -q "tcp and port 80" -w http_traffic.pcap
zeek -r http_traffic.pcap

Suricata Integration ​

bash
# Extract packets for Suricata analysis
stenocurl -q "tcp" -s "$(date -d '1 hour ago' -Iseconds)" -w recent_tcp.pcap

# Run Suricata on extracted packets
suricata -r recent_tcp.pcap -c /etc/suricata/suricata.yaml -l /var/log/suricata/

Wireshark Integration ​

bash
# Extract packets for Wireshark analysis
stenocurl -q "host 192.168.1.100" -w investigation.pcap

# Open in Wireshark
wireshark investigation.pcap

# Use tshark for command-line analysis
tshark -r investigation.pcap -T fields -e ip.src -e ip.dst -e tcp.port

SIEM Integration ​

bash
# Export packet metadata to JSON
stenocurl -q "dns" -j | jq '.[] | {timestamp, src_ip, dst_ip, protocol}'

# Send to Elasticsearch
stenocurl -q "http" -j | curl -X POST "localhost:9200/packets/_bulk" \
  -H "Content-Type: application/json" --data-binary @-

# Integration with Splunk
stenocurl -q "tcp" -j | splunk add oneshot -sourcetype stenographer

Monitoring and Alerting ​

Health Monitoring ​

bash
#!/bin/bash
# Stenographer health check script

# Check if stenographer is running
if ! pgrep -f stenographer > /dev/null; then
    echo "CRITICAL: Stenographer is not running"
    exit 2
fi

# Check disk space
DISK_USAGE=$(df /var/lib/stenographer | tail -1 | awk '{print $5}' | sed 's/%//')
if [ $DISK_USAGE -gt 90 ]; then
    echo "WARNING: Disk usage is ${DISK_USAGE}%"
    exit 1
fi

# Check packet capture rate
STATS=$(stenoread --stats --json)
PACKETS_PER_SEC=$(echo $STATS | jq '.packets_per_second')
if [ $(echo "$PACKETS_PER_SEC < 100" | bc) -eq 1 ]; then
    echo "WARNING: Low packet capture rate: $PACKETS_PER_SEC pps"
    exit 1
fi

echo "OK: Stenographer is healthy"
exit 0

Performance Monitoring ​

bash
# Monitor capture statistics
watch -n 5 'stenoread --stats'

# Monitor system resources
iostat -x 1 | grep -E "(Device|stenographer)"
top -p $(pgrep stenographer)

# Network interface statistics
watch -n 1 'cat /proc/net/dev | grep eth0'

Automated Cleanup ​

bash
#!/bin/bash
# Automated cleanup script

# Configuration
MAX_AGE_DAYS=7
DISK_THRESHOLD=85
PACKETS_DIR="/var/lib/stenographer/packets"

# Check disk usage
DISK_USAGE=$(df $PACKETS_DIR | tail -1 | awk '{print $5}' | sed 's/%//')

if [ $DISK_USAGE -gt $DISK_THRESHOLD ]; then
    echo "Disk usage ${DISK_USAGE}% exceeds threshold ${DISK_THRESHOLD}%"
    
    # Clean packets older than MAX_AGE_DAYS
    CUTOFF_DATE=$(date -d "${MAX_AGE_DAYS} days ago" -Iseconds)
    stenoread --clean --before "$CUTOFF_DATE"
    
    echo "Cleaned packets older than $CUTOFF_DATE"
fi

Troubleshooting ​

Common Issues ​

bash
# Check stenographer logs
sudo journalctl -u stenographer -f

# Verify configuration
stenographer --config /etc/stenographer/config --check

# Test network interface
sudo tcpdump -i eth0 -c 10

# Check permissions
ls -la /var/lib/stenographer/
sudo chown -R stenographer:stenographer /var/lib/stenographer/

Performance Issues ​

bash
# Check for packet drops
cat /proc/net/dev | grep eth0
ethtool -S eth0 | grep drop

# Monitor CPU usage
top -p $(pgrep stenographer)

# Check I/O wait
iostat -x 1

# Verify disk performance
dd if=/dev/zero of=/var/lib/stenographer/test bs=1M count=1000 oflag=direct

Network Issues ​

bash
# Test connectivity
stenocurl -q "icmp" -c

# Check interface configuration
ip addr show eth0
ip route show

# Verify capture filter
tcpdump -i eth0 -d "tcp and port 80"

Security Best Practices ​

Access Control ​

bash
# Create dedicated user
sudo useradd -r -s /bin/false stenographer

# Set file permissions
sudo chown -R stenographer:stenographer /var/lib/stenographer/
sudo chmod 750 /var/lib/stenographer/
sudo chmod 640 /etc/stenographer/config

# Restrict certificate access
sudo chmod 600 /etc/stenographer/certs/*

Network Security ​

bash
# Use TLS for remote access
stenocurl --cert /etc/stenographer/certs/client.crt \
         --key /etc/stenographer/certs/client.key \
         --ca /etc/stenographer/certs/ca.crt \
         -q "tcp"

# Firewall configuration
sudo ufw allow from 192.168.1.0/24 to any port 1234
sudo ufw deny 1234

Data Protection ​

bash
# Encrypt packet storage
sudo cryptsetup luksFormat /dev/sdb1
sudo cryptsetup luksOpen /dev/sdb1 stenographer-data
sudo mkfs.ext4 /dev/mapper/stenographer-data
sudo mount /dev/mapper/stenographer-data /var/lib/stenographer/

# Secure deletion
sudo shred -vfz -n 3 /var/lib/stenographer/packets/*

This comprehensive Stenographer cheatsheet covers installation, configuration, usage, and integration with security tools. Stenographer provides powerful full-packet capture capabilities for network security monitoring and incident response operations.