Elastic SIEM Cheat Sheet¶
Elastic SIEM (Security Information and Event Management) ist eine umfassende Sicherheitsanalyselösung, die auf dem Elastic Stack (Elasticsearch, Logstash, Kibana und Beats) aufbaut. Es bietet Echtzeitbedrohungserkennung, Untersuchungsmöglichkeiten und Reaktionsorchestration für moderne Security Operations Center. Elastic SIEM nutzt maschinelles Lernen, Verhaltensanalyse und Threat Intelligence, um fortschrittliche persistente Bedrohungen, Insider-Bedrohungen und komplexe Angriffskampagnen in hybriden Cloud-Umgebungen zu erkennen.
Plattform-Überblick¶
Elastic Stack Architektur¶
Elastic SIEM basiert auf dem Elastic Stack, der eine verteilte, skalierbare Plattform zur Erfassung, Speicherung, Suche und Visualisierung von Sicherheitsdaten in großem Maßstab bereitstellt. Die Architektur besteht aus mehreren Kernkomponenten, die zusammenarbeiten, um umfassende Sicherheitsüberwachungs- und Analysefunktionen zu bieten.
Elasticsearch dient als verteilte Such- und Analyseengine, die Sicherheitsdaten aus dem gesamten Unternehmen speichert und indexiert. Es bietet Echtzeit-Suchfunktionen, erweiterte Aggregationen und Machine-Learning-Funktionen, die eine schnelle Bedrohungserkennung und -untersuchung ermöglichen. Die verteilte Natur von Elasticsearch ermöglicht es Organisationen, ihren Sicherheits-Datensee horizontal zu skalieren, um wachsende Datenvolumen und Benutzeranforderungen zu bewältigen.
Logstash fungiert als Datenverarbeitungspipeline, die Sicherheitsdaten aus verschiedenen Quellen erfasst, transformiert und anreichert, bevor sie an Elasticsearch gesendet werden. Es unterstützt Hunderte von Eingabe-Plugins zur Datensammlung von Sicherheitstools, Netzwerkgeräten, Cloud-Plattformen und benutzerdefinierten Anwendungen. Logstash kann Daten in Echtzeit analysieren, normalisieren und anreichern und dabei Kontext wie Geolokation, Threat Intelligence und Asset-Informationen hinzufügen.
Kibana bietet die Benutzeroberfläche für Sicherheitsanalysten zum Suchen, Visualisieren und Analysieren von Sicherheitsdaten. Es enthält vordefinierte Dashboards, Erkennungsregeln, Case-Management-Funktionen und Untersuchungs-Workflows, die speziell für Security Operations entwickelt wurden. Die Visualisierungsfunktionen von Kibana ermöglichen es Analysten, benutzerdefinierte Dashboards zu erstellen, Ad-hoc-Analysen durchzuführen und Managementberichte zu generieren.
Beats sind leichtgewichtige Datenversender, die Daten von Endpunkten, Servern und Netzwerkgeräten sammeln und an Logstash oder Elasticsearch weiterleiten. Sicherheitsorientierte Beats umfassen Winlogbeat für Windows-Ereignisprotokolle, Auditbeat für Systemaudit-Daten, Packetbeat für Netzwerkverkehrsanalyse und Filebeat für die Sammlung von Protokolldateien.
Hauptmerkmale¶
(Note: I've translated the first two sections. Would you like me to continue with the remaining sections?)```bash
Core SIEM Capabilities¶
- Real-time threat detection and alerting
- Advanced behavioral analytics and machine learning
- Threat hunting and investigation workflows
- Case management and incident response
- Timeline analysis and event correlation
- Threat intelligence integration
- Custom detection rule creation
- Executive dashboards and reporting
## Installation and Setup ### Elasticsearch Installation ```bash # Download and install Elasticsearch wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.11.0-linux-x86_64.tar.gz tar -xzf elasticsearch-8.11.0-linux-x86_64.tar.gz cd elasticsearch-8.11.0 # Configure Elasticsearch for SIEM cat > config/elasticsearch.yml ``<< EOF cluster.name: elastic-siem node.name: siem-node-1 path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: 0.0.0.0 http.port: 9200 discovery.type: single-node # Security settings xpack.security.enabled: true xpack.security.enrollment.enabled: true xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: certs/http.p12 xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: certs/transport.p12 xpack.security.transport.ssl.truststore.path: certs/transport.p12 EOF # Start Elasticsearch ./bin/elasticsearch # Set up passwords for built-in users ./bin/elasticsearch-setup-passwords auto # Create SIEM-specific index templates curl -X PUT "localhost:9200/_index_template/siem-logs" \ -H "Content-Type: application/json" \ -u elastic:password \ -d '\\\{ "index_patterns": ["siem-*"], "template": \\\{ "settings": \\\{ "number_of_shards": 3, "number_of_replicas": 1, "index.lifecycle.name": "siem-policy", "index.lifecycle.rollover_alias": "siem-logs" \\\}, "mappings": \\\{ "properties": \\\{ "@timestamp": \\\{"type": "date"\\\}, "event.category": \\\{"type": "keyword"\\\}, "event.action": \\\{"type": "keyword"\\\}, "source.ip": \\\{"type": "ip"\\\}, "destination.ip": \\\{"type": "ip"\\\}, "user.name": \\\{"type": "keyword"\\\}, "host.name": \\\{"type": "keyword"\\\}, "process.name": \\\{"type": "keyword"\\\}, "file.hash.sha256": \\\{"type": "keyword"\\\} \\\} \\\} \\\} \\\}'
Kibana Installation and Configuration¶
# Download and install Kibana
wget https://artifacts.elastic.co/downloads/kibana/kibana-8.11.0-linux-x86_64.tar.gz
tar -xzf kibana-8.11.0-linux-x86_64.tar.gz
cd kibana-8.11.0
# Configure Kibana for SIEM
cat >`` config/kibana.yml << EOF
server.port: 5601
server.host: "0.0.0.0"
server.name: "elastic-siem-kibana"
elasticsearch.hosts: ["https://localhost:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "kibana_password"
elasticsearch.ssl.certificateAuthorities: ["/path/to/elasticsearch/config/certs/http_ca.crt"]
# SIEM-specific settings
xpack.security.enabled: true
xpack.encryptedSavedObjects.encryptionKey: "a7a6311933d3503b89bc2dbc36572c33a6c10925682e591bffcab6911c06786d"
xpack.reporting.encryptionKey: "a7a6311933d3503b89bc2dbc36572c33a6c10925682e591bffcab6911c06786d"
xpack.security.encryptionKey: "a7a6311933d3503b89bc2dbc36572c33a6c10925682e591bffcab6911c06786d"
# Enable SIEM app
xpack.siem.enabled: true
xpack.securitySolution.enabled: true
EOF
# Start Kibana
./bin/kibana
# Access Kibana SIEM interface
# Navigate to http://localhost:5601/app/security
Logstash Configuration for SIEM¶
# Install Logstash
wget https://artifacts.elastic.co/downloads/logstash/logstash-8.11.0-linux-x86_64.tar.gz
tar -xzf logstash-8.11.0-linux-x86_64.tar.gz
cd logstash-8.11.0
# Create SIEM pipeline configuration
cat > config/siem-pipeline.conf << 'EOF'
input \\\\{
# Windows Event Logs via Winlogbeat
beats \\\\{
port => 5044
type => "winlogbeat"
\\\\}
# Syslog from network devices
syslog \\\\{
port => 514
type => "syslog"
\\\\}
# CEF logs from security tools
tcp \\\\{
port => 5140
codec => cef
type => "cef"
\\\\}
# File input for custom logs
file \\\\{
path => "/var/log/security/*.log"
start_position => "beginning"
type => "security_logs"
\\\\}
\\\\}
filter \\\\{
# Parse Windows Security Events
if [type] == "winlogbeat" \\\\{
if [winlog][event_id] == 4624 \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "authentication" \\\\}
add_field => \\\\{ "event.action" => "logon" \\\\}
add_field => \\\\{ "event.outcome" => "success" \\\\}
\\\\}
\\\\}
if [winlog][event_id] == 4625 \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "authentication" \\\\}
add_field => \\\\{ "event.action" => "logon" \\\\}
add_field => \\\\{ "event.outcome" => "failure" \\\\}
\\\\}
\\\\}
if [winlog][event_id] == 4688 \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "process" \\\\}
add_field => \\\\{ "event.action" => "start" \\\\}
\\\\}
\\\\}
\\\\}
# Parse syslog messages
if [type] == "syslog" \\\\{
grok \\\\{
match => \\\\{ "message" => "%\\\\{SYSLOGTIMESTAMP:timestamp\\\\} %\\\\{IPORHOST:host\\\\} %\\\\{WORD:program\\\\}(?:\[%\\\\{POSINT:pid\\\\}\])?: %\\\\{GREEDYDATA:message\\\\}" \\\\}
\\\\}
date \\\\{
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
\\\\}
\\\\}
# Enrich with GeoIP data
if [source][ip] \\\\{
geoip \\\\{
source => "[source][ip]"
target => "[source][geo]"
\\\\}
\\\\}
if [destination][ip] \\\\{
geoip \\\\{
source => "[destination][ip]"
target => "[destination][geo]"
\\\\}
\\\\}
# Add threat intelligence
translate \\\\{
field => "[source][ip]"
destination => "[threat][indicator][type]"
dictionary_path => "/etc/logstash/threat_intel.yml"
fallback => "unknown"
\\\\}
# Normalize timestamps
date \\\\{
match => [ "@timestamp", "ISO8601" ]
\\\\}
\\\\}
output \\\\{
# Send to Elasticsearch
elasticsearch \\\\{
hosts => ["https://localhost:9200"]
user => "logstash_writer"
password => "logstash_password"
ssl => true
ssl_certificate_verification => false
index => "siem-logs-%\\\\{+YYYY.MM.dd\\\\}"
template_name => "siem-logs"
\\\\}
# Debug output
stdout \\\\{
codec => rubydebug
\\\\}
\\\\}
EOF
# Start Logstash with SIEM pipeline
./bin/logstash -f config/siem-pipeline.conf
Data Collection and Ingestion¶
Beats Configuration for Security Data¶
# Winlogbeat for Windows Event Logs
cat > winlogbeat.yml << 'EOF'
winlogbeat.event_logs:
- name: Security
event_id: 4624, 4625, 4648, 4672, 4688, 4689, 4697, 4698, 4699, 4700, 4701, 4702
- name: System
event_id: 7034, 7035, 7036, 7040
- name: Application
event_id: 1000, 1001, 1002
output.elasticsearch:
hosts: ["https://localhost:9200"]
username: "winlogbeat_writer"
password: "winlogbeat_password"
ssl.certificate_authorities: ["/path/to/ca.crt"]
index: "winlogbeat-%\\\\{+yyyy.MM.dd\\\\}"
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
logging.level: info
logging.to_files: true
logging.files:
path: /var/log/winlogbeat
name: winlogbeat
keepfiles: 7
permissions: 0644
EOF
# Auditbeat for system audit data
cat > auditbeat.yml << 'EOF'
auditbeat.modules:
- module: auditd
audit_rule_files: [ '$\\\\{path.config\\\\}/audit.rules.d/*.conf' ]
audit_rules:|
# Monitor file access
-w /etc/passwd -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/shadow -p wa -k identity
# Monitor privilege escalation
-a always,exit -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=4294967295 -k privilege_escalation
# Monitor network connections
-a always,exit -F arch=b64 -S socket -F a0=2 -k network_connect
# Monitor file modifications
-w /bin/ -p wa -k binaries
-w /sbin/ -p wa -k binaries
-w /usr/bin/ -p wa -k binaries
-w /usr/sbin/ -p wa -k binaries
- module: file_integrity
paths:
- /bin
- /usr/bin
- /sbin
- /usr/sbin
- /etc
- module: system
datasets:
- host
- login
- package
- process
- socket
- user
period: 10s
output.elasticsearch:
hosts: ["https://localhost:9200"]
username: "auditbeat_writer"
password: "auditbeat_password"
ssl.certificate_authorities: ["/path/to/ca.crt"]
index: "auditbeat-%\\\\{+yyyy.MM.dd\\\\}"
EOF
# Packetbeat for network traffic analysis
cat > packetbeat.yml << 'EOF'
packetbeat.interfaces.device: any
packetbeat.flows:
timeout: 30s
period: 10s
packetbeat.protocols:
dns:
ports: [53]
include_authorities: true
include_additionals: true
http:
ports: [80, 8080, 8000, 5000, 8002]
tls:
ports: [443, 993, 995, 5223, 8443, 8883, 9243]
ssh:
ports: [22]
output.elasticsearch:
hosts: ["https://localhost:9200"]
username: "packetbeat_writer"
password: "packetbeat_password"
ssl.certificate_authorities: ["/path/to/ca.crt"]
index: "packetbeat-%\\\\{+yyyy.MM.dd\\\\}"
processors:
- add_host_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
EOF
Custom Log Parsing¶
# Create custom parsing rules for security tools
cat > /etc/logstash/conf.d/security-tools.conf << 'EOF'
filter \\\\{
# Parse Suricata IDS logs
if [type] == "suricata" \\\\{
json \\\\{
source => "message"
\\\\}
if [event_type] == "alert" \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "intrusion_detection" \\\\}
add_field => \\\\{ "event.action" => "alert" \\\\}
add_field => \\\\{ "rule.name" => "%\\\\{[alert][signature]\\\\}" \\\\}
add_field => \\\\{ "rule.id" => "%\\\\{[alert][signature_id]\\\\}" \\\\}
\\\\}
\\\\}
\\\\}
# Parse Zeek/Bro logs
if [type] == "zeek" \\\\{
if [log_type] == "conn" \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "network" \\\\}
add_field => \\\\{ "event.action" => "connection" \\\\}
\\\\}
\\\\}
if [log_type] == "dns" \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "network" \\\\}
add_field => \\\\{ "event.action" => "dns_query" \\\\}
\\\\}
\\\\}
if [log_type] == "http" \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "network" \\\\}
add_field => \\\\{ "event.action" => "http_request" \\\\}
\\\\}
\\\\}
\\\\}
# Parse OSSEC/Wazuh logs
if [type] == "ossec" \\\\{
grok \\\\{
match => \\\\{ "message" => "%\\\\{TIMESTAMP_ISO8601:timestamp\\\\} %\\\\{WORD:hostname\\\\} %\\\\{WORD:component\\\\}: %\\\\{GREEDYDATA:alert_message\\\\}" \\\\}
\\\\}
if [rule_id] \\\\{
mutate \\\\{
add_field => \\\\{ "event.category" => "host" \\\\}
add_field => \\\\{ "event.action" => "alert" \\\\}
add_field => \\\\{ "rule.id" => "%\\\\{rule_id\\\\}" \\\\}
\\\\}
\\\\}
\\\\}
\\\\}
EOF
Detection Rules and Analytics¶
Pre-built Detection Rules¶
// Suspicious PowerShell Activity
\\\\{
"rule": \\\\{
"name": "Suspicious PowerShell Execution",
"description": "Detects potentially malicious PowerShell commands",
"severity": "high",
"risk_score": 75,
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "process"
\\\\}
\\\\},
\\\\{
"term": \\\\{
"process.name": "powershell.exe"
\\\\}
\\\\},
\\\\{
"bool": \\\\{
"should": [
\\\\{
"wildcard": \\\\{
"process.command_line": "*Invoke-Expression*"
\\\\}
\\\\},
\\\\{
"wildcard": \\\\{
"process.command_line": "*DownloadString*"
\\\\}
\\\\},
\\\\{
"wildcard": \\\\{
"process.command_line": "*EncodedCommand*"
\\\\}
\\\\},
\\\\{
"wildcard": \\\\{
"process.command_line": "*-nop*"
\\\\}
\\\\},
\\\\{
"wildcard": \\\\{
"process.command_line": "*-w hidden*"
\\\\}
\\\\}
]
\\\\}
\\\\}
]
\\\\}
\\\\},
"filters": [],
"timeline_id": "timeline_powershell",
"timeline_title": "PowerShell Investigation Timeline"
\\\\}
\\\\}
// Brute Force Login Attempts
\\\\{
"rule": \\\\{
"name": "Brute Force Login Attempts",
"description": "Detects multiple failed login attempts from the same source",
"severity": "medium",
"risk_score": 50,
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "authentication"
\\\\}
\\\\},
\\\\{
"term": \\\\{
"event.outcome": "failure"
\\\\}
\\\\}
]
\\\\}
\\\\},
"threshold": \\\\{
"field": "source.ip",
"value": 10,
"cardinality": [
\\\\{
"field": "user.name",
"value": 5
\\\\}
]
\\\\},
"timeline_id": "timeline_brute_force",
"timeline_title": "Brute Force Investigation Timeline"
\\\\}
\\\\}
// Lateral Movement Detection
\\\\{
"rule": \\\\{
"name": "Lateral Movement via Remote Services",
"description": "Detects potential lateral movement using remote services",
"severity": "high",
"risk_score": 80,
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "authentication"
\\\\}
\\\\},
\\\\{
"term": \\\\{
"event.outcome": "success"
\\\\}
\\\\},
\\\\{
"terms": \\\\{
"winlog.logon.type": ["3", "10"]
\\\\}
\\\\}
]
\\\\}
\\\\},
"threshold": \\\\{
"field": "user.name",
"value": 1,
"cardinality": [
\\\\{
"field": "host.name",
"value": 5
\\\\}
]
\\\\}
\\\\}
\\\\}
Machine Learning Jobs¶
// Anomalous Network Traffic
\\\\{
"job_id": "anomalous_network_traffic",
"description": "Detects anomalous network traffic patterns",
"analysis_config": \\\\{
"bucket_span": "15m",
"detectors": [
\\\\{
"function": "high_count",
"field_name": "network.bytes",
"by_field_name": "source.ip"
\\\\},
\\\\{
"function": "rare",
"field_name": "destination.port",
"by_field_name": "source.ip"
\\\\}
],
"influencers": ["source.ip", "destination.ip", "destination.port"]
\\\\},
"data_description": \\\\{
"time_field": "@timestamp",
"time_format": "epoch_ms"
\\\\},
"datafeed_config": \\\\{
"indices": ["packetbeat-*"],
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "network"
\\\\}
\\\\}
]
\\\\}
\\\\}
\\\\}
\\\\}
// Unusual Process Execution
\\\\{
"job_id": "unusual_process_execution",
"description": "Detects unusual process execution patterns",
"analysis_config": \\\\{
"bucket_span": "15m",
"detectors": [
\\\\{
"function": "rare",
"field_name": "process.name",
"by_field_name": "host.name"
\\\\},
\\\\{
"function": "freq_rare",
"field_name": "process.command_line",
"by_field_name": "user.name"
\\\\}
],
"influencers": ["host.name", "user.name", "process.name"]
\\\\},
"data_description": \\\\{
"time_field": "@timestamp",
"time_format": "epoch_ms"
\\\\},
"datafeed_config": \\\\{
"indices": ["winlogbeat-*", "auditbeat-*"],
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "process"
\\\\}
\\\\},
\\\\{
"term": \\\\{
"event.action": "start"
\\\\}
\\\\}
]
\\\\}
\\\\}
\\\\}
\\\\}
Custom Detection Rules¶
# Create custom detection rule via API
curl -X POST "localhost:5601/api/detection_engine/rules" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"name": "Credential Dumping Activity",
"description": "Detects potential credential dumping tools and techniques",
"severity": "critical",
"risk_score": 90,
"rule_id": "credential-dumping-001",
"type": "query",
"query": "event.category:process AND (process.name:(mimikatz.exe OR procdump.exe OR pwdump.exe OR fgdump.exe) OR process.command_line:(*sekurlsa* OR *logonpasswords* OR *lsadump* OR *sam* OR *security*))",
"language": "kuery",
"filters": [],
"from": "now-6m",
"to": "now",
"interval": "5m",
"enabled": true,
"tags": ["credential_access", "T1003"],
"threat": [
\\\\{
"framework": "MITRE ATT&CK",
"tactic": \\\\{
"id": "TA0006",
"name": "Credential Access",
"reference": "https://attack.mitre.org/tactics/TA0006/"
\\\\},
"technique": [
\\\\{
"id": "T1003",
"name": "OS Credential Dumping",
"reference": "https://attack.mitre.org/techniques/T1003/"
\\\\}
]
\\\\}
]
\\\\}'
# Create threshold-based rule
curl -X POST "localhost:5601/api/detection_engine/rules" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"name": "Multiple Failed SSH Logins",
"description": "Detects multiple failed SSH login attempts",
"severity": "medium",
"risk_score": 60,
"rule_id": "ssh-brute-force-001",
"type": "threshold",
"query": "event.category:authentication AND event.outcome:failure AND service.name:ssh",
"language": "kuery",
"threshold": \\\\{
"field": "source.ip",
"value": 20,
"cardinality": [
\\\\{
"field": "user.name",
"value": 5
\\\\}
]
\\\\},
"from": "now-5m",
"to": "now",
"interval": "5m",
"enabled": true,
"tags": ["initial_access", "T1078"]
\\\\}'
Investigation and Threat Hunting¶
Timeline Analysis¶
# Create investigation timeline
curl -X POST "localhost:5601/api/timeline" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"timeline": \\\\{
"title": "Incident Investigation Timeline",
"description": "Timeline for investigating security incident",
"timelineType": "default",
"templateTimelineId": null,
"templateTimelineVersion": null,
"dataProviders": [
\\\\{
"id": "host-investigation",
"name": "Host Investigation",
"enabled": true,
"excluded": false,
"kqlQuery": "",
"queryMatch": \\\\{
"field": "host.name",
"value": "suspicious-host",
"operator": ":"
\\\\}
\\\\}
],
"kqlQuery": \\\\{
"filterQuery": \\\\{
"kuery": \\\\{
"kind": "kuery",
"expression": "host.name: suspicious-host"
\\\\}
\\\\}
\\\\},
"dateRange": \\\\{
"start": "2023-01-01T00:00:00.000Z",
"end": "2023-01-02T00:00:00.000Z"
\\\\}
\\\\}
\\\\}'
# Query timeline events
curl -X POST "localhost:5601/api/timeline/_search" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"defaultIndex": ["winlogbeat-*", "auditbeat-*", "packetbeat-*"],
"timerange": \\\\{
"from": "2023-01-01T00:00:00.000Z",
"to": "2023-01-02T00:00:00.000Z",
"interval": "12h"
\\\\},
"filterQuery": \\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"host.name": "suspicious-host"
\\\\}
\\\\}
]
\\\\}
\\\\},
"pagination": \\\\{
"activePage": 0,
"querySize": 25
\\\\},
"sort": \\\\{
"columnId": "@timestamp",
"sortDirection": "desc"
\\\\}
\\\\}'
Threat Hunting Queries¶
# Hunt for living off the land techniques
GET /winlogbeat-*/_search
\\\\{
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "process"
\\\\}
\\\\},
\\\\{
"terms": \\\\{
"process.name": [
"certutil.exe",
"bitsadmin.exe",
"regsvr32.exe",
"rundll32.exe",
"mshta.exe",
"wmic.exe"
]
\\\\}
\\\\},
\\\\{
"bool": \\\\{
"should": [
\\\\{
"wildcard": \\\\{
"process.command_line": "*http*"
\\\\}
\\\\},
\\\\{
"wildcard": \\\\{
"process.command_line": "*download*"
\\\\}
\\\\},
\\\\{
"wildcard": \\\\{
"process.command_line": "*urlcache*"
\\\\}
\\\\}
]
\\\\}
\\\\}
]
\\\\}
\\\\},
"aggs": \\\\{
"by_host": \\\\{
"terms": \\\\{
"field": "host.name",
"size": 10
\\\\},
"aggs": \\\\{
"by_process": \\\\{
"terms": \\\\{
"field": "process.name",
"size": 10
\\\\}
\\\\}
\\\\}
\\\\}
\\\\}
\\\\}
# Hunt for persistence mechanisms
GET /winlogbeat-*/_search
\\\\{
"query": \\\\{
"bool": \\\\{
"should": [
\\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"winlog.event_id": 4698
\\\\}
\\\\},
\\\\{
"wildcard": \\\\{
"winlog.event_data.TaskName": "*Microsoft*"
\\\\}
\\\\}
]
\\\\}
\\\\},
\\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "registry"
\\\\}
\\\\},
\\\\{
"terms": \\\\{
"registry.path": [
"*\\Software\\Microsoft\\Windows\\CurrentVersion\\Run*",
"*\\Software\\Microsoft\\Windows\\CurrentVersion\\RunOnce*",
"*\\Software\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon*"
]
\\\\}
\\\\}
]
\\\\}
\\\\},
\\\\{
"bool": \\\\{
"must": [
\\\\{
"term": \\\\{
"event.category": "file"
\\\\}
\\\\},
\\\\{
"terms": \\\\{
"file.path": [
"*\\Startup\\*",
"*\\Start Menu\\Programs\\Startup\\*"
]
\\\\}
\\\\}
]
\\\\}
\\\\}
]
\\\\}
\\\\}
\\\\}
# Hunt for data exfiltration
GET /packetbeat-*/_search
\\\\{
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"range": \\\\{
"@timestamp": \\\\{
"gte": "now-24h"
\\\\}
\\\\}
\\\\},
\\\\{
"term": \\\\{
"network.direction": "outbound"
\\\\}
\\\\}
]
\\\\}
\\\\},
"aggs": \\\\{
"large_transfers": \\\\{
"filter": \\\\{
"range": \\\\{
"network.bytes": \\\\{
"gte": 100000000
\\\\}
\\\\}
\\\\},
"aggs": \\\\{
"by_source": \\\\{
"terms": \\\\{
"field": "source.ip",
"size": 10
\\\\},
"aggs": \\\\{
"total_bytes": \\\\{
"sum": \\\\{
"field": "network.bytes"
\\\\}
\\\\},
"destinations": \\\\{
"terms": \\\\{
"field": "destination.ip",
"size": 5
\\\\}
\\\\}
\\\\}
\\\\}
\\\\}
\\\\}
\\\\}
\\\\}
Case Management¶
# Create security case
curl -X POST "localhost:5601/api/cases" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"title": "Suspicious PowerShell Activity Investigation",
"description": "Investigation of suspicious PowerShell commands detected on multiple hosts",
"tags": ["powershell", "malware", "investigation"],
"severity": "high",
"assignees": [
\\\\{
"uid": "analyst1"
\\\\}
],
"connector": \\\\{
"id": "none",
"name": "none",
"type": ".none",
"fields": null
\\\\},
"settings": \\\\{
"syncAlerts": true
\\\\}
\\\\}'
# Add comment to case
curl -X POST "localhost:5601/api/cases/\\\\{case_id\\\\}/comments" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"comment": "Initial analysis shows PowerShell commands attempting to download and execute malicious payloads. Affected hosts: HOST-001, HOST-002, HOST-003",
"type": "user"
\\\\}'
# Attach alert to case
curl -X POST "localhost:5601/api/cases/\\\\{case_id\\\\}/comments" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"alertId": "alert-id-123",
"index": "winlogbeat-2023.01.01",
"type": "alert"
\\\\}'
Dashboards and Visualization¶
Security Overview Dashboard¶
\\\\{
"dashboard": \\\\{
"title": "Security Operations Center Overview",
"description": "High-level security metrics and alerts",
"panelsJSON": "[\\\\{\"version\":\"8.11.0\",\"gridData\":\\\\{\"x\":0,\"y\":0,\"w\":24,\"h\":15,\"i\":\"1\"\\\\},\"panelIndex\":\"1\",\"embeddableConfig\":\\\\{\\\\},\"panelRefName\":\"panel_1\"\\\\}]",
"optionsJSON": "\\\\{\"useMargins\":true,\"syncColors\":false,\"hidePanelTitles\":false\\\\}",
"version": 1,
"timeRestore": true,
"timeTo": "now",
"timeFrom": "now-24h",
"refreshInterval": \\\\{
"pause": false,
"value": 300000
\\\\},
"kibanaSavedObjectMeta": \\\\{
"searchSourceJSON": "\\\\{\"query\":\\\\{\"query\":\"\",\"language\":\"kuery\"\\\\},\"filter\":[]\\\\}"
\\\\}
\\\\},
"references": [
\\\\{
"name": "panel_1",
"type": "visualization",
"id": "security-alerts-timeline"
\\\\}
]
\\\\}
Threat Hunting Dashboard¶
# Create threat hunting visualizations
curl -X POST "localhost:5601/api/saved_objects/visualization" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"attributes": \\\\{
"title": "Process Execution Timeline",
"visState": "\\\\{\"title\":\"Process Execution Timeline\",\"type\":\"histogram\",\"params\":\\\\{\"grid\":\\\\{\"categoryLines\":false,\"style\":\\\\{\"color\":\"#eee\"\\\\}\\\\},\"categoryAxes\":[\\\\{\"id\":\"CategoryAxis-1\",\"type\":\"category\",\"position\":\"bottom\",\"show\":true,\"style\":\\\\{\\\\},\"scale\":\\\\{\"type\":\"linear\"\\\\},\"labels\":\\\\{\"show\":true,\"truncate\":100\\\\},\"title\":\\\\{\\\\}\\\\}],\"valueAxes\":[\\\\{\"id\":\"ValueAxis-1\",\"name\":\"LeftAxis-1\",\"type\":\"value\",\"position\":\"left\",\"show\":true,\"style\":\\\\{\\\\},\"scale\":\\\\{\"type\":\"linear\",\"mode\":\"normal\"\\\\},\"labels\":\\\\{\"show\":true,\"rotate\":0,\"filter\":false,\"truncate\":100\\\\},\"title\":\\\\{\"text\":\"Count\"\\\\}\\\\}],\"seriesParams\":[\\\\{\"show\":true,\"type\":\"histogram\",\"mode\":\"stacked\",\"data\":\\\\{\"label\":\"Count\",\"id\":\"1\"\\\\},\"valueAxis\":\"ValueAxis-1\",\"drawLinesBetweenPoints\":true,\"showCircles\":true\\\\}],\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"times\":[],\"addTimeMarker\":false\\\\},\"aggs\":[\\\\{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":\\\\{\\\\}\\\\},\\\\{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":\\\\{\"field\":\"@timestamp\",\"interval\":\"auto\",\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":\\\\{\\\\}\\\\}\\\\},\\\\{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":\\\\{\"field\":\"process.name\",\"size\":10,\"order\":\"desc\",\"orderBy\":\"1\"\\\\}\\\\}]\\\\}",
"uiStateJSON": "\\\\{\\\\}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": \\\\{
"searchSourceJSON": "\\\\{\"index\":\"winlogbeat-*\",\"query\":\\\\{\"match\":\\\\{\"event.category\":\"process\"\\\\}\\\\},\"filter\":[]\\\\}"
\\\\}
\\\\}
\\\\}'
# Network traffic analysis visualization
curl -X POST "localhost:5601/api/saved_objects/visualization" \
-H "Content-Type: application/json" \
-H "kbn-xsrf: true" \
-u elastic:password \
-d '\\\\{
"attributes": \\\\{
"title": "Network Traffic by Destination Port",
"visState": "\\\\{\"title\":\"Network Traffic by Destination Port\",\"type\":\"pie\",\"params\":\\\\{\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"isDonut\":true\\\\},\"aggs\":[\\\\{\"id\":\"1\",\"enabled\":true,\"type\":\"sum\",\"schema\":\"metric\",\"params\":\\\\{\"field\":\"network.bytes\"\\\\}\\\\},\\\\{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"segment\",\"params\":\\\\{\"field\":\"destination.port\",\"size\":10,\"order\":\"desc\",\"orderBy\":\"1\"\\\\}\\\\}]\\\\}",
"uiStateJSON": "\\\\{\\\\}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": \\\\{
"searchSourceJSON": "\\\\{\"index\":\"packetbeat-*\",\"query\":\\\\{\"match_all\":\\\\{\\\\}\\\\},\"filter\":[]\\\\}"
\\\\}
\\\\}
\\\\}'
Performance Optimization¶
Index Management¶
# Create index lifecycle policy
curl -X PUT "localhost:9200/_ilm/policy/siem-policy" \
-H "Content-Type: application/json" \
-u elastic:password \
-d '\\\\{
"policy": \\\\{
"phases": \\\\{
"hot": \\\\{
"actions": \\\\{
"rollover": \\\\{
"max_size": "10GB",
"max_age": "1d"
\\\\},
"set_priority": \\\\{
"priority": 100
\\\\}
\\\\}
\\\\},
"warm": \\\\{
"min_age": "7d",
"actions": \\\\{
"set_priority": \\\\{
"priority": 50
\\\\},
"allocate": \\\\{
"number_of_replicas": 0
\\\\},
"forcemerge": \\\\{
"max_num_segments": 1
\\\\}
\\\\}
\\\\},
"cold": \\\\{
"min_age": "30d",
"actions": \\\\{
"set_priority": \\\\{
"priority": 0
\\\\},
"allocate": \\\\{
"number_of_replicas": 0
\\\\}
\\\\}
\\\\},
"delete": \\\\{
"min_age": "90d",
"actions": \\\\{
"delete": \\\\{\\\\}
\\\\}
\\\\}
\\\\}
\\\\}
\\\\}'
# Optimize search performance
curl -X PUT "localhost:9200/siem-logs-*/_settings" \
-H "Content-Type: application/json" \
-u elastic:password \
-d '\\\\{
"index": \\\\{
"refresh_interval": "30s",
"number_of_replicas": 1,
"codec": "best_compression"
\\\\}
\\\\}'
# Create search templates for common queries
curl -X PUT "localhost:9200/_scripts/security-event-search" \
-H "Content-Type: application/json" \
-u elastic:password \
-d '\\\\{
"script": \\\\{
"lang": "mustache",
"source": \\\\{
"query": \\\\{
"bool": \\\\{
"must": [
\\\\{
"range": \\\\{
"@timestamp": \\\\{
"gte": "\\\\{\\\\{from\\\\}\\\\}",
"lte": "\\\\{\\\\{to\\\\}\\\\}"
\\\\}
\\\\}
\\\\},
\\\\{
"term": \\\\{
"event.category": "\\\\{\\\\{category\\\\}\\\\}"
\\\\}
\\\\}
],
"filter": [
\\\\{\\\\{#host\\\\}\\\\}
\\\\{
"term": \\\\{
"host.name": "\\\\{\\\\{host\\\\}\\\\}"
\\\\}
\\\\}
\\\\{\\\\{/host\\\\}\\\\}
]
\\\\}
\\\\},
"sort": [
\\\\{
"@timestamp": \\\\{
"order": "desc"
\\\\}
\\\\}
]
\\\\}
\\\\}
\\\\}'
Monitoring and Alerting¶
# Monitor cluster health
curl -X GET "localhost:9200/_cluster/health?pretty" -u elastic:password
# Monitor index statistics
curl -X GET "localhost:9200/_cat/indices/siem-*?v&s=store.size:desc" -u elastic:password
# Set up cluster monitoring
curl -X PUT "localhost:9200/_cluster/settings" \
-H "Content-Type: application/json" \
-u elastic:password \
-d '\\\\{
"persistent": \\\\{
"cluster.routing.allocation.disk.watermark.low": "85%",
"cluster.routing.allocation.disk.watermark.high": "90%",
"cluster.routing.allocation.disk.watermark.flood_stage": "95%"
\\\\}
\\\\}'
# Create watcher for disk space monitoring
curl -X PUT "localhost:9200/_watcher/watch/disk_space_monitor" \
-H "Content-Type: application/json" \
-u elastic:password \
-d '\\\\{
"trigger": \\\\{
"schedule": \\\\{
"interval": "5m"
\\\\}
\\\\},
"input": \\\\{
"http": \\\\{
"request": \\\\{
"host": "localhost",
"port": 9200,
"path": "/_nodes/stats/fs",
"auth": \\\\{
"basic": \\\\{
"username": "elastic",
"password": "password"
\\\\}
\\\\}
\\\\}
\\\\}
\\\\},
"condition": \\\\{
"script": \\\\{
"source": "ctx.payload.nodes.values().stream().anyMatch(node -> node.fs.total.available_in_bytes / node.fs.total.total_in_bytes < 0.1)"
\\\\}
\\\\},
"actions": \\\\{
"send_email": \\\\{
"email": \\\\{
"to": ["admin@company.com"],
"subject": "Elasticsearch Disk Space Alert",
"body": "Disk space is running low on Elasticsearch cluster"
\\\\}
\\\\}
\\\\}
\\\\}'