HTTPX Toolkit Cheat Sheet¶

Im Überblick

HTTPX ist ein schnelles und vielseitiges HTTP-Toolkit, das von Project Discovery entwickelt wurde und mehrere Sonden mithilfe der retryablehttp-Bibliothek ausführen kann. Es ist entworfen, um die Ergebnissicherheit bei erhöhten Gewinden zu erhalten und ist für großflächiges Scannen optimiert. HTTPX kann verwendet werden, um mehrere Sonden auf einer Liste von URLs oder Hosts auszuführen, die ein schnelles Webserver Fingerabdrucken und Probieren ermöglichen.

Was HTTPX von anderen HTTP-Tools unterscheidet, ist seine Vielseitigkeit und Geschwindigkeit. Es kann Tausende von Hosts in Minuten verarbeiten und wertvolle Informationen über jedes Ziel, einschließlich Statuscodes, Titel, Content-Typen, Webserver-Technologien und mehr. HTTPX wird häufig in Aufklärungsphasen von Sicherheitsbewertungen und Bug-Bounty-Jagd verwendet, um schnell interessante Ziele für weitere Untersuchungen zu identifizieren.

HTTPX unterstützt verschiedene Eingabeformate und kann problemlos in eine Pipeline mit anderen Tools integriert werden, was es zu einer wesentlichen Komponente in vielen Sicherheitstest-Workflows macht. Seine Fähigkeit, Ergebnisse basierend auf verschiedenen Kriterien zu filtern, hilft Sicherheitsexperten, sich auf die wichtigsten Ziele zu konzentrieren.

• Installation

Verwenden von Go¶

# Install using Go (requires Go 1.20 or later)
go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest

# Verify installation
httpx -version

Verwenden von Docker¶

# Pull the latest Docker image
docker pull projectdiscovery/httpx:latest

# Run HTTPX using Docker
docker run -it projectdiscovery/httpx:latest -h

Verwendung von Homebrew (macOS)¶

# Install using Homebrew
brew install httpx

# Verify installation
httpx -version

Verwenden von PDTM (Projekt Discovery Tools Manager)¶

# Install PDTM first if not already installed
go install -v github.com/projectdiscovery/pdtm/cmd/pdtm@latest

# Install HTTPX using PDTM
pdtm -i httpx

# Verify installation
httpx -version

Auf Kali Linux

# Install using apt
sudo apt install httpx

# Verify installation
httpx -version

oder Basisnutzung

Probing URLs und Hosts¶

# Probe a single URL
httpx -u https://example.com

# Probe multiple URLs
httpx -u https://example.com,https://projectdiscovery.io

# Probe from a list of URLs/hosts
httpx -l hosts.txt

# Probe from STDIN
cat hosts.txt|httpx

Ausgabeoptionen¶

# Save results to a file
httpx -l hosts.txt -o results.txt

# Output in JSON format
httpx -l hosts.txt -json -o results.json

# Output in CSV format
httpx -l hosts.txt -csv -o results.csv

# Silent mode (only URLs)
httpx -l hosts.txt -silent

Basic Filtering¶

# Filter by status code
httpx -l hosts.txt -status-code 200

# Filter by content length
httpx -l hosts.txt -content-length 100

# Match specific title
httpx -l hosts.txt -title "Dashboard"

# Match specific technology
httpx -l hosts.txt -tech wordpress

/ Fortgeschrittene Nutzung

Port Scanning¶

# Scan default ports (80, 443)
httpx -l hosts.txt

# Scan specific ports
httpx -l hosts.txt -ports 80,443,8080,8443

# Scan top 100 ports
httpx -l hosts.txt -ports top-100

# Scan all ports
httpx -l hosts.txt -ports all

Path Probing¶

# Probe specific paths
httpx -l hosts.txt -path /api/v1,/admin,/login

# Probe from a file containing paths
httpx -l hosts.txt -path-file paths.txt

# Automatically add trailing slash
httpx -l hosts.txt -path /api -add-slash

Protokolloptionen¶

# Force HTTPS
httpx -l hosts.txt -https

# Probe both HTTP and HTTPS
httpx -l hosts.txt -probe

# Skip HTTPS verification
httpx -l hosts.txt -no-verify

Customization anfordern¶

# Set custom headers
httpx -l hosts.txt -H "User-Agent: Mozilla/5.0" -H "Cookie: session=123456"

# Set HTTP method
httpx -l hosts.txt -method POST

# Set request body
httpx -l hosts.txt -method POST -body "username=admin&password=admin"

# Set content type
httpx -l hosts.txt -method POST -H "Content-Type: application/json" -body '\\\\{"username":"admin","password":"admin"\\\\}'

Response Filtering¶

# Match response containing specific string
httpx -l hosts.txt -match-string "admin"

# Match response using regex
httpx -l hosts.txt -match-regex "admin.*panel"

# Filter response not containing string
httpx -l hosts.txt -filter-string "not found"

# Filter response using regex
httpx -l hosts.txt -filter-regex "error|not found"

Screenshot Capture¶

# Capture screenshots
httpx -l hosts.txt -screenshot

# Specify screenshot output directory
httpx -l hosts.txt -screenshot -screenshot-output screenshots/

# Set screenshot timeout
httpx -l hosts.txt -screenshot -screenshot-timeout 20

Technology Detection¶

# Detect web technologies
httpx -l hosts.txt -tech-detect

# Output only specific technologies
httpx -l hosts.txt -tech-detect -match-tech wordpress,nginx

Â Leistungsoptimierung

Concurrency and Rate Limiting¶

# Set concurrency (default: 50)
httpx -l hosts.txt -concurrency 100

# Set rate limit
httpx -l hosts.txt -rate-limit 200

# Set request timeout
httpx -l hosts.txt -timeout 10

Retry and Delay Optionen¶

# Set maximum retries
httpx -l hosts.txt -retries 3

# Set delay between requests
httpx -l hosts.txt -delay 2s

# Set random delay
httpx -l hosts.txt -random-agent

Optimierung für große Scans¶

# Use stream mode for large inputs
httpx -l large-hosts.txt -stream

# Skip default ports probing
httpx -l hosts.txt -no-default-ports

# Skip failed host probes
httpx -l hosts.txt -skip-host-error

Integration mit anderen Tools

Pipeline mit Subfinder¶

# Find subdomains and probe them
subfinder -d example.com|httpx

# Find subdomains, probe them, and check for specific paths
subfinder -d example.com|httpx -path /api,/admin -status-code 200

Pipeline mit Nuclei¶

# Find active hosts and scan for vulnerabilities
httpx -l hosts.txt -silent|nuclei -t cves/

# Find hosts with specific tech and scan for related vulnerabilities
httpx -l hosts.txt -tech-detect -match-tech wordpress -silent|nuclei -t wordpress/

Pipeline mit Naabu¶

# Scan ports and probe HTTP services
naabu -host example.com -top-ports 1000 -silent|httpx

# Scan ports, probe HTTP services, and check for vulnerabilities
naabu -host example.com -top-ports 1000 -silent|httpx -silent|nuclei -t cves/

/ Output Customization

Individuelle Ausgabeformat¶

# Define custom output format
httpx -l hosts.txt -o results.txt -silent -format "\\\\{\\\\{.StatusCode\\\\}\\\\} \\\\{\\\\{.URL\\\\}\\\\} \\\\{\\\\{.Title\\\\}\\\\}"

# Include specific fields in output
httpx -l hosts.txt -include-response-time -include-chain -include-cdn

Response Extraction¶

# Extract title
httpx -l hosts.txt -title

# Extract favicon hash
httpx -l hosts.txt -favicon

# Extract response headers
httpx -l hosts.txt -response-header

# Extract TLS information
httpx -l hosts.txt -tls-grab

Response Storage¶

# Store response bodies
httpx -l hosts.txt -store-response

# Specify response storage directory
httpx -l hosts.txt -store-response -store-response-dir responses/

# Store chain responses
httpx -l hosts.txt -store-chain

Erweiterte Filterung

Status Code Filter¶

# Match specific status codes
httpx -l hosts.txt -status-code 200,301,302

# Filter out specific status codes
httpx -l hosts.txt -exclude-status-code 404,403

Inhalt filtern¶

# Filter by content length
httpx -l hosts.txt -content-length 100

# Match content length range
httpx -l hosts.txt -content-length-lt 1000 -content-length-gt 100

# Filter by content type
httpx -l hosts.txt -content-type "text/html"

Header Filter¶

# Match specific header
httpx -l hosts.txt -match-header "Server: nginx"

# Filter by header presence
httpx -l hosts.txt -include-headers "Server,Content-Type"

Proxy und Netzwerkoptionen

# Use HTTP proxy
httpx -l hosts.txt -proxy http://127.0.0.1:8080

# Use SOCKS5 proxy
httpx -l hosts.txt -proxy socks5://127.0.0.1:1080

# Follow redirects
httpx -l hosts.txt -follow-redirects

# Follow redirects with max depth
httpx -l hosts.txt -follow-redirects -follow-max-redirects 5

# Follow host redirects
httpx -l hosts.txt -follow-host-redirects

In den Warenkorb Eigenschaften

CRLF Injection Detection¶

# Check for CRLF injection
httpx -l hosts.txt -crlf

CORS Misconfiguration Check¶

# Check for CORS misconfigurations
httpx -l hosts.txt -cors

IP Geolocation¶

# Include IP geolocation information
httpx -l hosts.txt -location

Web Cache Detection¶

# Check for web cache
httpx -l hosts.txt -web-cache

Virtual Host Discovery¶

# Probe for virtual hosts
httpx -l hosts.txt -vhost

# Specify vhost wordlist
httpx -l hosts.txt -vhost -vhost-wordlist vhosts.txt

Fehlerbehebung

Häufige Fragen¶

ANHANG Verbrauchszeiten

   # Increase timeout
   httpx -l hosts.txt -timeout 15

   # Increase retries
   httpx -l hosts.txt -retries 3
   ```_

2. **Begrenzung durch Ziel* *
```bash
   # Reduce concurrency
   httpx -l hosts.txt -concurrency 10

   # Add delay between requests
   httpx -l hosts.txt -delay 2s
   ```_

3. **Memory Issues*
```bash
   # Use stream mode for large inputs
   httpx -l large-hosts.txt -stream

   # Reduce concurrency
   httpx -l hosts.txt -concurrency 25
   ```_

4. **TLS/SSL Fehler*
```bash
   # Skip TLS verification
   httpx -l hosts.txt -no-verify
   ```_

### Debugging

```bash
# Enable verbose mode
httpx -l hosts.txt -verbose

# Show request and response details
httpx -l hosts.txt -debug

# Show only failed requests
httpx -l hosts.txt -debug-req -debug-resp -silent

Konfiguration

Konfigurationsdatei¶

HTTPX verwendet eine Konfigurationsdatei unter $HOME/.config/httpx/config.yaml_. Sie können verschiedene Einstellungen in dieser Datei anpassen:

# Example configuration file
concurrency: 50
timeout: 5
retries: 2
rate-limit: 150
verbose: false
silent: false
output: httpx_output.txt

Umgebungsvariablen¶

# Set HTTPX configuration via environment variables
export HTTPX_CONCURRENCY=50
export HTTPX_TIMEOUT=5
export HTTPX_RETRIES=2
export HTTPX_RATE_LIMIT=150

Referenz

Kommandozeilenoptionen¶

Flag	Description
INLINE_CODE_41	Target URL/host to probe
INLINE_CODE_42	File containing list of URLs/hosts to probe
INLINE_CODE_43	File to write output to
INLINE_CODE_44	Write output in JSON format
INLINE_CODE_45	Write output in CSV format
INLINE_CODE_46	Show only URLs/hosts in output
INLINE_CODE_47	Show verbose output
INLINE_CODE_48	Show request/response details
INLINE_CODE_49	Show HTTPX version
INLINE_CODE_50	Ports to probe (default: 80,443)
INLINE_CODE_51	Path(s) to probe
INLINE_CODE_52	HTTP method to use
INLINE_CODE_53	Filter by status code
INLINE_CODE_54	Filter by title
INLINE_CODE_55	Filter by content length
INLINE_CODE_56	Detect web technologies
INLINE_CODE_57	Follow HTTP redirects
INLINE_CODE_58	Skip TLS verification
INLINE_CODE_59	Custom header to add to all requests
INLINE_CODE_60	Match response containing string
INLINE_CODE_61	Match response using regex
INLINE_CODE_62	Filter response not containing string
INLINE_CODE_63	Filter response not matching regex
INLINE_CODE_64	Take screenshots of websites
INLINE_CODE_65	Number of concurrent requests
INLINE_CODE_66	Maximum number of requests per second
INLINE_CODE_67	Timeout in seconds for HTTP requests
INLINE_CODE_68	Number of retries for failed requests
INLINE_CODE_69	Delay between requests
INLINE_CODE_70	HTTP/SOCKS5 proxy to use
_
### Ausgabefelder

Field	Description
INLINE_CODE_71	Target URL
INLINE_CODE_72	Original input
INLINE_CODE_73	URL scheme (http/https)
INLINE_CODE_74	Target host
INLINE_CODE_75	Target port
INLINE_CODE_76	URL path
INLINE_CODE_77	HTTP status code
INLINE_CODE_78	Page title
INLINE_CODE_79	Content type header
INLINE_CODE_80	Content length
INLINE_CODE_81	Response time in seconds
INLINE_CODE_82	Detected technologies
INLINE_CODE_83	Server header
INLINE_CODE_84	Detected web server
INLINE_CODE_85	Target IP address
INLINE_CODE_86	CDN information
INLINE_CODE_87	Favicon hash
INLINE_CODE_88	TLS information
INLINE_CODE_89	Redirect location
INLINE_CODE_90	Virtual host information
_
Ressourcen

offizielle Dokumentation
(GitHub Repository)(https://github.com/projectdiscovery/httpx)
Project Discovery Discord

--

*Dieses Betrügereiblatt bietet eine umfassende Referenz für die Verwendung von HTTPX, vom Grundvorschlag bis hin zur erweiterten Filterung und Integration mit anderen Tools. Für die aktuellsten Informationen finden Sie immer die offizielle Dokumentation. *