1337skills.com

Appearance

curl - HTTP Client and Data Transfer ​

Comprehensive curl commands for HTTP requests, API testing, and data transfer across all platforms.

Basic HTTP Requests ​

GET Requests ​

CommandDescription
curl https://api.example.comBasic GET request
curl -v https://api.example.comVerbose output with headers
curl -i https://api.example.comInclude response headers
curl -I https://api.example.comHEAD request only
curl -L https://example.comFollow redirects

POST Requests ​

CommandDescription
curl -X POST https://api.example.comBasic POST request
curl -d "data" https://api.example.comPOST with data
curl -d @file.json https://api.example.comPOST data from file
curl -F "file=@upload.txt" https://api.example.comFile upload

Other HTTP Methods ​

CommandDescription
curl -X PUT -d "data" https://api.example.comPUT request
curl -X DELETE https://api.example.com/item/1DELETE request
curl -X PATCH -d "data" https://api.example.comPATCH request

Headers and Authentication ​

Custom Headers ​

CommandDescription
curl -H "Content-Type: application/json" urlSet content type
curl -H "Authorization: Bearer token" urlBearer token auth
curl -H "User-Agent: MyApp/1.0" urlCustom user agent
curl -H "Accept: application/xml" urlAccept header

Authentication Methods ​

CommandDescription
curl -u username:password urlBasic authentication
curl -u username urlPrompt for password
curl --oauth2-bearer token urlOAuth2 bearer token
curl --digest -u user:pass urlDigest authentication

API Key Authentication ​

bash
# API key in header
curl -H "X-API-Key: your-api-key" https://api.example.com

# API key in query parameter
curl "https://api.example.com?api_key=your-api-key"

# Multiple headers
curl -H "Authorization: Bearer token" \
     -H "Content-Type: application/json" \
     https://api.example.com

Data Formats ​

JSON Data ​

bash
# POST JSON data
curl -X POST \
     -H "Content-Type: application/json" \
     -d '{"name":"John","age":30}' \
     https://api.example.com/users

# JSON from file
curl -X POST \
     -H "Content-Type: application/json" \
     -d @data.json \
     https://api.example.com/users

Form Data ​

bash
# URL-encoded form data
curl -d "name=John&age=30" https://api.example.com/users

# Multipart form data
curl -F "name=John" -F "age=30" https://api.example.com/users

# File upload with form data
curl -F "file=@document.pdf" \
     -F "description=Important document" \
     https://api.example.com/upload

XML Data ​

bash
# POST XML data
curl -X POST \
     -H "Content-Type: application/xml" \
     -d '<user><name>John</name><age>30</age></user>' \
     https://api.example.com/users

File Operations ​

Download Files ​

CommandDescription
curl -O https://example.com/file.zipDownload with original name
curl -o myfile.zip https://example.com/file.zipDownload with custom name
curl -C - -O https://example.com/file.zipResume interrupted download
curl --limit-rate 100k -O urlLimit download speed

Upload Files ​

CommandDescription
curl -T file.txt ftp://server/path/Upload via FTP
curl -F "file=@upload.txt" urlHTTP file upload
curl --upload-file file.txt urlPUT file upload

Multiple Files ​

bash
# Download multiple files
curl -O https://example.com/file1.zip -O https://example.com/file2.zip

# Upload multiple files
curl -F "file1=@doc1.pdf" -F "file2=@doc2.pdf" https://api.example.com/upload

Advanced Options ​

Timeouts and Retries ​

CommandDescription
curl --connect-timeout 10 urlConnection timeout (seconds)
curl --max-time 30 urlMaximum total time
curl --retry 3 urlRetry on failure
curl --retry-delay 5 urlDelay between retries

SSL/TLS Options ​

CommandDescription
curl -k urlIgnore SSL certificate errors
curl --cacert ca.pem urlUse custom CA certificate
curl --cert client.pem urlUse client certificate
curl --tlsv1.2 urlForce TLS version

Proxy and Network ​

CommandDescription
curl --proxy proxy.example.com:8080 urlUse HTTP proxy
curl --socks5 proxy.example.com:1080 urlUse SOCKS5 proxy
curl --interface eth0 urlUse specific network interface
curl --dns-servers 8.8.8.8 urlUse custom DNS servers

Output and Formatting ​

Output Control ​

CommandDescription
curl -s urlSilent mode (no progress)
curl -S urlShow errors even in silent mode
curl -o /dev/null urlDiscard output
curl -w "%{http_code}" urlShow only HTTP status code

Response Information ​

bash
# Get detailed timing information
curl -w "@curl-format.txt" https://example.com

# curl-format.txt content:
#     time_namelookup:  %{time_namelookup}\n
#        time_connect:  %{time_connect}\n
#     time_appconnect:  %{time_appconnect}\n
#    time_pretransfer:  %{time_pretransfer}\n
#       time_redirect:  %{time_redirect}\n
#  time_starttransfer:  %{time_starttransfer}\n
#                     ----------\n
#          time_total:  %{time_total}\n

JSON Processing ​

bash
# Pretty print JSON with jq
curl -s https://api.example.com/users | jq '.'

# Extract specific fields
curl -s https://api.example.com/users | jq '.[] | .name'

# Filter results
curl -s https://api.example.com/users | jq '.[] | select(.age > 25)'

Testing and Debugging ​

API Testing ​

bash
# Test REST API endpoints
curl -X GET https://api.example.com/users
curl -X POST -d '{"name":"John"}' https://api.example.com/users
curl -X PUT -d '{"name":"Jane"}' https://api.example.com/users/1
curl -X DELETE https://api.example.com/users/1

# Test with different content types
curl -H "Accept: application/json" https://api.example.com/users
curl -H "Accept: application/xml" https://api.example.com/users

Performance Testing ​

bash
# Measure response time
curl -w "Total time: %{time_total}s\n" -o /dev/null -s https://example.com

# Test multiple requests
for i in {1..10}; do
  curl -w "%{time_total}\n" -o /dev/null -s https://example.com
done

Error Handling ​

bash
# Check HTTP status codes
http_code=$(curl -s -o /dev/null -w "%{http_code}" https://example.com)
if [ $http_code -eq 200 ]; then
    echo "Success"
else
    echo "Error: HTTP $http_code"
fi

Configuration and Scripts ​

Configuration File ​

bash
# ~/.curlrc configuration file
user-agent = "MyApp/1.0"
connect-timeout = 10
max-time = 30
show-error
silent

Bash Scripting ​

bash
#!/bin/bash
# API testing script

BASE_URL="https://api.example.com"
API_KEY="your-api-key"

# Function to make authenticated requests
api_request() {
    local method=$1
    local endpoint=$2
    local data=$3
    
    curl -X "$method" \
         -H "Authorization: Bearer $API_KEY" \
         -H "Content-Type: application/json" \
         ${data:+-d "$data"} \
         "$BASE_URL$endpoint"
}

# Usage examples
api_request GET "/users"
api_request POST "/users" '{"name":"John","email":"john@example.com"}'

Security Best Practices ​

Secure Authentication ​

bash
# Use environment variables for sensitive data
export API_TOKEN="your-secret-token"
curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com

# Read credentials from file
curl -K credentials.txt https://api.example.com

# credentials.txt:
# header = "Authorization: Bearer your-token"

Certificate Verification ​

bash
# Always verify SSL certificates in production
curl --cacert /path/to/ca-bundle.crt https://api.example.com

# For development only (not recommended for production)
curl -k https://self-signed.example.com

Common Use Cases ​

Web Scraping ​

bash
# Download webpage
curl -L https://example.com > page.html

# Follow redirects and save cookies
curl -L -c cookies.txt -b cookies.txt https://example.com

# Set user agent to avoid blocking
curl -H "User-Agent: Mozilla/5.0 (compatible; bot)" https://example.com

API Integration ​

bash
# GitHub API example
curl -H "Authorization: token your-github-token" \
     https://api.github.com/user/repos

# Weather API example
curl "https://api.openweathermap.org/data/2.5/weather?q=London&appid=your-api-key"

# Slack webhook example
curl -X POST \
     -H "Content-Type: application/json" \
     -d '{"text":"Hello from curl!"}' \
     https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK

File Transfer ​

bash
# Upload to cloud storage
curl -X PUT \
     -H "Authorization: Bearer token" \
     --upload-file document.pdf \
     https://api.storage.com/files/document.pdf

# Download with progress bar
curl --progress-bar -O https://example.com/largefile.zip

Troubleshooting ​

Common Issues ​

ProblemSolution
SSL certificate errorsUse -k for testing, fix certificates for production
Connection timeoutIncrease --connect-timeout value
Slow downloadsUse --limit-rate to control bandwidth
Authentication failuresCheck credentials and authentication method

Debugging Commands ​

bash
# Verbose output for debugging
curl -v https://example.com

# Trace all network activity
curl --trace trace.txt https://example.com

# Show only headers
curl -I https://example.com

# Test connectivity
curl -I --connect-timeout 5 https://example.com