curl - HTTP Client and Data Transfer¶

Comprehensive curl commands for HTTP requests, API testing, and data transfer across all platforms.

Basic HTTP Requests¶

GET Requests¶

Command	Description
`curl https://api.example.com`	Basic GET request
`curl -v https://api.example.com`	Verbose output with headers
`curl -i https://api.example.com`	Include response headers
`curl -I https://api.example.com`	HEAD request only
`curl -L https://example.com`	Follow redirects

POST Requests¶

Command	Description
`curl -X POST https://api.example.com`	Basic POST request
`curl -d "data" https://api.example.com`	POST with data
`curl -d @file.json https://api.example.com`	POST data from file
`curl -F "file=@upload.txt" https://api.example.com`	File upload

Other HTTP Methods¶

Command	Description
`curl -X PUT -d "data" https://api.example.com`	PUT request
`curl -X DELETE https://api.example.com/item/1`	DELETE request
`curl -X PATCH -d "data" https://api.example.com`	PATCH request

Headers and Authentication¶

Custom Headers¶

Command	Description
`curl -H "Content-Type: application/json" url`	Set content type
`curl -H "Authorization: Bearer token" url`	Bearer token auth
`curl -H "User-Agent: MyApp/1.0" url`	Custom user agent
`curl -H "Accept: application/xml" url`	Accept header

Authentication Methods¶

Command	Description
`curl -u username:password url`	Basic authentication
`curl -u username url`	Prompt for password
`curl --oauth2-bearer token url`	OAuth2 bearer token
`curl --digest -u user:pass url`	Digest authentication

API Key Authentication¶

# API key in header
curl -H "X-API-Key: your-api-key" https://api.example.com

# API key in query parameter
curl "https://api.example.com?api_key=your-api-key"

# Multiple headers
curl -H "Authorization: Bearer token" \
     -H "Content-Type: application/json" \
     https://api.example.com

Data Formats¶

JSON Data¶

# POST JSON data
curl -X POST \
     -H "Content-Type: application/json" \
     -d '\\\\{"name":"John","age":30\\\\}' \
     https://api.example.com/users

# JSON from file
curl -X POST \
     -H "Content-Type: application/json" \
     -d @data.json \
     https://api.example.com/users

Form Data¶

# URL-encoded form data
curl -d "name=John&age=30" https://api.example.com/users

# Multipart form data
curl -F "name=John" -F "age=30" https://api.example.com/users

# File upload with form data
curl -F "file=@document.pdf" \
     -F "description=Important document" \
     https://api.example.com/upload

XML Data¶

# POST XML data
curl -X POST \
     -H "Content-Type: application/xml" \
     -d '<user><name>John</name><age>30</age></user>' \
     https://api.example.com/users

File Operations¶

Download Files¶

Command	Description
`curl -O https://example.com/file.zip`	Download with original name
`curl -o myfile.zip https://example.com/file.zip`	Download with custom name
`curl -C - -O https://example.com/file.zip`	Resume interrupted download
`curl --limit-rate 100k -O url`	Limit download speed

Upload Files¶

Command	Description
`curl -T file.txt ftp://server/path/`	Upload via FTP
`curl -F "file=@upload.txt" url`	HTTP file upload
`curl --upload-file file.txt url`	PUT file upload

Multiple Files¶

# Download multiple files
curl -O https://example.com/file1.zip -O https://example.com/file2.zip

# Upload multiple files
curl -F "file1=@doc1.pdf" -F "file2=@doc2.pdf" https://api.example.com/upload

Advanced Options¶

Timeouts and Retries¶

Command	Description
`curl --connect-timeout 10 url`	Connection timeout (seconds)
`curl --max-time 30 url`	Maximum total time
`curl --retry 3 url`	Retry on failure
`curl --retry-delay 5 url`	Delay between retries

SSL/TLS Options¶

Command	Description
`curl -k url`	Ignore SSL certificate errors
`curl --cacert ca.pem url`	Use custom CA certificate
`curl --cert client.pem url`	Use client certificate
`curl --tlsv1.2 url`	Force TLS version

Proxy and Network¶

Command	Description
`curl --proxy proxy.example.com:8080 url`	Use HTTP proxy
`curl --socks5 proxy.example.com:1080 url`	Use SOCKS5 proxy
`curl --interface eth0 url`	Use specific network interface
`curl --dns-servers 8.8.8.8 url`	Use custom DNS servers

Output and Formatting¶

Output Control¶

Command	Description
`curl -s url`	Silent mode (no progress)
`curl -S url`	Show errors even in silent mode
`curl -o /dev/null url`	Discard output
`curl -w "%\\{http_code\\}" url`	Show only HTTP status code

Response Information¶

# Get detailed timing information
curl -w "@curl-format.txt" https://example.com

# curl-format.txt content:
#     time_namelookup:  %\\\\{time_namelookup\\\\}\n
#        time_connect:  %\\\\{time_connect\\\\}\n
#     time_appconnect:  %\\\\{time_appconnect\\\\}\n
#    time_pretransfer:  %\\\\{time_pretransfer\\\\}\n
#       time_redirect:  %\\\\{time_redirect\\\\}\n
#  time_starttransfer:  %\\\\{time_starttransfer\\\\}\n
#                     ----------\n
#          time_total:  %\\\\{time_total\\\\}\n

JSON Processing¶

# Pretty print JSON with jq
curl -s https://api.example.com/users|jq '.'

# Extract specific fields
curl -s https://api.example.com/users|jq '.[]|.name'

# Filter results
curl -s https://api.example.com/users|jq '.[]|select(.age > 25)'

Testing and Debugging¶

API Testing¶

# Test REST API endpoints
curl -X GET https://api.example.com/users
curl -X POST -d '\\\\{"name":"John"\\\\}' https://api.example.com/users
curl -X PUT -d '\\\\{"name":"Jane"\\\\}' https://api.example.com/users/1
curl -X DELETE https://api.example.com/users/1

# Test with different content types
curl -H "Accept: application/json" https://api.example.com/users
curl -H "Accept: application/xml" https://api.example.com/users

Performance Testing¶

# Measure response time
curl -w "Total time: %\\\\{time_total\\\\}s\n" -o /dev/null -s https://example.com

# Test multiple requests
for i in \\\\{1..10\\\\}; do
  curl -w "%\\\\{time_total\\\\}\n" -o /dev/null -s https://example.com
done

Error Handling¶

# Check HTTP status codes
http_code=$(curl -s -o /dev/null -w "%\\\\{http_code\\\\}" https://example.com)
if [ $http_code -eq 200 ]; then
    echo "Success"
else
    echo "Error: HTTP $http_code"
fi

Configuration and Scripts¶

Configuration File¶

# ~/.curlrc configuration file
user-agent = "MyApp/1.0"
connect-timeout = 10
max-time = 30
show-error
silent

Bash Scripting¶

#!/bin/bash
# API testing script

BASE_URL="https://api.example.com"
API_KEY="your-api-key"

# Function to make authenticated requests
api_request() \\\\{
    local method=$1
    local endpoint=$2
    local data=$3

    curl -X "$method" \
         -H "Authorization: Bearer $API_KEY" \
         -H "Content-Type: application/json" \
         $\\\\{data:+-d "$data"\\\\} \
         "$BASE_URL$endpoint"
\\\\}

# Usage examples
api_request GET "/users"
api_request POST "/users" '\\\\{"name":"John","email":"john@example.com"\\\\}'

Security Best Practices¶

Secure Authentication¶

# Use environment variables for sensitive data
export API_TOKEN="your-secret-token"
curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com

# Read credentials from file
curl -K credentials.txt https://api.example.com

# credentials.txt:
# header = "Authorization: Bearer your-token"

Certificate Verification¶

# Always verify SSL certificates in production
curl --cacert /path/to/ca-bundle.crt https://api.example.com

# For development only (not recommended for production)
curl -k https://self-signed.example.com

Common Use Cases¶

Web Scraping¶

# Download webpage
curl -L https://example.com > page.html

# Follow redirects and save cookies
curl -L -c cookies.txt -b cookies.txt https://example.com

# Set user agent to avoid blocking
curl -H "User-Agent: Mozilla/5.0 (compatible; bot)" https://example.com

API Integration¶

# GitHub API example
curl -H "Authorization: token your-github-token" \
     https://api.github.com/user/repos

# Weather API example
curl "https://api.openweathermap.org/data/2.5/weather?q=London&appid=your-api-key"

# Slack webhook example
curl -X POST \
     -H "Content-Type: application/json" \
     -d '\\\\{"text":"Hello from curl!"\\\\}' \
     https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK

File Transfer¶

# Upload to cloud storage
curl -X PUT \
     -H "Authorization: Bearer token" \
     --upload-file document.pdf \
     https://api.storage.com/files/document.pdf

# Download with progress bar
curl --progress-bar -O https://example.com/largefile.zip

Troubleshooting¶

Common Issues¶

Problem	Solution
SSL certificate errors	Use `-k` for testing, fix certificates for production
Connection timeout	Increase `--connect-timeout` value
Slow downloads	Use `--limit-rate` to control bandwidth
Authentication failures	Check credentials and authentication method

Debugging Commands¶

# Verbose output for debugging
curl -v https://example.com

# Trace all network activity
curl --trace trace.txt https://example.com

# Show only headers
curl -I https://example.com

# Test connectivity
curl -I --connect-timeout 5 https://example.com