XML Processor¶

Installation¶

Platform	Command
Ubuntu/Debian	`wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq && chmod +x /usr/local/bin/yq`
Ubuntu (PPA)	`sudo add-apt-repository ppa:rmescandon/yq && sudo apt update && sudo apt install yq`
macOS (Homebrew)	`brew install yq`
macOS (MacPorts)	`sudo port install yq`
Windows (Chocolatey)	`choco install yq`
Windows (Scoop)	`scoop install yq`
Snap	`snap install yq`
Go	`go install github.com/mikefarah/yq/v4@latest`
Docker	`docker run --rm -v "${PWD}":/workdir mikefarah/yq`
Alpine Linux	`apk add yq`
Arch Linux	`yay -S yq`
Verify Installation	`yq --version`

Basic Commands - Reading & Displaying¶

Command	Description
`yq '.' file.yaml`	Display entire YAML file (pretty print)
`yq '.name' file.yaml`	Read a specific field value
`yq '.metadata.name' file.yaml`	Read nested field using dot notation
`yq '.items[0]' file.yaml`	Access first element of an array
`yq '.items[*]' file.yaml`	Access all elements of an array
`yq '.items[]' file.yaml`	Iterate through array elements
`yq '.items[].name' file.yaml`	Extract specific field from all array elements
`yq '.items[-1]' file.yaml`	Access last element of an array
`yq '.items[1:3]' file.yaml`	Array slice (elements 1 and 2)
`cat file.yaml \\| yq '.spec'`	Read from stdin
`yq -r '.name' file.yaml`	Output raw string (no quotes)
`yq '.items \\| length' file.yaml`	Get array or object length
`yq 'keys' file.yaml`	List all top-level keys
`yq '.[] \\| keys' file.yaml`	List keys of nested objects

Basic Commands - Writing & Updating¶

Command	Description
`yq '.name = "new-value"' file.yaml`	Update a field (prints to stdout)
`yq -i '.name = "new-value"' file.yaml`	Update field in-place (modifies file)
`yq '.spec.replicas = 3' file.yaml`	Update nested value
`yq '.items[0].name = "updated"' file.yaml`	Update array element
`yq '.newField = "value"' file.yaml`	Create new field
`yq '.metadata.labels.env = "prod"' file.yaml`	Create nested field
`yq 'del(.fieldName)' file.yaml`	Delete a field
`yq 'del(.metadata.annotations)' file.yaml`	Delete nested field
`yq 'del(.items[0])' file.yaml`	Delete array element
`yq '.items += {"name": "new"}' file.yaml`	Append to array
`yq '.items = []' file.yaml`	Clear array
`yq '.count += 1' file.yaml`	Increment numeric value
`yq '.total = .price * .quantity' file.yaml`	Arithmetic operations

Advanced Usage - Filtering & Selection¶

Command	Description
`yq '.items[] \\| select(.kind == "Pod")' file.yaml`	Filter items by condition
`yq '.items[] \\| select(.kind == "Pod" and .status == "Running")' file.yaml`	Multiple conditions (AND)
`yq '.items[] \\| select(.kind == "Pod" or .kind == "Service")' file.yaml`	Multiple conditions (OR)
`yq '.items[] \\| select(.name \\| test("^prod-"))' file.yaml`	Pattern matching with regex
`yq '.items[] \\| select(has("metadata"))' file.yaml`	Check if field exists
`yq '.items[] \\| select(.replicas > 3)' file.yaml`	Numeric comparison
`yq '.items[] \\| select(.tags \\| contains(["prod"]))' file.yaml`	Array contains check
`yq '.items[] \\| select(.name != null)' file.yaml`	Filter out null values
`yq '.[] \\| select(tag == "!!str")' file.yaml`	Filter by YAML tag type

Advanced Usage - Array Operations¶

Command	Description
`yq '.items \\|= sort_by(.name)' file.yaml`	Sort array by field
`yq '.items \\|= sort_by(.metadata.creationTimestamp)' file.yaml`	Sort by nested field
`yq '.items \\|= reverse' file.yaml`	Reverse array order
`yq '.tags \\| unique' file.yaml`	Get unique array values
`yq '.items \\| flatten' file.yaml`	Flatten nested arrays
`yq '.items \\| group_by(.kind)' file.yaml`	Group array elements
`yq '.items \\| map(.name)' file.yaml`	Map array to extract values
`yq '.items \\| map(select(.active))' file.yaml`	Filter and map combined
`yq '.items = .items + .newItems' file.yaml`	Concatenate arrays
`yq '.items \\|= unique_by(.name)' file.yaml`	Remove duplicates by field
`yq '[.items[].name]' file.yaml`	Collect values into new array

Advanced Usage - Merging & Combining¶

Command	Description
`yq ea 'select(fi == 0) * select(fi == 1)' f1.yaml f2.yaml`	Deep merge two files (f2 overrides f1)
`yq ea '. as $item ireduce ({}; . * $item)' *.yaml`	Merge multiple files
`yq ea 'select(fi == 0) *+ select(fi == 1)' f1.yaml f2.yaml`	Merge with array concatenation
`yq ea '[.]' file1.yaml file2.yaml`	Combine files into array
`yq '.config = load("config.yaml")' file.yaml`	Load and merge external file
`yq '.spec.template = load("template.yaml").spec' file.yaml`	Load specific path from file
`yq ea 'select(fi == 0) *d select(fi == 1)' f1.yaml f2.yaml`	Deep merge with deletion

Advanced Usage - String Operations¶

Command	Description
`yq '.fullName = .firstName + " " + .lastName' file.yaml`	String concatenation
`yq '.name \\|= upcase' file.yaml`	Convert to uppercase
`yq '.name \\|= downcase' file.yaml`	Convert to lowercase
`yq '.name \\|= trim' file.yaml`	Trim whitespace
`yq '.text \\|= sub("old", "new")' file.yaml`	Replace first occurrence
`yq '.text \\|= gsub("old", "new")' file.yaml`	Replace all occurrences
`yq '.path \\| split("/")' file.yaml`	Split string into array
`yq '.tags \\| join(", ")' file.yaml`	Join array into string
`yq '.name \\| length' file.yaml`	String length
`yq '.text \\| contains("substring")' file.yaml`	Check if string contains substring

Advanced Usage - Format Conversion¶

Command	Description
`yq -o=json '.' file.yaml`	Convert YAML to JSON
`yq -P '.' file.json`	Convert JSON to YAML
`yq -o=xml '.' file.yaml`	Convert YAML to XML
`yq -p=xml '.' file.xml`	Convert XML to YAML
`yq -o=csv '.items[]' file.yaml`	Convert YAML to CSV
`yq -o=props '.' file.yaml`	Convert YAML to properties format
`yq -o=json -I=4 '.' file.yaml`	JSON with custom indentation
`yq -o=yaml --yaml-output-version=1.1 '.' file.yaml`	Specify YAML version
`yq -p=csv -o=json '.' file.csv`	Convert CSV to JSON via YAML

Configuration¶

Command-Line Options¶

# Input/Output format options
-p, --input-format string    Input format (yaml/json/xml/csv/props)
-o, --output-format string   Output format (yaml/json/xml/csv/props)
-P, --prettyPrint           Pretty print (shorthand for -o=yaml)

# Modification options
-i, --inplace               Edit file in place
-I, --indent int            Indentation (default 2)

# Processing options
-e, --exit-status           Exit with status code based on result
-n, --null-input            Don't read input, start with null
-N, --no-colors             Disable colored output
-C, --colors                Force colored output

# Multiple file options
ea, eval-all                Evaluate all files together

Expression Syntax¶

# Basic path expressions
.field                      # Access field
.nested.field              # Nested access
.[0]                       # Array index
.[]                        # Array iteration
.*                         # All fields

# Operators
=                          # Assignment
|=                         # Update assignment
+=                         # Append/increment
*                          # Multiply/merge
+                          # Add/concatenate
-                          # Subtract
//                         # Alternative operator (default value)

# Functions
select()                   # Filter
map()                      # Transform array
has()                      # Check existence
keys                       # Get keys
length                     # Get length
sort_by()                  # Sort array
group_by()                 # Group array
unique                     # Remove duplicates

Environment Variables¶

# Use environment variables in expressions
export APP_NAME="my-app"
yq '.name = env(APP_NAME)' file.yaml

# With default value
yq '.name = (env(NAME) // "default")' file.yaml

# String interpolation
yq '.message = "Hello " + env(USER)' file.yaml

Common Use Cases¶

Use Case 1: Update Kubernetes Deployment Replicas¶

# Single deployment
yq -i '.spec.replicas = 5' deployment.yaml

# Multiple deployments in one file
yq -i '(.spec.replicas | select(. != null)) = 5' deployments.yaml

# Conditionally update specific deployment
yq -i '(select(.metadata.name == "api-server") | .spec.replicas) = 10' deployment.yaml

# Update all deployments in multiple files
yq -i '.spec.replicas = 3' k8s/*.yaml

Use Case 2: Merge Configuration Files¶

# Merge base config with environment-specific overrides
yq ea 'select(fi == 0) * select(fi == 1)' base-config.yaml prod-config.yaml > final-config.yaml

# Merge multiple environment files
yq ea '. as $item ireduce ({}; . * $item)' base.yaml dev.yaml local.yaml > merged.yaml

# Merge with array concatenation
yq ea 'select(fi == 0) *+ select(fi == 1)' config1.yaml config2.yaml > combined.yaml

Use Case 3: Extract and Transform Data¶

# Extract all container images from Kubernetes manifests
yq '.spec.template.spec.containers[].image' deployment.yaml

# Get all service names and ports
yq '.items[] | select(.kind == "Service") | .metadata.name + ":" + (.spec.ports[0].port | tostring)' services.yaml

# Create summary report
yq '.items[] | {"name": .metadata.name, "kind": .kind, "namespace": .metadata.namespace}' resources.yaml -o=json

Use Case 4: Bulk Updates Across Multiple Files¶

# Add label to all resources
find . -name "*.yaml" -exec yq -i '.metadata.labels.environment = "production"' {} \;

# Update image tag in all deployments
yq -i '(.spec.template.spec.containers[].image | select(. == "*:latest")) |= sub(":latest", ":v1.2.3")' k8s/**/*.yaml

# Add annotation to specific resources
yq -i 'select(.kind == "Service") | .metadata.annotations."prometheus.io/scrape" = "true"' *.yaml

Use Case 5: CI/CD Pipeline Configuration¶

# Update version in multiple config files
export VERSION="2.1.0"
yq -i '.version = env(VERSION)' chart/Chart.yaml
yq -i '.image.tag = env(VERSION)' values.yaml

# Inject secrets from environment
yq -i '.database.password = env(DB_PASSWORD)' config.yaml

# Generate environment-specific configs
for env in dev staging prod; do
  yq ea 'select(fi == 0) * select(fi == 1)' base.yaml "env-${env}.yaml" > "config-${env}.yaml"
done

Best Practices¶

Always test without -i first: Run commands without the in-place flag to preview changes before modifying files

yq '.spec.replicas = 5' deployment.yaml  # Preview first
yq -i '.spec.replicas = 5' deployment.yaml  # Then apply

Use version control: Commit files before bulk modifications to easily revert if needed
Quote expressions: Use single quotes for expressions to prevent shell interpretation
```
yq '.items[] | select(.name == "test")' file.yaml  # Correct
```

Validate YAML after modifications: Ensure your changes produce valid YAML

yq '.' modified.yaml > /dev/null && echo "Valid YAML" || echo "Invalid YAML"

Use select() for conditional updates: More precise than updating everything
```
yq '(select(.kind == "Deployment") | .spec.replicas) = 3' file.yaml
```
Preserve comments: yq preserves comments by default, but be careful with complex transformations
Use eval-all for multi-file operations: More efficient than processing files separately
```
yq ea '. as $item ireduce ({}; . * $item)' *.yaml
```

Leverage environment variables: Keep sensitive data out of scripts

export SECRET_KEY="..."
yq '.apiKey = env(SECRET_KEY)' config.yaml

Use raw output for scripting: Use -r flag when piping to other commands

IMAGE=$(yq -r '.spec.template.spec.containers[0].image' deployment.yaml)

Check exit codes for validation: Use -e flag to fail on null/false results

yq -e '.spec.replicas > 0' deployment.yaml && echo "Valid" || echo "Invalid"

Troubleshooting¶

Issue	Solution
Error: "bad file descriptor"	Use `-i` flag correctly or redirect output: `yq '.' file.yaml > temp && mv temp file.yaml`
Changes not persisted	Add `-i` flag for in-place editing: `yq -i '.field = "value"' file.yaml`
"null" appears in output	Field doesn't exist or is null. Use alternative operator: `yq '.field // "default"' file.yaml`
Comments are removed	Use `... comments=""` to explicitly remove, or check if using operations that don't preserve comments
Array merge replaces instead of concatenates	Use `+` instead of `` for merge: `yq ea 'select(fi==0) *+ select(fi==1)' f1.yaml f2.yaml`
"Error: bad expression"	Check expression syntax, ensure proper quoting, verify operators are correct
Output has extra quotes	Use `-r` flag for raw output: `yq -r '.name' file.yaml`
Cannot process multiple files	Use `ea` (eval-all) command: `yq ea '.' file1.yaml file2.yaml`
Wrong version of yq	Verify you have mikefarah/yq (not kislyuk/yq): `yq --version` should show github.com/mikefarah/yq
Permission denied on `-i`	Ensure write permissions: `chmod u+w file.yaml` or run with appropriate privileges
Encoding issues with special characters	Ensure UTF-8 encoding: `yq --encoding=utf-8 '.' file.yaml`
Large files cause memory issues	Process in chunks or use streaming: `yq -N '.items[]' large-file.yaml`
Path not found errors	Verify path exists: `yq 'has("path.to.field")' file.yaml` before accessing
Merge conflicts with complex structures	Use explicit merge strategies: `d` for deep merge with deletion, `+` for array concatenation

Quick Reference - Common Patterns¶

# Read value
yq '.path.to.field' file.yaml

# Update value
yq -i '.path.to.field = "new-value"' file.yaml

# Delete field
yq -i 'del(.path.to.field)' file.yaml

# Filter array
yq '.items[] | select(.name == "target")' file.yaml

# Merge files
yq ea 'select(fi==0) * select(fi==1)' base.yaml override.yaml

# Convert format
yq -o=json '.' file.yaml

# Use environment variable
yq '.field = env(VAR_NAME)' file.yaml

# Multiple operations
yq -i '.field1 = "value1" | .field2 = "value2"' file.yaml