sh - POSIX Shell

The POSIX shell (sh) represents the standardized foundation of Unix shell programming, defined by the IEEE POSIX.1 standard. As the most portable and widely available shell across Unix-like systems, sh provides the essential features and syntax that form the basis for shell scripting. While modern shells like Bash and Zsh offer extensive enhancements, understanding POSIX sh is crucial for writing portable scripts that work across different systems, from embedded devices to enterprise servers. The POSIX shell ensures maximum compatibility and serves as the common denominator for shell programming across diverse Unix environments.

POSIX Shell Fundamentals

Understanding POSIX Compliance

# Check if shell is POSIX compliant
echo $0
/bin/sh

# Verify POSIX mode (if using bash as sh)
set -o posix

# Check shell features
command -v command >/dev/null 2>&1 && echo "command builtin available"
test -n "${BASH_VERSION}" && echo "Running under Bash"
test -n "${ZSH_VERSION}" && echo "Running under Zsh"

Basic Shell Invocation

# Execute shell script
sh script.sh
sh -x script.sh                # Debug mode
sh -n script.sh                # Syntax check only
sh -e script.sh                # Exit on error
sh -u script.sh                # Exit on undefined variable

# Interactive shell
sh -i                          # Force interactive mode
sh -l                          # Login shell
sh -s                          # Read from stdin

# Combining options
sh -eux script.sh              # Exit on error, undefined vars, debug

Shell Options and Settings

# Set shell options
set -e                         # Exit on error
set -u                         # Exit on undefined variable
set -x                         # Debug mode (print commands)
set -v                         # Verbose mode (print input)
set -n                         # No execution (syntax check)

# Unset options
set +e                         # Don't exit on error
set +u                         # Allow undefined variables
set +x                         # Disable debug mode

# Check if option is set
case $- in
    *e*) echo "Exit on error is set" ;;
    *) echo "Exit on error is not set" ;;
esac

Variables and Parameter Expansion

Variable Assignment and Usage

# Variable assignment (no spaces around =)
name="John Doe"
age=30
path="/home/user"

# Using variables
echo $name
echo ${name}
echo "Hello, $name"
echo 'Literal: $name'          # Single quotes prevent expansion

# Special variables
echo $0                        # Script name
echo $1                        # First argument
echo $2                        # Second argument
echo $#                        # Number of arguments
echo $@                        # All arguments as separate words
echo $*                        # All arguments as single word
echo $$                        # Process ID
echo $?                        # Exit status of last command
echo $!                        # PID of last background job

Parameter Expansion

# Basic parameter expansion
echo ${variable}
echo ${variable:-default}      # Use default if unset or null
echo ${variable:=default}      # Set default if unset or null
echo ${variable:+alternate}    # Use alternate if set and not null
echo ${variable:?error}        # Error if unset or null

# String length
string="Hello, World!"
echo ${#string}                # 13

# Substring removal (POSIX)
filename="document.txt"
echo ${filename%.*}            # Remove shortest match from end: document
echo ${filename%%.*}           # Remove longest match from end: document
echo ${filename#*.}            # Remove shortest match from beginning: txt
echo ${filename##*.}           # Remove longest match from beginning: txt

# Pattern matching examples
path="/usr/local/bin/command"
echo ${path%/*}                # /usr/local/bin (dirname equivalent)
echo ${path##*/}               # command (basename equivalent)

Environment Variables

# Set environment variables
export PATH="/usr/local/bin:$PATH"
export EDITOR="vi"
export LANG="en_US.UTF-8"

# Unset variables
unset variable_name
unset PATH                     # Dangerous!

# Check if variable is set
if [ -n "${VARIABLE+set}" ]; then
    echo "VARIABLE is set"
fi

# Default values for environment
: ${HOME:=/tmp}                # Set HOME to /tmp if not set
: ${USER:=nobody}              # Set USER to nobody if not set

Command Execution and Substitution

Command Substitution

# POSIX command substitution (preferred)
current_date=$(date)
file_count=$(ls | wc -l)
user_home=$(eval echo ~$USER)

# Backtick substitution (legacy, avoid in new scripts)
current_date=`date`
file_count=`ls | wc -l`

# Nested command substitution
echo "Today is $(date +%A), $(date +%B) $(date +%d)"

# Command substitution in conditionals
if [ "$(whoami)" = "root" ]; then
    echo "Running as root"
fi

Command Execution

# Simple commands
ls -l
echo "Hello, World!"
date +"%Y-%m-%d %H:%M:%S"

# Command with arguments
grep "pattern" file.txt
find /path -name "*.txt" -type f

# Background execution
long_running_command &
background_pid=$!
echo "Started background job with PID: $background_pid"

# Wait for background jobs
wait $background_pid
wait                           # Wait for all background jobs

Pipeline and Redirection

# Pipes
ls -l | grep "txt"
ps aux | grep "process" | awk '{print $2}'
cat file.txt | sort | uniq

# Output redirection
command > file.txt             # Redirect stdout
command 2> error.log           # Redirect stderr
command > output.log 2>&1      # Redirect both stdout and stderr
command >> file.txt            # Append stdout

# Input redirection
command < input.txt
sort < unsorted.txt > sorted.txt

# Here documents
cat << EOF
This is a here document
Variables like $HOME are expanded
EOF

# Here documents with quoted delimiter (no expansion)
cat << 'EOF'
This is a literal here document
Variables like $HOME are not expanded
EOF

Control Structures

Conditional Statements

# if-then-else
if [ condition ]; then
    echo "Condition is true"
elif [ other_condition ]; then
    echo "Other condition is true"
else
    echo "No condition is true"
fi

# Test conditions
if [ "$var" = "value" ]; then          # String equality
    echo "String match"
fi

if [ "$var" != "value" ]; then         # String inequality
    echo "String mismatch"
fi

if [ -z "$var" ]; then                 # String is empty
    echo "Variable is empty"
fi

if [ -n "$var" ]; then                 # String is not empty
    echo "Variable is not empty"
fi

# Numeric comparisons
if [ "$num" -eq 10 ]; then             # Equal
    echo "Number is 10"
fi

if [ "$num" -ne 10 ]; then             # Not equal
    echo "Number is not 10"
fi

if [ "$num" -gt 5 ]; then              # Greater than
    echo "Number is greater than 5"
fi

if [ "$num" -lt 20 ]; then             # Less than
    echo "Number is less than 20"
fi

if [ "$num" -ge 5 ]; then              # Greater than or equal
    echo "Number is 5 or greater"
fi

if [ "$num" -le 20 ]; then             # Less than or equal
    echo "Number is 20 or less"
fi

File Test Operators

# File existence and type tests
if [ -e "file" ]; then                 # File exists
    echo "File exists"
fi

if [ -f "file" ]; then                 # Regular file
    echo "Is a regular file"
fi

if [ -d "directory" ]; then            # Directory
    echo "Is a directory"
fi

if [ -L "link" ]; then                 # Symbolic link
    echo "Is a symbolic link"
fi

if [ -p "pipe" ]; then                 # Named pipe
    echo "Is a named pipe"
fi

if [ -S "socket" ]; then               # Socket
    echo "Is a socket"
fi

# File permission tests
if [ -r "file" ]; then                 # Readable
    echo "File is readable"
fi

if [ -w "file" ]; then                 # Writable
    echo "File is writable"
fi

if [ -x "file" ]; then                 # Executable
    echo "File is executable"
fi

if [ -s "file" ]; then                 # File has size > 0
    echo "File is not empty"
fi

# File comparison
if [ "file1" -nt "file2" ]; then       # file1 newer than file2
    echo "file1 is newer"
fi

if [ "file1" -ot "file2" ]; then       # file1 older than file2
    echo "file1 is older"
fi

if [ "file1" -ef "file2" ]; then       # Same file (hard links)
    echo "Files are the same"
fi

Logical Operators

# AND operator
if [ condition1 ] && [ condition2 ]; then
    echo "Both conditions are true"
fi

# OR operator
if [ condition1 ] || [ condition2 ]; then
    echo "At least one condition is true"
fi

# NOT operator
if ! [ condition ]; then
    echo "Condition is false"
fi

# Complex logical expressions
if [ "$age" -ge 18 ] && [ "$age" -le 65 ]; then
    echo "Working age"
fi

if [ "$status" = "active" ] || [ "$force" = "true" ]; then
    echo "Proceeding"
fi

Case Statements

# Basic case statement
case $variable in
    pattern1)
        echo "Matched pattern1"
        ;;
    pattern2|pattern3)
        echo "Matched pattern2 or pattern3"
        ;;
    *)
        echo "No pattern matched"
        ;;
esac

# Case with file extensions
case $filename in
    *.txt)
        echo "Text file"
        ;;
    *.jpg|*.png|*.gif)
        echo "Image file"
        ;;
    *.sh)
        echo "Shell script"
        ;;
    *)
        echo "Unknown file type"
        ;;
esac

# Case with command line options
case $1 in
    -h|--help)
        show_help
        ;;
    -v|--version)
        show_version
        ;;
    -*)
        echo "Unknown option: $1"
        exit 1
        ;;
    *)
        process_file "$1"
        ;;
esac

Loops and Iteration

For Loops

# For loop with list
for item in apple banana orange; do
    echo "Fruit: $item"
done

# For loop with file glob
for file in *.txt; do
    echo "Processing: $file"
done

# For loop with command substitution
for user in $(cat users.txt); do
    echo "User: $user"
done

# For loop with positional parameters
for arg in "$@"; do
    echo "Argument: $arg"
done

# For loop with range (if seq is available)
for i in $(seq 1 10); do
    echo "Number: $i"
done

# C-style for loop (not POSIX, but common)
# Use while loop instead for POSIX compliance
i=1
while [ $i -le 10 ]; do
    echo "Counter: $i"
    i=$((i + 1))
done

While Loops

# Basic while loop
counter=1
while [ $counter -le 10 ]; do
    echo "Counter: $counter"
    counter=$((counter + 1))
done

# While loop reading from file
while read line; do
    echo "Line: $line"
done < file.txt

# While loop with IFS
while IFS=: read user pass uid gid gecos home shell; do
    echo "User: $user, Home: $home, Shell: $shell"
done < /etc/passwd

# Infinite loop
while true; do
    echo "Press Ctrl+C to stop"
    sleep 1
done

# While loop with condition
while [ -f "lockfile" ]; do
    echo "Waiting for lock to be released..."
    sleep 5
done

Until Loops

# Basic until loop
counter=1
until [ $counter -gt 10 ]; do
    echo "Counter: $counter"
    counter=$((counter + 1))
done

# Until loop waiting for condition
until [ -f "ready.flag" ]; do
    echo "Waiting for ready flag..."
    sleep 2
done

# Until loop with command
until ping -c 1 google.com >/dev/null 2>&1; do
    echo "Waiting for network connection..."
    sleep 5
done

Loop Control

# Break and continue
for i in $(seq 1 10); do
    if [ $i -eq 5 ]; then
        continue                # Skip iteration
    fi
    if [ $i -eq 8 ]; then
        break                   # Exit loop
    fi
    echo $i
done

# Nested loops with labeled break (not POSIX standard)
# Use functions or flags instead
found=false
for dir in /usr /opt /var; do
    for file in "$dir"/*; do
        if [ "$file" = "/usr/bin/vim" ]; then
            echo "Found vim at $file"
            found=true
            break
        fi
    done
    if [ "$found" = "true" ]; then
        break
    fi
done

Functions

Function Definition and Usage

# POSIX function definition
function_name() {
    echo "This is a function"
}

# Alternative syntax (POSIX)
function_name() {
    echo "This is a function"
}

# Function with parameters
greet() {
    echo "Hello, $1!"
}

# Function with multiple parameters
calculate_sum() {
    result=$(($1 + $2))
    echo $result
}

# Function with local variables (not POSIX, use carefully)
process_file() {
    filename="$1"
    line_count=$(wc -l < "$filename")
    echo "File $filename has $line_count lines"
}

# Function usage
greet "World"
sum=$(calculate_sum 5 3)
echo "Sum: $sum"

Function Best Practices

# Function with error handling
safe_copy() {
    if [ $# -ne 2 ]; then
        echo "Usage: safe_copy <source> <destination>" >&2
        return 1
    fi
    
    source_file="$1"
    dest_file="$2"
    
    if [ ! -f "$source_file" ]; then
        echo "Error: Source file '$source_file' does not exist" >&2
        return 1
    fi
    
    if [ -f "$dest_file" ]; then
        echo "Warning: Destination file '$dest_file' already exists" >&2
        printf "Overwrite? (y/N): "
        read answer
        case $answer in
            [Yy]|[Yy][Ee][Ss])
                ;;
            *)
                echo "Copy cancelled"
                return 1
                ;;
        esac
    fi
    
    if cp "$source_file" "$dest_file"; then
        echo "Successfully copied '$source_file' to '$dest_file'"
        return 0
    else
        echo "Error: Failed to copy file" >&2
        return 1
    fi
}

# Function with return values
is_number() {
    case $1 in
        ''|*[!0-9]*) return 1 ;;  # Not a number
        *) return 0 ;;            # Is a number
    esac
}

# Usage
if is_number "$input"; then
    echo "$input is a number"
else
    echo "$input is not a number"
fi

Function Libraries

# Create function library file: lib.sh
#!/bin/sh

# Logging functions
log_info() {
    echo "[INFO] $(date '+%Y-%m-%d %H:%M:%S'): $*"
}

log_error() {
    echo "[ERROR] $(date '+%Y-%m-%d %H:%M:%S'): $*" >&2
}

log_debug() {
    if [ "$DEBUG" = "1" ]; then
        echo "[DEBUG] $(date '+%Y-%m-%d %H:%M:%S'): $*" >&2
    fi
}

# File utilities
backup_file() {
    if [ -f "$1" ]; then
        cp "$1" "$1.backup.$(date +%Y%m%d_%H%M%S)"
        log_info "Backed up $1"
    fi
}

# Source library in main script
. ./lib.sh

# Use library functions
log_info "Script started"
backup_file "important.conf"
log_debug "Debug information"

Arithmetic Operations

POSIX Arithmetic Expansion

# Basic arithmetic
result=$((5 + 3))               # 8
result=$((10 - 4))              # 6
result=$((6 * 7))               # 42
result=$((20 / 4))              # 5
result=$((17 % 5))              # 2 (modulo)

# Arithmetic with variables
num1=10
num2=5
sum=$((num1 + num2))            # 15
product=$((num1 * num2))        # 50

# Increment and decrement
counter=0
counter=$((counter + 1))        # Increment
counter=$((counter - 1))        # Decrement

# Comparison in arithmetic context
if [ $((num1 > num2)) -eq 1 ]; then
    echo "num1 is greater than num2"
fi

# Complex expressions
result=$(( (num1 + num2) * 2 ))
result=$(( num1 ** 2 ))         # Exponentiation (if supported)

External Arithmetic Tools

# Using expr (portable but slower)
result=$(expr 5 + 3)
result=$(expr $num1 \* $num2)   # Note: * must be escaped
result=$(expr $num1 / $num2)

# Using bc for floating point
result=$(echo "scale=2; 10/3" | bc)
result=$(echo "scale=4; sqrt(16)" | bc -l)

# Using awk for calculations
result=$(awk "BEGIN {print 10/3}")
result=$(awk "BEGIN {printf \"%.2f\", 10/3}")

Input and Output

Reading Input

# Read single line
echo "Enter your name: "
read name
echo "Hello, $name"

# Read with prompt
read -p "Enter your age: " age

# Read multiple variables
echo "Enter first and last name: "
read first last
echo "Hello, $first $last"

# Read with timeout (if supported)
if read -t 10 -p "Enter input (10 seconds): " input; then
    echo "You entered: $input"
else
    echo "Timeout reached"
fi

# Read password (hidden input, if supported)
read -s -p "Enter password: " password
echo  # New line after hidden input

# Read from file
while read line; do
    echo "Line: $line"
done < file.txt

Output Formatting

# Basic output
echo "Simple message"
printf "Formatted message\n"

# Printf formatting
printf "Name: %s, Age: %d\n" "$name" "$age"
printf "Price: $%.2f\n" "$price"
printf "Hex: %x, Octal: %o\n" "$number" "$number"

# Escape sequences
echo "Line 1\nLine 2"           # May not work in all shells
printf "Line 1\nLine 2\n"       # Portable

# Output to stderr
echo "Error message" >&2
printf "Error: %s\n" "$error_msg" >&2

File Operations

# Create files
touch file.txt
> file.txt                      # Create empty file

# Write to files
echo "Content" > file.txt       # Overwrite
echo "More content" >> file.txt # Append

# Read files
content=$(cat file.txt)
while read line; do
    echo "Line: $line"
done < file.txt

# Copy files
cp source.txt destination.txt

# Move/rename files
mv old_name.txt new_name.txt

# Remove files
rm file.txt
rm -f file.txt                  # Force removal
rm -r directory/                # Recursive removal

Error Handling and Debugging

Exit Status and Error Checking

# Check command success
if command; then
    echo "Command succeeded"
else
    echo "Command failed with exit code $?"
fi

# Alternative syntax
command && echo "Success" || echo "Failed"

# Set exit status
exit 0                          # Success
exit 1                          # General error
exit 2                          # Misuse of shell builtins

# Function return values
my_function() {
    if [ condition ]; then
        return 0                # Success
    else
        return 1                # Failure
    fi
}

if my_function; then
    echo "Function succeeded"
fi

Error Handling Patterns

# Exit on error
set -e
command_that_might_fail
echo "This won't execute if command fails"

# Disable exit on error for specific command
set -e
if ! command_that_might_fail; then
    echo "Command failed, but script continues"
fi

# Error handling with cleanup
cleanup() {
    echo "Cleaning up..."
    rm -f temp_file
}

trap cleanup EXIT              # Run cleanup on script exit
trap cleanup INT TERM          # Run cleanup on interrupt/terminate

# Validate input
validate_file() {
    if [ ! -f "$1" ]; then
        echo "Error: File '$1' does not exist" >&2
        exit 1
    fi
}

validate_file "$input_file"

Debugging Techniques

# Debug mode
set -x                          # Print commands as executed
set +x                          # Disable debug mode

# Verbose mode
set -v                          # Print input lines as read
set +v                          # Disable verbose mode

# Check syntax without execution
sh -n script.sh

# Debug function
debug() {
    if [ "$DEBUG" = "1" ]; then
        echo "DEBUG: $*" >&2
    fi
}

# Usage
DEBUG=1 ./script.sh

# Trace function calls
trace() {
    echo "TRACE: Entering function $1" >&2
}

my_function() {
    trace "my_function"
    # Function body
}

Script Structure and Best Practices

Script Template

#!/bin/sh
# Script description
# Author: Your Name
# Date: YYYY-MM-DD
# Version: 1.0

# Exit on error
set -e

# Global variables
SCRIPT_NAME=$(basename "$0")
SCRIPT_DIR=$(dirname "$0")
VERSION="1.0"

# Functions
usage() {
    cat << EOF
Usage: $SCRIPT_NAME [OPTIONS] [ARGUMENTS]

Description of what the script does.

OPTIONS:
    -h, --help      Show this help message
    -v, --version   Show version information
    -d, --debug     Enable debug mode

ARGUMENTS:
    file            Input file to process

EXAMPLES:
    $SCRIPT_NAME input.txt
    $SCRIPT_NAME -d input.txt

EOF
}

version() {
    echo "$SCRIPT_NAME version $VERSION"
}

main() {
    # Parse command line arguments
    while [ $# -gt 0 ]; do
        case $1 in
            -h|--help)
                usage
                exit 0
                ;;
            -v|--version)
                version
                exit 0
                ;;
            -d|--debug)
                DEBUG=1
                set -x
                shift
                ;;
            -*)
                echo "Error: Unknown option $1" >&2
                usage >&2
                exit 1
                ;;
            *)
                break
                ;;
        esac
    done

    # Validate arguments
    if [ $# -eq 0 ]; then
        echo "Error: No input file specified" >&2
        usage >&2
        exit 1
    fi

    input_file="$1"
    
    # Validate input file
    if [ ! -f "$input_file" ]; then
        echo "Error: File '$input_file' does not exist" >&2
        exit 1
    fi

    # Main script logic
    echo "Processing file: $input_file"
    # Add your code here
    
    echo "Script completed successfully"
}

# Run main function with all arguments
main "$@"

Portability Considerations

# Use POSIX-compliant constructs
[ condition ] instead of [[ condition ]]
$(command) instead of `command`
$((arithmetic)) instead of $[arithmetic]

# Avoid bash-specific features
# No: array=(element1 element2)
# Yes: Use space-separated strings or multiple variables

# No: [[ string =~ regex ]]
# Yes: Use case or expr for pattern matching

# No: ${parameter,,} (lowercase)
# Yes: Use tr or awk for case conversion

# Portable shebang
#!/bin/sh                       # Most portable
#!/usr/bin/env sh              # Alternative

# Check for required commands
command -v required_command >/dev/null 2>&1 || {
    echo "Error: required_command is not installed" >&2
    exit 1
}

# Portable temporary files
temp_file="${TMPDIR:-/tmp}/script.$$"
trap 'rm -f "$temp_file"' EXIT

Security Best Practices

# Quote variables to prevent word splitting
rm "$filename"                  # Correct
rm $filename                    # Dangerous

# Validate input
case $input in
    [a-zA-Z0-9_-]*)
        # Valid input
        ;;
    *)
        echo "Invalid input" >&2
        exit 1
        ;;
esac

# Use full paths for commands in scripts
/bin/rm "$file"
/usr/bin/find "$dir" -name "*.tmp"

# Set secure umask
umask 077                       # Restrictive permissions

# Avoid eval with user input
# eval "$user_input"            # Dangerous
# Use case statements or functions instead

# Handle signals properly
cleanup() {
    rm -f "$temp_file"
    exit 1
}
trap cleanup INT TERM

The POSIX shell serves as the universal foundation for shell scripting across Unix-like systems, providing essential functionality while maintaining maximum portability. Its standardized syntax and features ensure that scripts written for POSIX sh will work consistently across different platforms, from minimal embedded systems to large enterprise servers. While it lacks the advanced features of modern shells, understanding POSIX sh is fundamental for writing robust, portable shell scripts that stand the test of time and work reliably in diverse computing environments.