Forseti Security Cheat Sheet
Overview
Forseti Security is an open-source security toolkit for Google Cloud Platform (GCP) that provides comprehensive security monitoring, policy enforcement, and compliance management. Originally developed by Google, Forseti Security helps organizations maintain security posture across their GCP environments through automated scanning, policy validation, and real-time monitoring. The platform offers inventory management, policy enforcement, and violation detection capabilities that are essential for maintaining secure and compliant cloud infrastructure.
💡 Key Features: Real-time inventory scanning, policy enforcement engine, compliance monitoring, security violation detection, automated remediation, multi-project support, custom rule engine, and integration with Cloud Security Command Center.
Installation and Setup
Prerequisites and Environment Setup
# System requirements check
echo "Checking system requirements for Forseti Security..."
# Check Python installation (Python 3.6+ required)
if command -v python3 &> /dev/null; then
python_version=$(python3 --version | awk '{print $2}')
echo "✅ Python found: $python_version"
else
echo "❌ Python not found. Installing Python 3..."
sudo apt update
sudo apt install -y python3 python3-pip python3-venv
fi
# Check gcloud CLI installation
if command -v gcloud &> /dev/null; then
gcloud_version=$(gcloud --version | head -n1)
echo "✅ Google Cloud CLI found: $gcloud_version"
else
echo "❌ Google Cloud CLI not found. Installing gcloud..."
curl https://sdk.cloud.google.com | bash
exec -l $SHELL
gcloud init
fi
# Check Terraform installation (for infrastructure setup)
if command -v terraform &> /dev/null; then
terraform_version=$(terraform --version | head -n1)
echo "✅ Terraform found: $terraform_version"
else
echo "❌ Terraform not found. Installing Terraform..."
wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install terraform
fi
# Check Docker installation (for containerized deployment)
if command -v docker &> /dev/null; then
docker_version=$(docker --version)
echo "✅ Docker found: $docker_version"
else
echo "❌ Docker not found. Installing Docker..."
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER
fi
# Verify installations
echo "Verifying installations..."
python3 --version
gcloud --version
terraform --version
docker --version
echo "Prerequisites check completed"
GCP Project Setup and Authentication
# Set up GCP project for Forseti Security
export PROJECT_ID="your-forseti-project"
export ORGANIZATION_ID="your-organization-id"
export BILLING_ACCOUNT="your-billing-account"
# Authenticate with Google Cloud
gcloud auth login
gcloud auth application-default login
# Create or select project
gcloud projects create $PROJECT_ID --organization=$ORGANIZATION_ID
gcloud config set project $PROJECT_ID
# Link billing account
gcloud billing projects link $PROJECT_ID --billing-account=$BILLING_ACCOUNT
# Enable required APIs
echo "Enabling required GCP APIs..."
gcloud services enable cloudasset.googleapis.com
gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable compute.googleapis.com
gcloud services enable container.googleapis.com
gcloud services enable iam.googleapis.com
gcloud services enable logging.googleapis.com
gcloud services enable monitoring.googleapis.com
gcloud services enable securitycenter.googleapis.com
gcloud services enable sql-component.googleapis.com
gcloud services enable storage-api.googleapis.com
gcloud services enable storage-component.googleapis.com
# Wait for APIs to be enabled
echo "Waiting for APIs to be enabled..."
sleep 30
# Verify API enablement
gcloud services list --enabled --filter="name:cloudasset.googleapis.com OR name:cloudresourcemanager.googleapis.com"
# Create Forseti service account
gcloud iam service-accounts create forseti-security \
--display-name="Forseti Security Service Account" \
--description="Service account for Forseti Security operations"
# Get service account email
export FORSETI_SA_EMAIL="forseti-security@${PROJECT_ID}.iam.gserviceaccount.com"
# Grant necessary permissions to Forseti service account
echo "Granting permissions to Forseti service account..."
# Organization-level permissions
gcloud organizations add-iam-policy-binding $ORGANIZATION_ID \
--member="serviceAccount:$FORSETI_SA_EMAIL" \
--role="roles/browser"
gcloud organizations add-iam-policy-binding $ORGANIZATION_ID \
--member="serviceAccount:$FORSETI_SA_EMAIL" \
--role="roles/cloudasset.viewer"
gcloud organizations add-iam-policy-binding $ORGANIZATION_ID \
--member="serviceAccount:$FORSETI_SA_EMAIL" \
--role="roles/iam.securityReviewer"
gcloud organizations add-iam-policy-binding $ORGANIZATION_ID \
--member="serviceAccount:$FORSETI_SA_EMAIL" \
--role="roles/securitycenter.findingsEditor"
# Project-level permissions
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:$FORSETI_SA_EMAIL" \
--role="roles/cloudsql.client"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:$FORSETI_SA_EMAIL" \
--role="roles/compute.instanceAdmin"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:$FORSETI_SA_EMAIL" \
--role="roles/storage.objectAdmin"
# Create and download service account key
gcloud iam service-accounts keys create forseti-security-key.json \
--iam-account=$FORSETI_SA_EMAIL
echo "GCP project setup completed"
echo "Service account email: $FORSETI_SA_EMAIL"
echo "Service account key saved to: forseti-security-key.json"
Terraform Infrastructure Deployment
# Create Terraform configuration for Forseti Security infrastructure
mkdir -p forseti-terraform
cd forseti-terraform
# Create main Terraform configuration
cat > main.tf << 'EOF'
terraform {
required_version = ">= 0.14"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 4.0"
}
google-beta = {
source = "hashicorp/google-beta"
version = "~> 4.0"
}
}
}
provider "google" {
project = var.project_id
region = var.region
zone = var.zone
}
provider "google-beta" {
project = var.project_id
region = var.region
zone = var.zone
}
# Variables
variable "project_id" {
description = "GCP Project ID"
type = string
}
variable "organization_id" {
description = "GCP Organization ID"
type = string
}
variable "region" {
description = "GCP Region"
type = string
default = "us-central1"
}
variable "zone" {
description = "GCP Zone"
type = string
default = "us-central1-a"
}
variable "forseti_version" {
description = "Forseti Security version"
type = string
default = "v2.25.1"
}
# VPC Network for Forseti
resource "google_compute_network" "forseti_network" {
name = "forseti-security-network"
auto_create_subnetworks = false
description = "VPC network for Forseti Security"
}
resource "google_compute_subnetwork" "forseti_subnetwork" {
name = "forseti-security-subnetwork"
ip_cidr_range = "10.0.0.0/24"
region = var.region
network = google_compute_network.forseti_network.id
description = "Subnetwork for Forseti Security"
}
# Firewall rules
resource "google_compute_firewall" "forseti_server_allow_grpc" {
name = "forseti-server-allow-grpc"
network = google_compute_network.forseti_network.name
allow {
protocol = "tcp"
ports = ["50051"]
}
source_ranges = ["10.0.0.0/24"]
target_tags = ["forseti-server"]
description = "Allow gRPC communication to Forseti server"
}
resource "google_compute_firewall" "forseti_allow_ssh" {
name = "forseti-allow-ssh"
network = google_compute_network.forseti_network.name
allow {
protocol = "tcp"
ports = ["22"]
}
source_ranges = ["0.0.0.0/0"]
target_tags = ["forseti-security"]
description = "Allow SSH access to Forseti instances"
}
# Cloud SQL instance for Forseti database
resource "google_sql_database_instance" "forseti_database" {
name = "forseti-security-db"
database_version = "MYSQL_8_0"
region = var.region
deletion_protection = false
settings {
tier = "db-n1-standard-1"
activation_policy = "ALWAYS"
availability_type = "ZONAL"
backup_configuration {
enabled = true
start_time = "03:00"
location = var.region
binary_log_enabled = true
transaction_log_retention_days = 7
}
database_flags {
name = "slow_query_log"
value = "on"
}
ip_configuration {
ipv4_enabled = true
private_network = google_compute_network.forseti_network.id
require_ssl = true
authorized_networks {
name = "forseti-subnetwork"
value = "10.0.0.0/24"
}
}
maintenance_window {
day = 7
hour = 3
update_track = "stable"
}
}
depends_on = [google_service_networking_connection.private_vpc_connection]
}
# Private service connection for Cloud SQL
resource "google_compute_global_address" "private_ip_address" {
name = "forseti-private-ip-address"
purpose = "VPC_PEERING"
address_type = "INTERNAL"
prefix_length = 16
network = google_compute_network.forseti_network.id
}
resource "google_service_networking_connection" "private_vpc_connection" {
network = google_compute_network.forseti_network.id
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.private_ip_address.name]
}
# Forseti database
resource "google_sql_database" "forseti_database" {
name = "forseti_security"
instance = google_sql_database_instance.forseti_database.name
}
# Forseti database user
resource "google_sql_user" "forseti_user" {
name = "forseti_security_user"
instance = google_sql_database_instance.forseti_database.name
password = random_password.forseti_db_password.result
}
resource "random_password" "forseti_db_password" {
length = 16
special = true
}
# Cloud Storage bucket for Forseti
resource "google_storage_bucket" "forseti_server_bucket" {
name = "${var.project_id}-forseti-security-bucket"
location = var.region
force_destroy = true
uniform_bucket_level_access = true
versioning {
enabled = true
}
lifecycle_rule {
condition {
age = 30
}
action {
type = "Delete"
}
}
}
# Forseti Server VM instance
resource "google_compute_instance" "forseti_server" {
name = "forseti-security-server"
machine_type = "n1-standard-2"
zone = var.zone
tags = ["forseti-server", "forseti-security"]
boot_disk {
initialize_params {
image = "ubuntu-os-cloud/ubuntu-2004-lts"
size = 50
type = "pd-standard"
}
}
network_interface {
network = google_compute_network.forseti_network.name
subnetwork = google_compute_subnetwork.forseti_subnetwork.name
access_config {
// Ephemeral public IP
}
}
service_account {
email = google_service_account.forseti_server_sa.email
scopes = ["cloud-platform"]
}
metadata = {
enable-oslogin = "TRUE"
}
metadata_startup_script = templatefile("${path.module}/startup-script.sh", {
project_id = var.project_id
forseti_version = var.forseti_version
db_host = google_sql_database_instance.forseti_database.private_ip_address
db_name = google_sql_database.forseti_database.name
db_user = google_sql_user.forseti_user.name
db_password = google_sql_user.forseti_user.password
bucket_name = google_storage_bucket.forseti_server_bucket.name
organization_id = var.organization_id
})
depends_on = [
google_sql_database_instance.forseti_database,
google_storage_bucket.forseti_server_bucket
]
}
# Forseti Client VM instance
resource "google_compute_instance" "forseti_client" {
name = "forseti-security-client"
machine_type = "n1-standard-1"
zone = var.zone
tags = ["forseti-client", "forseti-security"]
boot_disk {
initialize_params {
image = "ubuntu-os-cloud/ubuntu-2004-lts"
size = 30
type = "pd-standard"
}
}
network_interface {
network = google_compute_network.forseti_network.name
subnetwork = google_compute_subnetwork.forseti_subnetwork.name
access_config {
// Ephemeral public IP
}
}
service_account {
email = google_service_account.forseti_client_sa.email
scopes = ["cloud-platform"]
}
metadata = {
enable-oslogin = "TRUE"
}
metadata_startup_script = templatefile("${path.module}/client-startup-script.sh", {
project_id = var.project_id
forseti_version = var.forseti_version
server_ip = google_compute_instance.forseti_server.network_interface[0].network_ip
})
}
# Service accounts
resource "google_service_account" "forseti_server_sa" {
account_id = "forseti-server-gcp-sa"
display_name = "Forseti Server Service Account"
description = "Service account for Forseti Security server"
}
resource "google_service_account" "forseti_client_sa" {
account_id = "forseti-client-gcp-sa"
display_name = "Forseti Client Service Account"
description = "Service account for Forseti Security client"
}
# IAM bindings for Forseti server service account
resource "google_organization_iam_member" "forseti_server_browser" {
org_id = var.organization_id
role = "roles/browser"
member = "serviceAccount:${google_service_account.forseti_server_sa.email}"
}
resource "google_organization_iam_member" "forseti_server_cloudasset_viewer" {
org_id = var.organization_id
role = "roles/cloudasset.viewer"
member = "serviceAccount:${google_service_account.forseti_server_sa.email}"
}
resource "google_organization_iam_member" "forseti_server_security_reviewer" {
org_id = var.organization_id
role = "roles/iam.securityReviewer"
member = "serviceAccount:${google_service_account.forseti_server_sa.email}"
}
resource "google_organization_iam_member" "forseti_server_security_center" {
org_id = var.organization_id
role = "roles/securitycenter.findingsEditor"
member = "serviceAccount:${google_service_account.forseti_server_sa.email}"
}
# Project-level IAM bindings
resource "google_project_iam_member" "forseti_server_cloudsql_client" {
project = var.project_id
role = "roles/cloudsql.client"
member = "serviceAccount:${google_service_account.forseti_server_sa.email}"
}
resource "google_project_iam_member" "forseti_server_storage_admin" {
project = var.project_id
role = "roles/storage.objectAdmin"
member = "serviceAccount:${google_service_account.forseti_server_sa.email}"
}
# Outputs
output "forseti_server_ip" {
description = "Internal IP address of Forseti server"
value = google_compute_instance.forseti_server.network_interface[0].network_ip
}
output "forseti_server_external_ip" {
description = "External IP address of Forseti server"
value = google_compute_instance.forseti_server.network_interface[0].access_config[0].nat_ip
}
output "forseti_client_ip" {
description = "Internal IP address of Forseti client"
value = google_compute_instance.forseti_client.network_interface[0].network_ip
}
output "forseti_database_ip" {
description = "Private IP address of Forseti database"
value = google_sql_database_instance.forseti_database.private_ip_address
}
output "forseti_bucket_name" {
description = "Name of Forseti storage bucket"
value = google_storage_bucket.forseti_server_bucket.name
}
output "forseti_server_sa_email" {
description = "Email of Forseti server service account"
value = google_service_account.forseti_server_sa.email
}
EOF
# Create startup script for Forseti server
cat > startup-script.sh << 'EOF'
#!/bin/bash
# Forseti Server Installation Script
set -e
# Variables from Terraform
PROJECT_ID="${project_id}"
FORSETI_VERSION="${forseti_version}"
DB_HOST="${db_host}"
DB_NAME="${db_name}"
DB_USER="${db_user}"
DB_PASSWORD="${db_password}"
BUCKET_NAME="${bucket_name}"
ORGANIZATION_ID="${organization_id}"
# Update system
apt-get update
apt-get install -y python3 python3-pip python3-venv git mysql-client
# Create forseti user
useradd -m -s /bin/bash forseti
usermod -aG sudo forseti
# Install Forseti Security
cd /home/forseti
git clone https://github.com/forseti-security/forseti-security.git
cd forseti-security
git checkout $FORSETI_VERSION
# Create virtual environment
python3 -m venv forseti-env
source forseti-env/bin/activate
# Install dependencies
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
# Create Forseti configuration directory
mkdir -p /home/forseti/forseti-security/configs
# Create Forseti server configuration
cat > /home/forseti/forseti-security/configs/forseti_conf_server.yaml << EOFCONFIG
global:
# Database configuration
db_host: $DB_HOST
db_user: $DB_USER
db_name: $DB_NAME
db_password: $DB_PASSWORD
# GCS bucket for Forseti data
bucket_name: $BUCKET_NAME
# Email notification settings
email_recipient: admin@company.com
email_sender: forseti-noreply@$PROJECT_ID.iam.gserviceaccount.com
sendgrid_api_key: ""
# Organization and project settings
organization_id: $ORGANIZATION_ID
inventory:
# Inventory configuration
gcs_summary_path: gs://$BUCKET_NAME/inventory_summary.json
# Root resource to start inventory from
root_resource_id: $ORGANIZATION_ID
# Inventory modules to enable
api_quota_enabled: true
cai_enabled: true
scanner:
# Scanner configuration
enabled_scanners:
- audit_logging
- bigquery
- blacklist
- bucket_acl
- cloudsql_acl
- enabled_apis
- firewall_rule
- forwarding_rule
- group
- iam_policy
- iap
- instance_network_interface
- ke_cluster
- ke_node_pool
- lien
- location
- log_sink
- resource
- service_account_key
# Scanner output path
output_path: gs://$BUCKET_NAME/scanner_violations
# Violation configs
rules_path: /home/forseti/forseti-security/rules
notifier:
# Notification configuration
enabled: true
# Notification channels
email_enabled: true
gcs_enabled: true
cscc_enabled: true
# CSCC (Cloud Security Command Center) settings
cscc_source_id: ""
# Violation summary
violation_summary_enabled: true
explain:
# Explain configuration
enabled: true
model:
# Model configuration
enabled: true
# Model location
model_location: gs://$BUCKET_NAME/model
EOFCONFIG
# Set up database
mysql -h $DB_HOST -u $DB_USER -p$DB_PASSWORD $DB_NAME < install/gcp/scripts/create_forseti_tables.sql
# Create systemd service for Forseti server
cat > /etc/systemd/system/forseti-server.service << EOFSERVICE
[Unit]
Description=Forseti Security Server
After=network.target
[Service]
Type=simple
User=forseti
Group=forseti
WorkingDirectory=/home/forseti/forseti-security
Environment=PATH=/home/forseti/forseti-security/forseti-env/bin
ExecStart=/home/forseti/forseti-security/forseti-env/bin/python -m google.cloud.forseti.services.server --config_file_path=/home/forseti/forseti-security/configs/forseti_conf_server.yaml
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
EOFSERVICE
# Set permissions
chown -R forseti:forseti /home/forseti
chmod +x /home/forseti/forseti-security/forseti-env/bin/*
# Enable and start Forseti server
systemctl daemon-reload
systemctl enable forseti-server
systemctl start forseti-server
# Create log rotation
cat > /etc/logrotate.d/forseti << EOFLOGROTATE
/var/log/forseti/*.log {
daily
missingok
rotate 30
compress
delaycompress
notifempty
create 644 forseti forseti
postrotate
systemctl reload forseti-server
endscript
}
EOFLOGROTATE
echo "Forseti Security server installation completed"
EOF
# Create startup script for Forseti client
cat > client-startup-script.sh << 'EOF'
#!/bin/bash
# Forseti Client Installation Script
set -e
# Variables from Terraform
PROJECT_ID="${project_id}"
FORSETI_VERSION="${forseti_version}"
SERVER_IP="${server_ip}"
# Update system
apt-get update
apt-get install -y python3 python3-pip python3-venv git
# Create forseti user
useradd -m -s /bin/bash forseti
usermod -aG sudo forseti
# Install Forseti Security client
cd /home/forseti
git clone https://github.com/forseti-security/forseti-security.git
cd forseti-security
git checkout $FORSETI_VERSION
# Create virtual environment
python3 -m venv forseti-env
source forseti-env/bin/activate
# Install dependencies
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
# Create Forseti client configuration
mkdir -p /home/forseti/forseti-security/configs
cat > /home/forseti/forseti-security/configs/forseti_conf_client.yaml << EOFCONFIG
server_ip: $SERVER_IP
server_port: 50051
EOFCONFIG
# Set permissions
chown -R forseti:forseti /home/forseti
echo "Forseti Security client installation completed"
EOF
# Create terraform.tfvars file
cat > terraform.tfvars << EOF
project_id = "$PROJECT_ID"
organization_id = "$ORGANIZATION_ID"
region = "us-central1"
zone = "us-central1-a"
forseti_version = "v2.25.1"
EOF
# Initialize and apply Terraform
terraform init
terraform plan
terraform apply -auto-approve
echo "Forseti Security infrastructure deployment completed"
Docker Deployment
# Create Docker Compose setup for Forseti Security
mkdir -p forseti-docker
cd forseti-docker
# Create Docker Compose file
cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
# MySQL Database for Forseti
forseti-db:
image: mysql:8.0
container_name: forseti-mysql
environment:
MYSQL_ROOT_PASSWORD: rootpassword
MYSQL_DATABASE: forseti_security
MYSQL_USER: forseti_security_user
MYSQL_PASSWORD: forseti123
ports:
- "3306:3306"
volumes:
- forseti_mysql_data:/var/lib/mysql
- ./init-scripts:/docker-entrypoint-initdb.d
networks:
- forseti-network
command: --default-authentication-plugin=mysql_native_password
# Forseti Security Server
forseti-server:
build:
context: .
dockerfile: Dockerfile.server
container_name: forseti-server
depends_on:
- forseti-db
environment:
- FORSETI_DB_HOST=forseti-db
- FORSETI_DB_PORT=3306
- FORSETI_DB_NAME=forseti_security
- FORSETI_DB_USER=forseti_security_user
- FORSETI_DB_PASSWORD=forseti123
- GOOGLE_APPLICATION_CREDENTIALS=/app/credentials/service-account.json
- FORSETI_ORGANIZATION_ID=${ORGANIZATION_ID}
- FORSETI_PROJECT_ID=${PROJECT_ID}
ports:
- "50051:50051"
volumes:
- ./config:/app/config
- ./credentials:/app/credentials
- ./rules:/app/rules
- ./logs:/app/logs
networks:
- forseti-network
# Forseti Security Client
forseti-client:
build:
context: .
dockerfile: Dockerfile.client
container_name: forseti-client
depends_on:
- forseti-server
environment:
- FORSETI_SERVER_HOST=forseti-server
- FORSETI_SERVER_PORT=50051
- GOOGLE_APPLICATION_CREDENTIALS=/app/credentials/service-account.json
volumes:
- ./config:/app/config
- ./credentials:/app/credentials
- ./logs:/app/logs
networks:
- forseti-network
stdin_open: true
tty: true
volumes:
forseti_mysql_data:
networks:
forseti-network:
driver: bridge
EOF
# Create Dockerfile for Forseti server
cat > Dockerfile.server << 'EOF'
FROM python:3.8-slim
# Install system dependencies
RUN apt-get update && apt-get install -y \
git \
mysql-client \
&& rm -rf /var/lib/apt/lists/*
# Create app directory
WORKDIR /app
# Clone Forseti Security
RUN git clone https://github.com/forseti-security/forseti-security.git .
RUN git checkout v2.25.1
# Install Python dependencies
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
RUN pip install -e .
# Create necessary directories
RUN mkdir -p /app/config /app/credentials /app/rules /app/logs
# Copy configuration files
COPY config/ /app/config/
COPY rules/ /app/rules/
# Set permissions
RUN chmod +x /app/google/cloud/forseti/services/server.py
# Expose gRPC port
EXPOSE 50051
# Start Forseti server
CMD ["python", "-m", "google.cloud.forseti.services.server", "--config_file_path=/app/config/forseti_conf_server.yaml"]
EOF
# Create Dockerfile for Forseti client
cat > Dockerfile.client << 'EOF'
FROM python:3.8-slim
# Install system dependencies
RUN apt-get update && apt-get install -y \
git \
&& rm -rf /var/lib/apt/lists/*
# Create app directory
WORKDIR /app
# Clone Forseti Security
RUN git clone https://github.com/forseti-security/forseti-security.git .
RUN git checkout v2.25.1
# Install Python dependencies
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
RUN pip install -e .
# Create necessary directories
RUN mkdir -p /app/config /app/credentials /app/logs
# Copy configuration files
COPY config/ /app/config/
# Start bash shell for interactive use
CMD ["/bin/bash"]
EOF
# Create configuration directory and files
mkdir -p config credentials rules logs init-scripts
# Create Forseti server configuration
cat > config/forseti_conf_server.yaml << 'EOF'
global:
# Database configuration
db_host: forseti-db
db_user: forseti_security_user
db_name: forseti_security
db_password: forseti123
# Email notification settings
email_recipient: admin@company.com
email_sender: forseti-noreply@company.com
sendgrid_api_key: ""
# Organization and project settings
organization_id: "123456789012"
inventory:
# Inventory configuration
root_resource_id: "123456789012"
# Inventory modules to enable
api_quota_enabled: true
cai_enabled: true
scanner:
# Scanner configuration
enabled_scanners:
- audit_logging
- bigquery
- blacklist
- bucket_acl
- cloudsql_acl
- enabled_apis
- firewall_rule
- forwarding_rule
- group
- iam_policy
- iap
- instance_network_interface
- ke_cluster
- ke_node_pool
- lien
- location
- log_sink
- resource
- service_account_key
# Violation configs
rules_path: /app/rules
notifier:
# Notification configuration
enabled: true
# Notification channels
email_enabled: true
gcs_enabled: false
cscc_enabled: false
# Violation summary
violation_summary_enabled: true
explain:
# Explain configuration
enabled: true
model:
# Model configuration
enabled: true
EOF
# Create Forseti client configuration
cat > config/forseti_conf_client.yaml << 'EOF'
server_ip: forseti-server
server_port: 50051
EOF
# Create MySQL initialization script
cat > init-scripts/01-init-forseti.sql << 'EOF'
-- Forseti Security Database Initialization
USE forseti_security;
-- Create tables for Forseti Security
CREATE TABLE IF NOT EXISTS violations (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
resource_id VARCHAR(255),
resource_type VARCHAR(255),
rule_name VARCHAR(255),
rule_index INT,
violation_type VARCHAR(255),
violation_data TEXT,
resource_data TEXT,
created_at_datetime TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS inventory (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
inventory_index_id BIGINT,
inventory_type VARCHAR(255),
resource_data TEXT,
created_at_datetime TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS inventory_index (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
inventory_index_id BIGINT UNIQUE,
completed_at_datetime TIMESTAMP,
created_at_datetime TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
count_objects INT DEFAULT 0,
schema_version VARCHAR(255)
);
-- Insert sample data
INSERT INTO inventory_index (inventory_index_id, completed_at_datetime, count_objects, schema_version) VALUES
(1, NOW(), 0, '2.25.1');
COMMIT;
EOF
# Create sample rules directory
mkdir -p rules
# Create sample security rules
cat > rules/firewall_rules.yaml << 'EOF'
rules:
- name: 'Firewall rule allows all traffic'
mode: blacklist
resource:
- type: firewall_rule
filters:
- key: direction
op: eq
value: INGRESS
- key: sourceRanges
op: contains
value: '0.0.0.0/0'
- key: allowed.ports
op: contains
value: '*'
actions:
- type: add_to_whitelist
value: 'firewall-rule-*'
EOF
cat > rules/iam_rules.yaml << 'EOF'
rules:
- name: 'Service account has too many roles'
mode: blacklist
resource:
- type: iam_policy
filters:
- key: bindings.members
op: regex
value: 'serviceAccount:.*'
- key: bindings.role
op: regex
value: 'roles/.*'
actions:
- type: add_to_whitelist
value: 'serviceAccount:*@*.iam.gserviceaccount.com'
EOF
# Create environment file
cat > .env << EOF
PROJECT_ID=your-project-id
ORGANIZATION_ID=your-organization-id
EOF
# Start Forseti with Docker Compose
echo "Starting Forseti Security with Docker Compose..."
docker-compose up -d
# Wait for services to start
echo "Waiting for services to start..."
sleep 60
# Check service status
docker-compose ps
# Test connectivity
echo "Testing service connectivity..."
docker-compose exec forseti-db mysql -u forseti_security_user -pforseti123 -e "SELECT 1"
docker-compose logs forseti-server | tail -20
echo "Forseti Security Docker deployment completed"
echo "Access Forseti server at: localhost:50051"
echo "Connect to client: docker-compose exec forseti-client /bin/bash"
Configuration and Rule Management
Basic Configuration
# Create Forseti configuration management script
cat > configure_forseti.sh << 'EOF'
#!/bin/bash
# Forseti Security Configuration Management
FORSETI_HOME=${FORSETI_HOME:-/home/forseti/forseti-security}
CONFIG_DIR="$FORSETI_HOME/configs"
RULES_DIR="$FORSETI_HOME/rules"
# Create directories
mkdir -p "$CONFIG_DIR" "$RULES_DIR"
# Function to configure inventory settings
configure_inventory() {
echo "Configuring Forseti inventory settings..."
cat > "$CONFIG_DIR/inventory_config.yaml" << 'INVCONFIG'
# Forseti Inventory Configuration
inventory:
# Root resource configuration
root_resource_id: "organizations/123456789012"
# Inventory modules
api_quota_enabled: true
cai_enabled: true
# Resource types to inventory
resource_types:
- organization
- folder
- project
- compute_instance
- compute_disk
- compute_network
- compute_subnetwork
- compute_firewall
- storage_bucket
- cloudsql_instance
- iam_policy
- service_account
- service_account_key
- bigquery_dataset
- bigquery_table
- kubernetes_cluster
- kubernetes_node_pool
# Inventory frequency (in seconds)
inventory_frequency: 3600
# Parallel processing
max_workers: 10
# Timeout settings
inventory_timeout: 1800
# Output settings
output_format: json
gcs_summary_path: gs://forseti-bucket/inventory_summary.json
INVCONFIG
echo "Inventory configuration completed"
}
# Function to configure scanner settings
configure_scanner() {
echo "Configuring Forseti scanner settings..."
cat > "$CONFIG_DIR/scanner_config.yaml" << 'SCANCONFIG'
# Forseti Scanner Configuration
scanner:
# Enabled scanners
enabled_scanners:
- audit_logging
- bigquery
- blacklist
- bucket_acl
- cloudsql_acl
- enabled_apis
- firewall_rule
- forwarding_rule
- group
- iam_policy
- iap
- instance_network_interface
- ke_cluster
- ke_node_pool
- lien
- location
- log_sink
- resource
- service_account_key
# Scanner frequency (in seconds)
scanner_frequency: 7200
# Output settings
output_path: gs://forseti-bucket/scanner_violations
# Violation retention (in days)
violation_retention_days: 30
# Scanner timeout (in seconds)
scanner_timeout: 3600
# Parallel processing
max_scanner_processes: 5
# Rules configuration
rules_path: /home/forseti/forseti-security/rules
# Whitelist configuration
whitelist_enabled: true
whitelist_path: /home/forseti/forseti-security/rules/whitelist.yaml
SCANCONFIG
echo "Scanner configuration completed"
}
# Function to configure notifier settings
configure_notifier() {
echo "Configuring Forseti notifier settings..."
cat > "$CONFIG_DIR/notifier_config.yaml" << 'NOTIFYCONFIG'
# Forseti Notifier Configuration
notifier:
# Notification enabled
enabled: true
# Email notifications
email_enabled: true
email_recipient: security-team@company.com
email_sender: forseti-noreply@company.com
# SendGrid configuration
sendgrid_api_key: ""
# GCS notifications
gcs_enabled: true
gcs_path: gs://forseti-bucket/notifications
# Cloud Security Command Center
cscc_enabled: true
cscc_source_id: "organizations/123456789012/sources/1234567890123456789"
# Slack notifications
slack_enabled: false
slack_webhook_url: ""
# Notification frequency
notification_frequency: 86400
# Violation summary
violation_summary_enabled: true
# Notification filters
notification_filters:
- severity: HIGH
enabled: true
- severity: MEDIUM
enabled: true
- severity: LOW
enabled: false
# Template configuration
email_template_path: /home/forseti/forseti-security/templates/email_template.html
NOTIFYCONFIG
echo "Notifier configuration completed"
}
# Function to configure explain settings
configure_explain() {
echo "Configuring Forseti explain settings..."
cat > "$CONFIG_DIR/explain_config.yaml" << 'EXPLAINCONFIG'
# Forseti Explain Configuration
explain:
# Explain enabled
enabled: true
# Explain timeout (in seconds)
explain_timeout: 300
# Maximum number of policies to explain
max_policies: 1000
# Output format
output_format: json
# Cache settings
cache_enabled: true
cache_ttl: 3600
# Explain frequency (in seconds)
explain_frequency: 21600
EXPLAINCONFIG
echo "Explain configuration completed"
}
# Function to configure model settings
configure_model() {
echo "Configuring Forseti model settings..."
cat > "$CONFIG_DIR/model_config.yaml" << 'MODELCONFIG'
# Forseti Model Configuration
model:
# Model enabled
enabled: true
# Model location
model_location: gs://forseti-bucket/model
# Model timeout (in seconds)
model_timeout: 1800
# Model frequency (in seconds)
model_frequency: 3600
# Model retention (in days)
model_retention_days: 7
# Model verification
verify_policy_awesomeness: true
MODELCONFIG
echo "Model configuration completed"
}
# Main configuration function
main() {
echo "Starting Forseti Security configuration..."
configure_inventory
configure_scanner
configure_notifier
configure_explain
configure_model
# Set permissions
chmod -R 755 "$CONFIG_DIR"
echo "Forseti Security configuration completed successfully"
echo "Configuration files created in: $CONFIG_DIR"
}
# Run configuration
main "$@"
EOF
chmod +x configure_forseti.sh
./configure_forseti.sh
Custom Rule Creation
#!/usr/bin/env python3
# Forseti Security Custom Rule Creation
import yaml
import json
import os
from datetime import datetime
class ForsetiRuleManager:
"""Manage Forseti Security custom rules"""
def __init__(self, rules_directory='/home/forseti/forseti-security/rules'):
self.rules_directory = rules_directory
self.ensure_rules_directory()
def ensure_rules_directory(self):
"""Ensure rules directory exists"""
os.makedirs(self.rules_directory, exist_ok=True)
def create_firewall_rules(self):
"""Create firewall security rules"""
firewall_rules = {
'rules': [
{
'name': 'Firewall allows all traffic from internet',
'mode': 'blacklist',
'resource': [{'type': 'firewall_rule'}],
'filters': [
{'key': 'direction', 'op': 'eq', 'value': 'INGRESS'},
{'key': 'sourceRanges', 'op': 'contains', 'value': '0.0.0.0/0'},
{'key': 'allowed.ports', 'op': 'contains', 'value': '*'}
],
'actions': [
{'type': 'add_to_whitelist', 'value': 'firewall-rule-web-*'}
]
},
{
'name': 'Firewall allows SSH from internet',
'mode': 'blacklist',
'resource': [{'type': 'firewall_rule'}],
'filters': [
{'key': 'direction', 'op': 'eq', 'value': 'INGRESS'},
{'key': 'sourceRanges', 'op': 'contains', 'value': '0.0.0.0/0'},
{'key': 'allowed.ports', 'op': 'contains', 'value': '22'}
]
},
{
'name': 'Firewall allows RDP from internet',
'mode': 'blacklist',
'resource': [{'type': 'firewall_rule'}],
'filters': [
{'key': 'direction', 'op': 'eq', 'value': 'INGRESS'},
{'key': 'sourceRanges', 'op': 'contains', 'value': '0.0.0.0/0'},
{'key': 'allowed.ports', 'op': 'contains', 'value': '3389'}
]
}
]
}
with open(f"{self.rules_directory}/firewall_rules.yaml", 'w') as f:
yaml.dump(firewall_rules, f, default_flow_style=False)
print("Firewall rules created successfully")
def create_iam_rules(self):
"""Create IAM security rules"""
iam_rules = {
'rules': [
{
'name': 'Service account has owner role',
'mode': 'blacklist',
'resource': [{'type': 'iam_policy'}],
'filters': [
{'key': 'bindings.members', 'op': 'regex', 'value': 'serviceAccount:.*'},
{'key': 'bindings.role', 'op': 'eq', 'value': 'roles/owner'}
]
},
{
'name': 'User has primitive roles',
'mode': 'blacklist',
'resource': [{'type': 'iam_policy'}],
'filters': [
{'key': 'bindings.members', 'op': 'regex', 'value': 'user:.*'},
{'key': 'bindings.role', 'op': 'regex', 'value': 'roles/(owner|editor|viewer)'}
],
'actions': [
{'type': 'add_to_whitelist', 'value': 'user:admin@company.com'}
]
},
{
'name': 'Service account key is old',
'mode': 'blacklist',
'resource': [{'type': 'service_account_key'}],
'filters': [
{'key': 'validAfterTime', 'op': 'older_than', 'value': '90d'}
]
}
]
}
with open(f"{self.rules_directory}/iam_rules.yaml", 'w') as f:
yaml.dump(iam_rules, f, default_flow_style=False)
print("IAM rules created successfully")
def create_storage_rules(self):
"""Create Cloud Storage security rules"""
storage_rules = {
'rules': [
{
'name': 'Storage bucket is publicly readable',
'mode': 'blacklist',
'resource': [{'type': 'bucket_acl'}],
'filters': [
{'key': 'entity', 'op': 'eq', 'value': 'allUsers'},
{'key': 'role', 'op': 'eq', 'value': 'READER'}
],
'actions': [
{'type': 'add_to_whitelist', 'value': 'public-website-*'}
]
},
{
'name': 'Storage bucket is publicly writable',
'mode': 'blacklist',
'resource': [{'type': 'bucket_acl'}],
'filters': [
{'key': 'entity', 'op': 'eq', 'value': 'allUsers'},
{'key': 'role', 'op': 'eq', 'value': 'WRITER'}
]
},
{
'name': 'Storage bucket allows authenticated users',
'mode': 'blacklist',
'resource': [{'type': 'bucket_acl'}],
'filters': [
{'key': 'entity', 'op': 'eq', 'value': 'allAuthenticatedUsers'}
]
}
]
}
with open(f"{self.rules_directory}/storage_rules.yaml", 'w') as f:
yaml.dump(storage_rules, f, default_flow_style=False)
print("Storage rules created successfully")
def create_compute_rules(self):
"""Create Compute Engine security rules"""
compute_rules = {
'rules': [
{
'name': 'Instance has public IP',
'mode': 'blacklist',
'resource': [{'type': 'instance'}],
'filters': [
{'key': 'networkInterfaces.accessConfigs.type', 'op': 'eq', 'value': 'ONE_TO_ONE_NAT'}
],
'actions': [
{'type': 'add_to_whitelist', 'value': 'bastion-*'},
{'type': 'add_to_whitelist', 'value': 'web-*'}
]
},
{
'name': 'Instance allows HTTP traffic',
'mode': 'blacklist',
'resource': [{'type': 'instance'}],
'filters': [
{'key': 'tags.items', 'op': 'contains', 'value': 'http-server'}
]
},
{
'name': 'Instance has default service account',
'mode': 'blacklist',
'resource': [{'type': 'instance'}],
'filters': [
{'key': 'serviceAccounts.email', 'op': 'regex', 'value': '.*-compute@developer.gserviceaccount.com'}
]
}
]
}
with open(f"{self.rules_directory}/compute_rules.yaml", 'w') as f:
yaml.dump(compute_rules, f, default_flow_style=False)
print("Compute rules created successfully")
def create_cloudsql_rules(self):
"""Create Cloud SQL security rules"""
cloudsql_rules = {
'rules': [
{
'name': 'Cloud SQL instance allows public access',
'mode': 'blacklist',
'resource': [{'type': 'cloudsql_instance'}],
'filters': [
{'key': 'settings.ipConfiguration.authorizedNetworks.value', 'op': 'eq', 'value': '0.0.0.0/0'}
]
},
{
'name': 'Cloud SQL instance has SSL disabled',
'mode': 'blacklist',
'resource': [{'type': 'cloudsql_instance'}],
'filters': [
{'key': 'settings.ipConfiguration.requireSsl', 'op': 'eq', 'value': False}
]
},
{
'name': 'Cloud SQL instance has backup disabled',
'mode': 'blacklist',
'resource': [{'type': 'cloudsql_instance'}],
'filters': [
{'key': 'settings.backupConfiguration.enabled', 'op': 'eq', 'value': False}
]
}
]
}
with open(f"{self.rules_directory}/cloudsql_rules.yaml", 'w') as f:
yaml.dump(cloudsql_rules, f, default_flow_style=False)
print("Cloud SQL rules created successfully")
def create_kubernetes_rules(self):
"""Create Kubernetes security rules"""
kubernetes_rules = {
'rules': [
{
'name': 'GKE cluster has legacy ABAC enabled',
'mode': 'blacklist',
'resource': [{'type': 'ke_cluster'}],
'filters': [
{'key': 'legacyAbac.enabled', 'op': 'eq', 'value': True}
]
},
{
'name': 'GKE cluster has basic authentication enabled',
'mode': 'blacklist',
'resource': [{'type': 'ke_cluster'}],
'filters': [
{'key': 'masterAuth.username', 'op': 'ne', 'value': ''}
]
},
{
'name': 'GKE cluster has client certificate enabled',
'mode': 'blacklist',
'resource': [{'type': 'ke_cluster'}],
'filters': [
{'key': 'masterAuth.clientCertificateConfig.issueClientCertificate', 'op': 'eq', 'value': True}
]
},
{
'name': 'GKE node pool has legacy metadata endpoints enabled',
'mode': 'blacklist',
'resource': [{'type': 'ke_node_pool'}],
'filters': [
{'key': 'config.metadata.disable-legacy-endpoints', 'op': 'ne', 'value': 'true'}
]
}
]
}
with open(f"{self.rules_directory}/kubernetes_rules.yaml", 'w') as f:
yaml.dump(kubernetes_rules, f, default_flow_style=False)
print("Kubernetes rules created successfully")
def create_whitelist_rules(self):
"""Create whitelist configuration"""
whitelist_config = {
'whitelist': {
'firewall_rule': [
'firewall-rule-web-allow-80',
'firewall-rule-web-allow-443',
'firewall-rule-bastion-ssh'
],
'instance': [
'bastion-host-prod',
'web-server-prod-*'
],
'bucket_acl': [
'public-website-bucket',
'public-assets-bucket'
],
'iam_policy': [
'user:admin@company.com',
'serviceAccount:terraform@project.iam.gserviceaccount.com'
]
}
}
with open(f"{self.rules_directory}/whitelist.yaml", 'w') as f:
yaml.dump(whitelist_config, f, default_flow_style=False)
print("Whitelist configuration created successfully")
def create_custom_rule(self, rule_name, resource_type, filters, mode='blacklist', actions=None):
"""Create a custom rule"""
rule = {
'rules': [
{
'name': rule_name,
'mode': mode,
'resource': [{'type': resource_type}],
'filters': filters
}
]
}
if actions:
rule['rules'][0]['actions'] = actions
filename = f"{rule_name.lower().replace(' ', '_')}_rule.yaml"
with open(f"{self.rules_directory}/{filename}", 'w') as f:
yaml.dump(rule, f, default_flow_style=False)
print(f"Custom rule '{rule_name}' created successfully: {filename}")
def validate_rule(self, rule_file):
"""Validate rule syntax"""
try:
with open(rule_file, 'r') as f:
rule_data = yaml.safe_load(f)
# Basic validation
if 'rules' not in rule_data:
return False, "Missing 'rules' key"
for rule in rule_data['rules']:
required_keys = ['name', 'mode', 'resource', 'filters']
for key in required_keys:
if key not in rule:
return False, f"Missing required key: {key}"
# Validate mode
if rule['mode'] not in ['blacklist', 'whitelist']:
return False, f"Invalid mode: {rule['mode']}"
# Validate resource type
valid_types = [
'firewall_rule', 'instance', 'bucket_acl', 'iam_policy',
'service_account_key', 'cloudsql_instance', 'ke_cluster', 'ke_node_pool'
]
for resource in rule['resource']:
if resource['type'] not in valid_types:
return False, f"Invalid resource type: {resource['type']}"
return True, "Rule validation passed"
except Exception as e:
return False, f"Validation error: {e}"
def list_rules(self):
"""List all rule files"""
rule_files = []
for file in os.listdir(self.rules_directory):
if file.endswith('.yaml') or file.endswith('.yml'):
rule_files.append(file)
return rule_files
def generate_all_rules(self):
"""Generate all default security rules"""
print("Generating Forseti Security rules...")
self.create_firewall_rules()
self.create_iam_rules()
self.create_storage_rules()
self.create_compute_rules()
self.create_cloudsql_rules()
self.create_kubernetes_rules()
self.create_whitelist_rules()
print(f"All rules generated in: {self.rules_directory}")
# List generated rules
rules = self.list_rules()
print(f"Generated {len(rules)} rule files:")
for rule in rules:
print(f" - {rule}")
def main():
"""Main function for rule management"""
import argparse
parser = argparse.ArgumentParser(description='Forseti Security Rule Manager')
parser.add_argument('--action', choices=['generate', 'validate', 'list', 'custom'],
required=True, help='Action to perform')
parser.add_argument('--rules-dir', default='/home/forseti/forseti-security/rules',
help='Rules directory path')
parser.add_argument('--rule-file', help='Rule file to validate')
parser.add_argument('--rule-name', help='Custom rule name')
parser.add_argument('--resource-type', help='Resource type for custom rule')
parser.add_argument('--filters', help='Filters for custom rule (JSON format)')
args = parser.parse_args()
manager = ForsetiRuleManager(args.rules_dir)
if args.action == 'generate':
manager.generate_all_rules()
elif args.action == 'validate' and args.rule_file:
valid, message = manager.validate_rule(args.rule_file)
print(f"Validation result: {message}")
exit(0 if valid else 1)
elif args.action == 'list':
rules = manager.list_rules()
print(f"Found {len(rules)} rule files:")
for rule in rules:
print(f" - {rule}")
elif args.action == 'custom':
if not all([args.rule_name, args.resource_type, args.filters]):
print("Custom rule requires: --rule-name, --resource-type, --filters")
exit(1)
try:
filters = json.loads(args.filters)
manager.create_custom_rule(args.rule_name, args.resource_type, filters)
except json.JSONDecodeError:
print("Invalid JSON format for filters")
exit(1)
else:
parser.print_help()
if __name__ == "__main__":
main()
Monitoring and Reporting
Compliance Monitoring
#!/usr/bin/env python3
# Forseti Security Monitoring and Reporting
import mysql.connector
import json
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from datetime import datetime, timedelta
import smtplib
from email.mime.text import MimeText
from email.mime.multipart import MimeMultipart
from email.mime.base import MimeBase
from email import encoders
import logging
from google.cloud import storage
from google.cloud import securitycenter
class ForsetiMonitor:
"""Monitor and report on Forseti Security findings"""
def __init__(self, config):
self.config = config
self.setup_logging()
self.setup_connections()
def setup_logging(self):
"""Setup logging configuration"""
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('forseti_monitor.log'),
logging.StreamHandler()
]
)
self.logger = logging.getLogger(__name__)
def setup_connections(self):
"""Setup database and GCP connections"""
# MySQL connection
try:
self.db_connection = mysql.connector.connect(
host=self.config['database']['host'],
port=self.config['database']['port'],
database=self.config['database']['database'],
user=self.config['database']['username'],
password=self.config['database']['password']
)
self.logger.info("Database connection established")
except Exception as e:
self.logger.error(f"Failed to connect to database: {e}")
self.db_connection = None
# GCS client
try:
self.storage_client = storage.Client(project=self.config['gcp']['project_id'])
self.logger.info("GCS client initialized")
except Exception as e:
self.logger.error(f"Failed to initialize GCS client: {e}")
self.storage_client = None
# Security Command Center client
try:
self.scc_client = securitycenter.SecurityCenterClient()
self.logger.info("Security Command Center client initialized")
except Exception as e:
self.logger.error(f"Failed to initialize SCC client: {e}")
self.scc_client = None
def get_violations_summary(self, days=30):
"""Get violations summary from database"""
if not self.db_connection:
return None
try:
cursor = self.db_connection.cursor(dictionary=True)
end_date = datetime.now()
start_date = end_date - timedelta(days=days)
# Get violations by rule
cursor.execute("""
SELECT
rule_name,
violation_type,
COUNT(*) as violation_count,
COUNT(DISTINCT resource_id) as affected_resources
FROM violations
WHERE created_at_datetime BETWEEN %s AND %s
GROUP BY rule_name, violation_type
ORDER BY violation_count DESC
""", (start_date, end_date))
violations_by_rule = cursor.fetchall()
# Get violations by resource type
cursor.execute("""
SELECT
resource_type,
COUNT(*) as violation_count,
COUNT(DISTINCT resource_id) as affected_resources
FROM violations
WHERE created_at_datetime BETWEEN %s AND %s
GROUP BY resource_type
ORDER BY violation_count DESC
""", (start_date, end_date))
violations_by_type = cursor.fetchall()
# Get daily trend
cursor.execute("""
SELECT
DATE(created_at_datetime) as violation_date,
COUNT(*) as violation_count
FROM violations
WHERE created_at_datetime BETWEEN %s AND %s
GROUP BY DATE(created_at_datetime)
ORDER BY violation_date
""", (start_date, end_date))
daily_trend = cursor.fetchall()
cursor.close()
summary = {
'period_days': days,
'violations_by_rule': violations_by_rule,
'violations_by_type': violations_by_type,
'daily_trend': daily_trend,
'total_violations': sum(v['violation_count'] for v in violations_by_rule),
'generated_at': datetime.now().isoformat()
}
return summary
except Exception as e:
self.logger.error(f"Error getting violations summary: {e}")
return None
def get_inventory_summary(self):
"""Get inventory summary from database"""
if not self.db_connection:
return None
try:
cursor = self.db_connection.cursor(dictionary=True)
# Get latest inventory
cursor.execute("""
SELECT
inventory_type,
COUNT(*) as resource_count
FROM inventory i
JOIN inventory_index ii ON i.inventory_index_id = ii.inventory_index_id
WHERE ii.id = (SELECT MAX(id) FROM inventory_index WHERE completed_at_datetime IS NOT NULL)
GROUP BY inventory_type
ORDER BY resource_count DESC
""")
inventory_by_type = cursor.fetchall()
# Get inventory history
cursor.execute("""
SELECT
completed_at_datetime,
count_objects
FROM inventory_index
WHERE completed_at_datetime IS NOT NULL
ORDER BY completed_at_datetime DESC
LIMIT 30
""")
inventory_history = cursor.fetchall()
cursor.close()
summary = {
'inventory_by_type': inventory_by_type,
'inventory_history': inventory_history,
'total_resources': sum(i['resource_count'] for i in inventory_by_type),
'generated_at': datetime.now().isoformat()
}
return summary
except Exception as e:
self.logger.error(f"Error getting inventory summary: {e}")
return None
def get_compliance_score(self, violations_summary):
"""Calculate compliance score based on violations"""
if not violations_summary:
return 0
total_violations = violations_summary['total_violations']
# Weight violations by severity (based on rule name patterns)
critical_weight = 10
high_weight = 5
medium_weight = 2
low_weight = 1
weighted_score = 0
for violation in violations_summary['violations_by_rule']:
rule_name = violation['rule_name'].lower()
count = violation['violation_count']
if any(keyword in rule_name for keyword in ['public', 'internet', 'all users', 'owner']):
weighted_score += count * critical_weight
elif any(keyword in rule_name for keyword in ['ssl', 'backup', 'encryption']):
weighted_score += count * high_weight
elif any(keyword in rule_name for keyword in ['logging', 'monitoring']):
weighted_score += count * medium_weight
else:
weighted_score += count * low_weight
# Calculate compliance score (0-100)
max_possible_score = 1000 # Arbitrary baseline
compliance_score = max(0, 100 - (weighted_score / max_possible_score * 100))
return round(compliance_score, 2)
def generate_compliance_report(self, days=30):
"""Generate comprehensive compliance report"""
violations_summary = self.get_violations_summary(days)
inventory_summary = self.get_inventory_summary()
if not violations_summary:
self.logger.error("Failed to generate compliance report")
return None
compliance_score = self.get_compliance_score(violations_summary)
report = {
'report_metadata': {
'generated_at': datetime.now().isoformat(),
'period_days': days,
'report_type': 'forseti_compliance'
},
'executive_summary': {
'compliance_score': compliance_score,
'total_violations': violations_summary['total_violations'],
'total_resources': inventory_summary['total_resources'] if inventory_summary else 0,
'period_days': days
},
'violations_summary': violations_summary,
'inventory_summary': inventory_summary,
'recommendations': self._generate_recommendations(violations_summary)
}
return report
def _generate_recommendations(self, violations_summary):
"""Generate recommendations based on violations"""
recommendations = []
if not violations_summary:
return recommendations
# Analyze top violations
top_violations = sorted(
violations_summary['violations_by_rule'],
key=lambda x: x['violation_count'],
reverse=True
)[:5]
for violation in top_violations:
rule_name = violation['rule_name']
count = violation['violation_count']
if 'public' in rule_name.lower() or 'internet' in rule_name.lower():
recommendations.append({
'priority': 'HIGH',
'title': 'Address Public Access Violations',
'description': f"{count} resources have public access configured",
'action': f"Review and restrict public access for {rule_name}",
'rule_name': rule_name
})
elif 'firewall' in rule_name.lower():
recommendations.append({
'priority': 'HIGH',
'title': 'Review Firewall Rules',
'description': f"{count} firewall rule violations detected",
'action': f"Audit and update firewall rules for {rule_name}",
'rule_name': rule_name
})
elif 'iam' in rule_name.lower() or 'service account' in rule_name.lower():
recommendations.append({
'priority': 'MEDIUM',
'title': 'Review IAM Permissions',
'description': f"{count} IAM-related violations found",
'action': f"Review and update IAM policies for {rule_name}",
'rule_name': rule_name
})
return recommendations
def create_compliance_dashboard(self, report_data, output_file='forseti_dashboard.png'):
"""Create compliance dashboard visualization"""
try:
plt.style.use('seaborn-v0_8')
fig, axes = plt.subplots(2, 3, figsize=(18, 12))
violations_summary = report_data['violations_summary']
inventory_summary = report_data['inventory_summary']
executive_summary = report_data['executive_summary']
# 1. Compliance Score Gauge
compliance_score = executive_summary['compliance_score']
colors = ['#e74c3c' if compliance_score < 70 else '#f39c12' if compliance_score < 85 else '#27ae60']
axes[0, 0].pie([compliance_score, 100-compliance_score],
labels=[f'{compliance_score}%', ''],
colors=[colors[0], '#ecf0f1'],
startangle=90)
axes[0, 0].set_title('Compliance Score')
# 2. Violations by Rule (Top 10)
if violations_summary['violations_by_rule']:
top_rules = violations_summary['violations_by_rule'][:10]
rule_names = [r['rule_name'][:30] + '...' if len(r['rule_name']) > 30 else r['rule_name']
for r in top_rules]
violation_counts = [r['violation_count'] for r in top_rules]
axes[0, 1].barh(rule_names, violation_counts, color='#e74c3c')
axes[0, 1].set_title('Top 10 Violating Rules')
axes[0, 1].set_xlabel('Violation Count')
# 3. Violations by Resource Type
if violations_summary['violations_by_type']:
resource_types = [v['resource_type'] for v in violations_summary['violations_by_type']]
type_counts = [v['violation_count'] for v in violations_summary['violations_by_type']]
axes[0, 2].pie(type_counts, labels=resource_types, autopct='%1.1f%%')
axes[0, 2].set_title('Violations by Resource Type')
# 4. Daily Violation Trend
if violations_summary['daily_trend']:
dates = [datetime.strptime(str(d['violation_date']), '%Y-%m-%d')
for d in violations_summary['daily_trend']]
counts = [d['violation_count'] for d in violations_summary['daily_trend']]
axes[1, 0].plot(dates, counts, marker='o', color='#e74c3c')
axes[1, 0].set_title('Daily Violation Trend')
axes[1, 0].set_ylabel('Violation Count')
axes[1, 0].tick_params(axis='x', rotation=45)
# 5. Inventory by Resource Type
if inventory_summary and inventory_summary['inventory_by_type']:
inv_types = [i['inventory_type'] for i in inventory_summary['inventory_by_type'][:8]]
inv_counts = [i['resource_count'] for i in inventory_summary['inventory_by_type'][:8]]
axes[1, 1].bar(inv_types, inv_counts, color='#3498db')
axes[1, 1].set_title('Inventory by Resource Type')
axes[1, 1].set_ylabel('Resource Count')
axes[1, 1].tick_params(axis='x', rotation=45)
# 6. Summary Metrics
metrics_text = f"""
Total Violations: {executive_summary['total_violations']}
Total Resources: {executive_summary['total_resources']}
Compliance Score: {executive_summary['compliance_score']}%
Period: {executive_summary['period_days']} days
Generated: {report_data['report_metadata']['generated_at'][:10]}
"""
axes[1, 2].text(0.1, 0.5, metrics_text, fontsize=12, verticalalignment='center')
axes[1, 2].set_xlim(0, 1)
axes[1, 2].set_ylim(0, 1)
axes[1, 2].axis('off')
axes[1, 2].set_title('Summary Metrics')
plt.tight_layout()
plt.savefig(output_file, dpi=300, bbox_inches='tight')
plt.close()
self.logger.info(f"Compliance dashboard created: {output_file}")
return output_file
except Exception as e:
self.logger.error(f"Error creating compliance dashboard: {e}")
return None
def send_compliance_report(self, report_data, dashboard_file=None):
"""Send compliance report via email"""
if not self.config.get('notifications', {}).get('email', {}).get('enabled', False):
self.logger.info("Email notifications disabled")
return False
try:
email_config = self.config['notifications']['email']
# Create email message
msg = MimeMultipart()
msg['From'] = email_config['sender']
msg['To'] = email_config['recipient']
msg['Subject'] = f"Forseti Security Compliance Report - {datetime.now().strftime('%Y-%m-%d')}"
# Create email body
executive_summary = report_data['executive_summary']
body = f"""
Forseti Security Compliance Report
Compliance Score: {executive_summary['compliance_score']}%
Total Violations: {executive_summary['total_violations']}
Total Resources: {executive_summary['total_resources']}
Period: {executive_summary['period_days']} days
Top Recommendations:
"""
for rec in report_data['recommendations'][:5]:
body += f"\n- {rec['title']}: {rec['description']}"
body += f"\n\nGenerated at: {report_data['report_metadata']['generated_at']}"
msg.attach(MimeText(body, 'plain'))
# Attach dashboard if available
if dashboard_file:
with open(dashboard_file, 'rb') as attachment:
part = MimeBase('application', 'octet-stream')
part.set_payload(attachment.read())
encoders.encode_base64(part)
part.add_header(
'Content-Disposition',
f'attachment; filename= {dashboard_file}'
)
msg.attach(part)
# Send email
server = smtplib.SMTP(email_config['smtp_host'], email_config['smtp_port'])
server.starttls()
server.login(email_config['username'], email_config['password'])
server.send_message(msg)
server.quit()
self.logger.info("Compliance report sent via email")
return True
except Exception as e:
self.logger.error(f"Error sending compliance report: {e}")
return False
def upload_to_gcs(self, report_data, bucket_name):
"""Upload report to Google Cloud Storage"""
if not self.storage_client:
return False
try:
bucket = self.storage_client.bucket(bucket_name)
# Upload JSON report
report_blob_name = f"forseti-reports/compliance-report-{datetime.now().strftime('%Y%m%d-%H%M%S')}.json"
report_blob = bucket.blob(report_blob_name)
report_blob.upload_from_string(json.dumps(report_data, indent=2))
self.logger.info(f"Report uploaded to GCS: gs://{bucket_name}/{report_blob_name}")
return True
except Exception as e:
self.logger.error(f"Error uploading to GCS: {e}")
return False
def run_monitoring_workflow(self, days=30):
"""Run complete monitoring workflow"""
self.logger.info("Starting Forseti Security monitoring workflow")
# Generate report
report = self.generate_compliance_report(days)
if not report:
self.logger.error("Failed to generate compliance report")
return False
# Save report to file
report_file = f"forseti_compliance_report_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json"
with open(report_file, 'w') as f:
json.dump(report, f, indent=2)
# Create dashboard
dashboard_file = f"forseti_dashboard_{datetime.now().strftime('%Y%m%d_%H%M%S')}.png"
self.create_compliance_dashboard(report, dashboard_file)
# Send report
self.send_compliance_report(report, dashboard_file)
# Upload to GCS
if self.config.get('gcp', {}).get('bucket_name'):
self.upload_to_gcs(report, self.config['gcp']['bucket_name'])
self.logger.info("Forseti Security monitoring workflow completed")
return True
def main():
"""Main function for Forseti monitoring"""
import argparse
import yaml
parser = argparse.ArgumentParser(description='Forseti Security Monitor')
parser.add_argument('--config', default='forseti_monitor_config.yaml', help='Configuration file')
parser.add_argument('--days', type=int, default=30, help='Number of days to analyze')
args = parser.parse_args()
# Load configuration
try:
with open(args.config, 'r') as f:
config = yaml.safe_load(f)
except FileNotFoundError:
# Create default configuration
config = {
'database': {
'host': 'localhost',
'port': 3306,
'database': 'forseti_security',
'username': 'forseti_security_user',
'password': 'forseti123'
},
'gcp': {
'project_id': 'your-project-id',
'bucket_name': 'forseti-security-bucket'
},
'notifications': {
'email': {
'enabled': True,
'smtp_host': 'smtp.gmail.com',
'smtp_port': 587,
'username': 'forseti@company.com',
'password': 'app-password',
'sender': 'forseti@company.com',
'recipient': 'security-team@company.com'
}
}
}
with open(args.config, 'w') as f:
yaml.dump(config, f, default_flow_style=False)
print(f"Created default configuration: {args.config}")
print("Please update the configuration file with your settings")
return
# Run monitoring
monitor = ForsetiMonitor(config)
monitor.run_monitoring_workflow(args.days)
if __name__ == "__main__":
main()
Automation and Integration
CI/CD Integration
# .github/workflows/forseti-security-scan.yml
name: Forseti Security Scan
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
schedule:
# Run daily at 2 AM UTC
- cron: '0 2 * * *'
workflow_dispatch:
inputs:
scan_type:
description: 'Type of scan to run'
required: false
default: 'full'
type: choice
options:
- full
- inventory
- scanner
- explain
jobs:
forseti-security-scan:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.8'
- name: Install dependencies
run: |
pip install google-cloud-storage google-cloud-securitycenter mysql-connector-python pyyaml pandas matplotlib seaborn
- name: Setup Google Cloud credentials
uses: google-github-actions/auth@v1
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
- name: Setup Google Cloud SDK
uses: google-github-actions/setup-gcloud@v1
- name: Run Forseti Security scan
env:
FORSETI_SERVER_HOST: ${{ secrets.FORSETI_SERVER_HOST }}
FORSETI_SERVER_PORT: ${{ secrets.FORSETI_SERVER_PORT }}
FORSETI_DB_HOST: ${{ secrets.FORSETI_DB_HOST }}
FORSETI_DB_PASSWORD: ${{ secrets.FORSETI_DB_PASSWORD }}
GCP_PROJECT_ID: ${{ secrets.GCP_PROJECT_ID }}
GCP_ORGANIZATION_ID: ${{ secrets.GCP_ORGANIZATION_ID }}
run: |
# Create Forseti client configuration
cat > forseti_client_config.yaml << EOF
server_ip: ${FORSETI_SERVER_HOST}
server_port: ${FORSETI_SERVER_PORT}
EOF
# Create monitoring configuration
cat > forseti_monitor_config.yaml << EOF
database:
host: ${FORSETI_DB_HOST}
port: 3306
database: forseti_security
username: forseti_security_user
password: ${FORSETI_DB_PASSWORD}
gcp:
project_id: ${GCP_PROJECT_ID}
organization_id: ${GCP_ORGANIZATION_ID}
bucket_name: forseti-security-bucket
notifications:
email:
enabled: false
EOF
# Install Forseti Security client
git clone https://github.com/forseti-security/forseti-security.git
cd forseti-security
git checkout v2.25.1
pip install -r requirements.txt
pip install -e .
# Run Forseti operations based on scan type
SCAN_TYPE="${{ github.event.inputs.scan_type || 'full' }}"
if [ "$SCAN_TYPE" = "full" ] || [ "$SCAN_TYPE" = "inventory" ]; then
echo "Running inventory scan..."
python -m google.cloud.forseti.services.inventory.storage \
--config_file_path=../forseti_client_config.yaml
fi
if [ "$SCAN_TYPE" = "full" ] || [ "$SCAN_TYPE" = "scanner" ]; then
echo "Running scanner..."
python -m google.cloud.forseti.scanner.scanner \
--config_file_path=../forseti_client_config.yaml
fi
if [ "$SCAN_TYPE" = "full" ] || [ "$SCAN_TYPE" = "explain" ]; then
echo "Running explain..."
python -m google.cloud.forseti.services.explain.explain \
--config_file_path=../forseti_client_config.yaml
fi
# Generate compliance report
cd ..
python scripts/forseti_monitor.py \
--config forseti_monitor_config.yaml \
--days 7
- name: Evaluate security gate
run: |
python << 'EOF'
import json
import sys
import glob
# Find compliance report
report_files = glob.glob('forseti_compliance_report_*.json')
if not report_files:
print("No compliance report found")
sys.exit(0)
with open(report_files[0], 'r') as f:
report = json.load(f)
executive_summary = report['executive_summary']
compliance_score = executive_summary['compliance_score']
total_violations = executive_summary['total_violations']
print(f"Forseti Security Assessment Results:")
print(f"Compliance Score: {compliance_score}%")
print(f"Total Violations: {total_violations}")
# Security gate logic
if compliance_score < 70:
print("❌ SECURITY FAILURE!")
print("Compliance score below acceptable threshold (70%)")
sys.exit(1)
if total_violations > 100:
print("⚠️ WARNING: High number of violations!")
sys.exit(1)
print("✅ Security gate passed")
EOF
- name: Upload scan results
uses: actions/upload-artifact@v3
with:
name: forseti-security-results
path: |
forseti_compliance_report_*.json
forseti_dashboard_*.png
- name: Comment PR with security status
if: github.event_name == 'pull_request'
uses: actions/github-script@v6
with:
script: |
const fs = require('fs');
const glob = require('glob');
// Find compliance report
const reportFiles = glob.sync('forseti_compliance_report_*.json');
if (reportFiles.length === 0) {
console.log('No compliance report found');
return;
}
const report = JSON.parse(fs.readFileSync(reportFiles[0], 'utf8'));
const summary = report.executive_summary;
const comment = `## 🔒 Forseti Security Scan Results
**Compliance Score:** ${summary.compliance_score}%
**Total Violations:** ${summary.total_violations}
**Total Resources:** ${summary.total_resources}
**Top Violations:**
${report.violations_summary.violations_by_rule.slice(0, 5).map(v =>
`- ${v.rule_name}: ${v.violation_count} violations`
).join('\n')}
**Recommendations:**
${report.recommendations.slice(0, 3).map(r =>
`- **${r.title}**: ${r.description}`
).join('\n')}
${summary.compliance_score < 70 ? '⚠️ **Compliance score below threshold! Please review and remediate violations.**' : '✅ Security scan passed.'}
[View detailed report](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: comment
});
Resources and Documentation
Official Resources
- Forseti Security GitHub Repository - Source code and documentation
- Forseti Security Documentation - Official documentation and guides
- Google Cloud Security - Google Cloud security best practices
- Cloud Security Command Center - Integration with CSCC
GCP Security Resources
- GCP Security Best Practices - Google Cloud security guidelines
- IAM Best Practices - Identity and Access Management
- VPC Security - Virtual Private Cloud security
- Cloud Asset Inventory - Asset discovery and monitoring
Compliance and Governance
- CIS Google Cloud Platform Benchmark - CIS benchmarks for GCP
- NIST Cybersecurity Framework - NIST security framework
- SOC 2 Compliance - SOC 2 compliance on GCP
- GDPR Compliance - GDPR compliance resources