kafka-iac-deployment

# Kafka Infrastructure as Code (IaC) Deployment

Expert guidance for deploying Apache Kafka using Terraform across multiple platforms.

## When to Use This Skill

I activate when you need help with:
- **Terraform deployments**: "Deploy Kafka with Terraform", "provision Kafka cluster"
- **Platform selection**: "Should I use AWS MSK or self-hosted Kafka?", "compare Kafka platforms"
- **Infrastructure planning**: "How to size Kafka infrastructure", "Kafka on AWS vs Azure"
- **IaC automation**: "Automate Kafka deployment", "CI/CD for Kafka infrastructure"

## What I Know

### Available Terraform Modules

This plugin provides 3 production-ready Terraform modules:

#### 1. **Apache Kafka (Self-Hosted, KRaft Mode)**
- **Location**: `plugins/specweave-kafka/terraform/apache-kafka/`
- **Platform**: AWS EC2 (can adapt to other clouds)
- **Architecture**: KRaft mode (no ZooKeeper dependency)
- **Features**:
  - Multi-broker cluster (3-5 brokers recommended)
  - Security groups with SASL_SSL
  - IAM roles for S3 backups
  - CloudWatch metrics and alarms
  - Auto-scaling group support
  - Custom VPC and subnet configuration
- **Use When**:
  - ✅ You need full control over Kafka configuration
  - ✅ Running Kafka 3.6+ (KRaft mode)
  - ✅ Want to avoid ZooKeeper operational overhead
  - ✅ Multi-cloud or hybrid deployments
- **Variables**:
  ```hcl
  module "kafka" {
    source = "../../plugins/specweave-kafka/terraform/apache-kafka"

    environment         = "production"
    broker_count        = 3
    kafka_version       = "3.7.0"
    instance_type       = "m5.xlarge"
    vpc_id              = var.vpc_id
    subnet_ids          = var.subnet_ids
    domain              = "example.com"
    enable_s3_backups   = true
    enable_monitoring   = true
  }
  ```

#### 2. **AWS MSK (Managed Streaming for Kafka)**
- **Location**: `plugins/specweave-kafka/terraform/aws-msk/`
- **Platform**: AWS Managed Service
- **Features**:
  - Fully managed Kafka service
  - IAM authentication + SASL/SCRA