kafka-observability

# Kafka Monitoring & Observability

Expert guidance for implementing comprehensive monitoring and observability for Apache Kafka using Prometheus and Grafana.

## When to Use This Skill

I activate when you need help with:
- **Monitoring setup**: "Set up Kafka monitoring", "configure Prometheus for Kafka", "Grafana dashboards for Kafka"
- **Metrics collection**: "Kafka JMX metrics", "export Kafka metrics to Prometheus"
- **Alerting**: "Kafka alerting rules", "alert on under-replicated partitions", "critical Kafka metrics"
- **Troubleshooting**: "Monitor Kafka performance", "track consumer lag", "broker health monitoring"

## What I Know

### Available Monitoring Components

This plugin provides a complete monitoring stack:

#### 1. **Prometheus JMX Exporter Configuration**
- **Location**: `plugins/specweave-kafka/monitoring/prometheus/kafka-jmx-exporter.yml`
- **Purpose**: Export Kafka JMX metrics to Prometheus format
- **Metrics Exported**:
  - Broker topic metrics (bytes in/out, messages in, request rate)
  - Replica manager (under-replicated partitions, ISR shrinks/expands)
  - Controller metrics (active controller, offline partitions, leader elections)
  - Request metrics (produce/fetch latency)
  - Log metrics (flush rate, flush latency)
  - JVM metrics (heap, GC, threads, file descriptors)

#### 2. **Grafana Dashboards** (5 Dashboards)
- **Location**: `plugins/specweave-kafka/monitoring/grafana/dashboards/`
- **Dashboards**:
  1. **kafka-cluster-overview.json** - Cluster health and throughput
  2. **kafka-broker-metrics.json** - Per-broker performance
  3. **kafka-consumer-lag.json** - Consumer lag monitoring
  4. **kafka-topic-metrics.json** - Topic-level metrics
  5. **kafka-jvm-metrics.json** - JVM health (heap, GC, threads)

#### 3. **Grafana Provisioning**
- **Location**: `plugins/specweave-kafka/monitoring/grafana/provisioning/`
- **Files**:
  - `dashboards/kafka.yml` - Dashboard provisioning config
  - `datasources/prometheus.yml` - Prometheus datasourc