observability-patterns

# Observability Patterns

Patterns for implementing comprehensive observability including logs, metrics, traces, and their correlation.

## When to Use This Skill

- Designing observability strategy
- Implementing the three pillars
- Correlating signals across systems
- Choosing observability tools
- Building monitoring dashboards

## What is Observability?

```text
Observability = Ability to understand internal state
                from external outputs

Not just monitoring (known-unknowns)
But understanding (unknown-unknowns)

Traditional monitoring: "Is CPU > 80%?"
Observability: "Why are users experiencing latency?"
```

## The Three Pillars

### Overview

```text
┌─────────────────────────────────────────────────────────┐
│                    OBSERVABILITY                         │
│                                                          │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐         │
│   │   LOGS   │    │ METRICS  │    │  TRACES  │         │
│   │          │    │          │    │          │         │
│   │ Events   │    │ Counters │    │ Requests │         │
│   │ Details  │    │ Gauges   │    │ Spans    │         │
│   │ Context  │    │ Trends   │    │ Flow     │         │
│   └──────────┘    └──────────┘    └──────────┘         │
│        │               │               │                │
│        └───────────────┼───────────────┘                │
│                        │                                │
│               ┌────────┴────────┐                       │
│               │   CORRELATION   │                       │
│               │  (trace_id)     │                       │
│               └─────────────────┘                       │
└─────────────────────────────────────────────────────────┘

Each pillar answers different questions:
- Logs: What happened? (events)
- Metrics: How much/many? (aggregates)
- Traces: Where? (request flow)
```

### Logs

```text
Purpose: Discrete events with context

Structure:
{
  "timestamp": "2024-01-15T