distributed-tracing

# Distributed Tracing

Patterns and practices for implementing distributed tracing across microservices and understanding request flows in distributed systems.

## When to Use This Skill

- Implementing distributed tracing in microservices
- Debugging cross-service request issues
- Understanding trace propagation
- Choosing tracing infrastructure
- Correlating logs, metrics, and traces

## Why Distributed Tracing?

```text
Problem: Request flows through multiple services
How do you debug when something fails?

Without tracing:
User → API → ??? → ??? → Error somewhere

With tracing:
User → API (50ms) → OrderService (20ms) → PaymentService (ERROR: timeout)
         └── Full visibility into request flow
```

## Core Concepts

### Traces, Spans, and Context

```text
Trace: End-to-end request journey
├── Span: Single operation within a service
│   ├── SpanID: Unique identifier
│   ├── ParentSpanID: Link to parent span
│   ├── TraceID: Shared across all spans
│   ├── Operation Name: What is being done
│   ├── Start/End Time: Duration
│   ├── Status: Success/Error
│   ├── Attributes: Key-value metadata
│   └── Events: Point-in-time annotations
│
└── Context: Propagated across service boundaries
    ├── TraceID
    ├── SpanID
    ├── Trace Flags
    └── Trace State
```

### Trace Visualization

```text
TraceID: abc123

Service A (API Gateway)
├──────────────────────────────────────────────────────┤ 200ms
    │
    └─► Service B (Order Service)
        ├───────────────────────────────────┤ 150ms
            │
            ├─► Service C (Inventory)
            │   ├───────────────┤ 50ms
            │
            └─► Service D (Payment)
                ├───────────────────────┤ 80ms
                    │
                    └─► External API
                        ├─────────┤ 60ms
```

## OpenTelemetry

### Overview

```text
OpenTelemetry = Unified observability framework

Components:
┌─────────────────────────────────────────────────────┐
│  Application