detect-anomalies

# Anomaly Detection

Detect anomalies in Axiom datasets by comparing recent patterns to historical baselines using statistical analysis.

## Arguments

When invoked with a dataset name (e.g., `/detect-anomalies logs`), it's available as `$ARGUMENTS`.

## Prerequisites

Statistical anomaly detection requires sufficient data:

- **Minimum data points**: Z-score and standard deviation need ≥30 samples per bucket for statistical significance
- **Historical baseline**: At least 24 hours of data for meaningful comparison (methods use 25h lookback)
- **Consistent ingestion**: Gaps in data collection will skew baselines

If these aren't met, results may be misleading. Consider using simpler threshold-based alerting instead.

## Schema Discovery

**Always verify field names first:**

```bash
axiom query "['<dataset>'] | getschema" --start-time -1h
```

## Anomaly Detection Methods

### 1. Volume Anomaly Detection

Compare recent volume to baseline:

**Calculate baseline (past 24h excluding last hour):**

```bash
axiom query "['<dataset>']
| where _time between (ago(25h) .. ago(1h))
| summarize count() by bin(_time, 1h)
| summarize
    avg_hourly = avg(count_),
    stdev_hourly = stdev(count_)" --start-time -25h -f json
```

**Check recent volume:**

```bash
axiom query "['<dataset>']
| where _time >= ago(1h)
| summarize
    current_count = count(),
    current_hour = min(_time)" --start-time -1h -f json
```

**Z-score calculation:**
- `z_score = (current - avg) / stdev`
- `|z_score| > 2` indicates anomaly

### 2. New Value Detection

Find values that appeared recently but weren't seen before:

```bash
axiom query "['<dataset>']
| where _time >= ago(1h)
| summarize by error_code
| join kind=leftanti (
    ['<dataset>']
    | where _time between (ago(25h) .. ago(1h))
    | summarize by error_code
  ) on error_code" --start-time -25h -f json
```

Replace `error_code` with any categorical field (service, endpoint, status).

### 3. Statistical Outliers

Find values outside normal