ml-pipeline-orchestrator

# ML Pipeline Orchestrator

## Overview

This skill transforms ML development into a SpecWeave increment-based workflow, ensuring every ML project follows the same disciplined approach: spec → plan → tasks → implement → validate. It orchestrates the complete ML lifecycle from data exploration to model deployment, with full traceability and living documentation.

## Core Philosophy

**SpecWeave + ML = Disciplined Data Science**

Traditional ML development often lacks structure:
- ❌ Jupyter notebooks with no version control
- ❌ Experiments without documentation
- ❌ Models deployed with no reproducibility
- ❌ Team knowledge trapped in individual notebooks

SpecWeave brings discipline:
- ✅ Every ML feature is an increment (with spec, plan, tasks)
- ✅ Experiments tracked and documented automatically
- ✅ Model versions tied to increments
- ✅ Living docs capture learnings and decisions

## How It Works

### Phase 1: ML Increment Planning

When you request "build a recommendation model", the skill:

1. **Creates ML increment structure**:
```
.specweave/increments/0042-recommendation-model/
├── spec.md                    # ML requirements, success metrics
├── plan.md                    # Pipeline architecture
├── tasks.md                   # Implementation tasks
├── tests.md                   # Evaluation criteria
├── experiments/               # Experiment tracking
│   ├── exp-001-baseline/
│   ├── exp-002-xgboost/
│   └── exp-003-neural-net/
├── data/                      # Data samples, schemas
│   ├── schema.yaml
│   └── sample.csv
├── models/                    # Trained models
│   ├── model-v1.pkl
│   └── model-v2.pkl
└── notebooks/                 # Exploratory notebooks
    ├── 01-eda.ipynb
    └── 02-feature-engineering.ipynb
```

2. **Generates ML-specific spec** (spec.md):
```markdown
## ML Problem Definition
- Problem type: Recommendation (collaborative filtering)
- Input: User behavior history
- Output: Top-N product recommendations
- Success metrics: Precision@