dataform-debugging

# Dataform & BigQuery Debugging Guide

This skill provides troubleshooting knowledge for Dataform and BigQuery environments. Reference when diagnosing compilation, authentication, data quality, or runtime errors.

---

## Quick Diagnostics Checklist

Run these checks in order when debugging:

### 1. Authentication Status

```bash
gcloud auth list

gcloud config get-value project

gcloud config get-value account
```

**Expected**: Active account with correct project set

### 2. Dataform Compile Check

```bash
dataform compile 2>&1 | head -100
```

**Expected**: "Compiled successfully"

### 3. BigQuery Access Check

```bash
bq ls expanly-beta-query-builder:expanly_query_builder_data | head -20
```

**Expected**: List of tables

---

## Authentication Issues

### Problem: "Request had insufficient authentication scopes"

**Solution**:
```bash
gcloud auth application-default login

gcloud auth application-default set-quota-project expanly-beta-query-builder
```

### Problem: "Could not automatically determine credentials"

**Solution**:
```bash
gcloud auth login

gcloud auth application-default login
```

### Problem: Wrong Project

**Diagnosis**:
```bash
gcloud config get-value project
```

**Solution**:
```bash
gcloud config set project expanly-beta-query-builder
```

### Problem: Service Account Issues

**Diagnosis**:
```bash
echo $GOOGLE_APPLICATION_CREDENTIALS

cat $GOOGLE_APPLICATION_CREDENTIALS | jq '.client_email'
```

**Solution**: Set correct service account file

---

## Dataform Compile Errors

### Error: "Table not found: table_name"

**Diagnosis**:
```bash
grep -r "ref(\"table_name\")" definitions/

ls definitions/**/table_name.sqlx
```

**Common Causes**:
1. Typo in `${ref("table_name")}`
2. File doesn't exist
3. File has different target name in config

**Fix**: Verify file exists and target name matches reference

### Error: "Duplicate declaration for table"

**Diagnosis**:
```bash
grep -l "schema:.*table_name" definitions/**/*.sqlx
```

**Cause**: Two