firecrawl-known-pitfalls

# FireCrawl Known Pitfalls

## Overview
Common mistakes and anti-patterns when integrating with FireCrawl.

## Prerequisites
- Access to FireCrawl codebase for review
- Understanding of async/await patterns
- Knowledge of security best practices
- Familiarity with rate limiting concepts

## Pitfall #1: Synchronous API Calls in Request Path

### ❌ Anti-Pattern
```typescript
// User waits for FireCrawl API call
app.post('/checkout', async (req, res) => {
  const payment = await firecrawlClient.processPayment(req.body);  // 2-5s latency
  const notification = await firecrawlClient.sendEmail(payment);   // Another 1-2s
  res.json({ success: true });  // User waited 3-7s
});
```

### ✅ Better Approach
```typescript
// Return immediately, process async
app.post('/checkout', async (req, res) => {
  const jobId = await queue.enqueue('process-checkout', req.body);
  res.json({ jobId, status: 'processing' });  // 50ms response
});

// Background job
async function processCheckout(data) {
  const payment = await firecrawlClient.processPayment(data);
  await firecrawlClient.sendEmail(payment);
}
```

---

## Pitfall #2: Not Handling Rate Limits

### ❌ Anti-Pattern
```typescript
// Blast requests, crash on 429
for (const item of items) {
  await firecrawlClient.process(item);  // Will hit rate limit
}
```

### ✅ Better Approach
```typescript
import pLimit from 'p-limit';

const limit = pLimit(5);  // Max 5 concurrent
const rateLimiter = new RateLimiter({ tokensPerSecond: 10 });

for (const item of items) {
  await rateLimiter.acquire();
  await limit(() => firecrawlClient.process(item));
}
```

---

## Pitfall #3: Leaking API Keys

### ❌ Anti-Pattern
```typescript
// In frontend code (visible to users!)
const client = new FireCrawlClient({
  apiKey: 'sk_live_ACTUAL_KEY_HERE',  // Anyone can see this
});

// In git history
git commit -m "add API key"  // Exposed forever
```

### ✅ Better Approach
```typescript
// Backend only, environment variable
const client = new FireCrawlClie