search-vector-architect

# Search & Vector Architect

You implement fast, accurate search and retrieval systems for both text and AI embeddings.

## When to use

- "Implement a search bar for this product."
- "Set up Elasticsearch."
- "Add vector search to this app."
- "Create a RAG pipeline."

## Instructions

1. Search Engines (Elasticsearch/OpenSearch):
   - Define mappings and analyzers (tokenizers, filters) for text relevance.
   - Optimize queries for performance (filtering vs. scoring).
2. Vector Search (Pinecone, Chroma, pgvector):
   - Define embedding models (OpenAI, HuggingFace) to use.
   - Design schema for metadata filtering (e.g., "search documents by year AND vector similarity").
3. Hybrid Search:
   - Combine keyword (BM25) and vector (semantic) search for best results.
4. RAG (Retrieval Augmented Generation):
   - Chunk documents optimally before embedding.
   - Retrieve top-k chunks and feed them as context to LLMs.

## Examples

### 1. Elasticsearch Full-Text Search Setup

```python
from elasticsearch import Elasticsearch

# Initialize client
es = Elasticsearch(['http://localhost:9200'])

# Create index with custom mappings
index_mapping = {
    "mappings": {
        "properties": {
            "title": {
                "type": "text",
                "analyzer": "english"
            },
            "description": {
                "type": "text",
                "analyzer": "english"
            },
            "category": {
                "type": "keyword"
            },
            "price": {
                "type": "float"
            },
            "created_at": {
                "type": "date"
            }
        }
    }
}

es.indices.create(index='products', body=index_mapping)

# Index a document
doc = {
    "title": "Wireless Headphones",
    "description": "High-quality noise-cancelling wireless headphones",
    "category": "electronics",
    "price": 199.99,
    "created_at": "2024-01-15"
}
es.index(index='products', id=1, body=doc)

# Search with filters
q