cloudflare-vectorize

Comprehensive guide for building semantic search, RAG, and AI-powered applications with Cloudflare Vectorize

Auto-Trigger Keywords

This skill automatically activates when you mention:

Vector Database Operations: vectorize, vector database, vector index, vector search, similarity search, semantic search, nearest neighbor, knn search, ann search, vector embeddings, embedding database

RAG & AI Patterns: RAG, retrieval augmented generation, chat with data, document search, semantic Q&A, context retrieval, document retrieval, knowledge base search, AI search, conversational search

Embedding Models: bge-base, @cf/baai/bge-base-en-v1.5, Workers AI embeddings, text-embedding-3-small, text-embedding-3-large, openai embeddings, embedding generation, vector generation

Operations: insert vectors, upsert vectors, query vectors, delete vectors, list vectors, vector metadata, metadata filtering, namespace filtering, topK search, cosine similarity, euclidean distance, dot product similarity

Setup & Configuration: create vectorize index, vectorize dimensions, vectorize metric, vectorize binding, wrangler vectorize, metadata index, metadata filtering, vector namespace

Use Cases: semantic document search, product recommendations, image similarity, content recommendations, duplicate detection, anomaly detection, classification, clustering

Integration: vectorize + workers ai, vectorize + d1, vectorize + r2, vectorize + kv, vectorize + openai, cloudflare vector database

What This Skill Does

Prevents 8 Common Errors

1. Metadata Index Timing: Creating metadata indexes AFTER inserting vectors (vectors won't be indexed)
2. Dimension Mismatch: Using wrong dimensions for embedding model (768 vs 1536 vs 3072)
3. Metadata Key Syntax: Invalid keys with dots, quotes, or dollar signs
4. Filter Syntax: Incorrect operator usage or exceeding 2048 byte filter limit
5. Insert vs Upsert: Using insert() when you need to update existing vectors
6. Missing Bindings: Forgetting vectorize binding in wrangler.jsonc
7. High Cardinality Range Queries: Performance issues from large range scans
8. Namespace vs Metadata: Confusion about when to use namespace vs metadata filtering

Saves ~65% Tokens

Instead of repeatedly explaining:

• How to create indexes with correct dimensions
• Metadata index creation timing
• Filter operator syntax
• Insert vs upsert semantics
• Embedding model configurations
• RAG implementation patterns

You get complete, tested code templates ready to use.

Saves ~2.5 Hours Development Time

Provides:

• ✅ 4 working TypeScript templates
• ✅ Complete wrangler CLI reference
• ✅ Workers AI + OpenAI integration examples
• ✅ Document chunking strategies
• ✅ Metadata filtering patterns
• ✅ RAG implementation examples

Core Features

Index Management

• Create indexes with fixed dimensions and distance metrics
• Configure metadata indexes for filtering
• Manage namespaces for multi-tenant isolation
• List, get, and delete indexes via CLI or API

Vector Operations

• Insert: Add vectors (keeps first if ID exists)
• Upsert: Add or update vectors (overwrites if ID exists)
• Query: Semantic search with topK, filters, namespaces
• Delete: Remove vectors by ID
• List: Paginated vector ID listing
• Get: Fetch specific vectors by ID

Metadata Filtering

• 10 metadata indexes per Vectorize index
• String, number, boolean types supported
• Operators: $eq, $ne, $in, $nin, $lt, $lte, $gt, $gte
• Nested metadata with dot notation
• Range queries for numbers and strings (prefix search)
• Combined filters with implicit AND

Embedding Integration

• Workers AI: @cf/baai/bge-base-en-v1.5 (768 dimensions, free)
• OpenAI: text-embedding-3-small (1536), text-embedding-3-large (3072)
• Batch processing for multiple documents
• Automatic dimension matching

Quick Start

1. Create Index

# CRITICAL: Dimensions and metric CANNOT be changed later!
npx wrangler vectorize create my-index \
  --dimensions=768 \
  --metric=cosine

2. Create Metadata Indexes (BEFORE Inserting Vectors!)

npx wrangler vectorize create-metadata-index my-index \
  --property-name=category \
  --type=string

npx wrangler vectorize create-metadata-index my-index \
  --property-name=timestamp \
  --type=number

3. Configure Wrangler Binding

wrangler.jsonc:

{
  "vectorize": [
    {
      "binding": "VECTORIZE_INDEX",
      "index_name": "my-index"
    }
  ],
  "ai": {
    "binding": "AI"
  }
}

4. Use in Worker

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    // Generate embedding
    const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
      text: "Search query"
    });

    // Search vectors
    const results = await env.VECTORIZE_INDEX.query(
      embedding.data[0],
      {
        topK: 5,
        filter: { category: "docs" },
        returnMetadata: 'all'
      }
    );

    return Response.json(results);
  }
};

Templates

1. basic-search.ts

Simple semantic search with Workers AI embeddings. Perfect for documentation search, FAQ lookup, or product search.

2. rag-chat.ts

Complete RAG chatbot that retrieves relevant context and generates answers using LLM. Includes conversation history and source citations.

3. document-ingestion.ts

Document processing pipeline with text chunking, batch embedding generation, and metadata tagging. Handles large documents efficiently.

4. metadata-filtering.ts

Advanced filtering examples: range queries, nested metadata, multi-condition filters, namespace isolation for multi-tenant apps.

Reference Documentation

wrangler-commands.md

Complete CLI reference for all vectorize commands with flags and examples.

index-operations.md

Index creation, configuration, management. Covers dimensions, metrics, and when to use each.

vector-operations.md

Detailed guide for insert, upsert, query, delete, list, and get operations with examples.

metadata-guide.md

Metadata indexes, filtering operators, cardinality considerations, and performance optimization.

embedding-models.md

Configuration for Workers AI and OpenAI models, dimension requirements, and batch processing.

Examples

workers-ai-bge-base.md

Complete integration guide for Workers AI's @cf/baai/bge-base-en-v1.5 model (768 dimensions, cosine metric).

openai-embeddings.md

OpenAI text-embedding-3-small/large integration with API key management and error handling.

When NOT to Use This Skill

• Traditional database queries → Use D1
• Key-value lookups → Use KV
• File storage → Use R2
• Real-time collaborative state → Use Durable Objects
• Exact text matching → Use D1 with FTS or KV

When to Use This Skill

✅ Semantic search over documents, products, or content ✅ RAG chatbots with context retrieval ✅ Recommendation engines based on similarity ✅ Multi-tenant applications (namespaces) ✅ Classification and clustering ✅ Anomaly detection ✅ Duplicate content detection ✅ Image/audio similarity (with embeddings)

Version Information

• Skill Version: 1.0.0
• Status: Production Ready ✅
• Wrangler Required: 3.71.0+
• Workers AI Models: @cf/baai/bge-base-en-v1.5, @cf/meta/llama-3-8b-instruct
• OpenAI Models: text-embedding-3-small, text-embedding-3-large
• Vectorize API: V2 (GA)

Official Documentation

• Vectorize Docs
• Workers AI Embeddings
• RAG Tutorial
• Metadata Filtering

---

Author: Jezweb License: MIT Category: Cloudflare Infrastructure