Embeddings & vector search.

What an Embedding Is

Structurally, an embedding is just a list[float] — the Kotlin equivalent would be a DoubleArray. What makes it useful isn't the type, it's how it was produced: a model trained so that semantic closeness in meaning becomes geometric closeness in this list of numbers.

Generating Embeddings

Same request shape as Topic 2's chat completions, simpler payload: text in, a vector out. No message roles, no conversation — just an input and its coordinate.

from openai import OpenAI
client = OpenAI()

r = client.embeddings.create(
    model="text-embedding-3-small",
    input="How do I reset my password?",
)
vector = r.data[0].embedding
print(len(vector))  # 1536

from google import genai
client = genai.Client()

r = client.models.embed_content(
    model="gemini-embedding-001",
    contents="How do I reset my password?",
)
vector = r.embeddings[0].values
print(len(vector))  # 768 (default)

Measuring Similarity

Cosine similarity is the standard measure: it compares the angle between two vectors, ignoring their length, and returns a value from -1 (opposite meaning) to 1 (identical meaning). Most embedding models normalize vectors to unit length, which makes cosine similarity equivalent to a plain dot product — cheaper to compute.

import numpy as np

def cosine_similarity(a: list[float], b: list[float]) -> float:
    a, b = np.array(a), np.array(b)
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

password_reset = get_embedding("How do I reset my password?")
forgot_login = get_embedding("I forgot my login")
weather = get_embedding("What's the weather today?")

print(cosine_similarity(password_reset, forgot_login))  # ~0.7-0.9 — related
print(cosine_similarity(password_reset, weather))        # ~0.0-0.2 — unrelated

Brute-Force Search in Python

For a handful to a few thousand items, comparing a query against every stored vector directly is simple, correct, and fast enough — no database required. This is the entire mechanism a vector database automates and accelerates.

documents = [
    "Reset your password from the account settings page.",
    "Our refund policy allows returns within 30 days.",
    "The mobile app supports offline mode since version 4.2.",
]
doc_vectors = [get_embedding(doc) for doc in documents]  # do this once, cache it

def search(query: str, top_k: int = 2) -> list[tuple[str, float]]:
    query_vector = get_embedding(query)
    scored = [(doc, cosine_similarity(query_vector, vec))
              for doc, vec in zip(documents, doc_vectors)]
    scored.sort(key=lambda pair: pair[1], reverse=True)
    return scored[:top_k]

for doc, score in search("how do I get my money back"):
    print(f"{score:.3f}  {doc}")
# 0.71  Our refund policy allows returns within 30 days.
# 0.31  The mobile app supports offline mode since version 4.2.

Why Brute Force Stops Scaling

Section 3's search is O(n) — every query compares against every stored vector. Fine at a few thousand documents, unworkable at a few million: that's where Approximate Nearest Neighbor (ANN) indexes come in, trading a small amount of accuracy for a massive speed gain by not checking every vector.

Chunking Text Before Embedding

Embedding models have an input length limit, and — more importantly — a single vector represents a single "topic" poorly once the text covers many unrelated things. A 50-page document embedded as one vector produces a blurry average that matches nothing well. Split it into smaller pieces first.

def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap  # overlap keeps context from being cut mid-thought
    return chunks

Capstone: A Tiny Semantic Search Engine

Combine embedding generation, cosine similarity, and chunking into one reusable search function over a small set of documents — the exact shape Topic 7 wraps an LLM around.

import numpy as np
from openai import OpenAI

client = OpenAI()

def get_embedding(text: str) -> list[float]:
    r = client.embeddings.create(model="text-embedding-3-small", input=text)
    return r.data[0].embedding

def cosine_similarity(a: list[float], b: list[float]) -> float:
    a, b = np.array(a), np.array(b)
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

class TinySearchIndex:
    def __init__(self, documents: list[str]):
        self.documents = documents
        self.vectors = [get_embedding(doc) for doc in documents]

    def search(self, query: str, top_k: int = 3) -> list[tuple[str, float]]:
        query_vector = get_embedding(query)
        scored = [(doc, cosine_similarity(query_vector, vec))
                  for doc, vec in zip(self.documents, self.vectors)]
        scored.sort(key=lambda pair: pair[1], reverse=True)
        return scored[:top_k]


if __name__ == "__main__":
    index = TinySearchIndex([
        "Reset your password from the account settings page.",
        "Our refund policy allows returns within 30 days of purchase.",
        "The mobile app supports offline mode since version 4.2.",
        "You can export your data as a CSV from the settings menu.",
        "Support is available Monday through Friday, 9am to 6pm IST.",
    ])
    for doc, score in index.search("can I get a refund"):
        print(f"{score:.3f}  {doc}")