04. Embeddings & Vector Search Foundations

Android Architecture Mental Model: Traditional mobile search relies on exact keyword indexing via SQLite / Room FTS4/FTS5 (Full-Text Search). If a user queries "flight delay," FTS matches literal string tokens; it completely misses documents saying "plane departure postponed." An embedding transforms an arbitrary text string into a fixed-length array of floats ($\mathbb{R}^N$), mapping concepts into coordinates in geometric space. Searching is no longer string matching—it is finding the $K$-nearest geometric neighbors using vector trigonometry.

1. The Embedding Space Pipeline

An embedding model (such as OpenAI text-embedding-3-small or Google text-embedding-004) maps varying sequence lengths into a deterministic, normalized coordinate array of fixed dimensions (e.g., 768 or 1536 floats).

flowchart LR A["Raw Document Chunk"] --> B["Tokenizer Engine"] B --> C["Transformer Encoder Layers"] C --> D["Dense Vector Float Array
[0.021, -0.093, 0.441, ... 1536 dims]"] D --> E[("Vector Storage / Memory Index")]

2. Mathematical Distance Metrics

To evaluate semantic similarity between a user query vector ($\mathbf{u}$) and an indexed document vector ($\mathbf{v}$), vector search engines calculate geometric distance:

Metric	Mathematical Definition	Computational Profile & Use Case
Cosine Similarity	$$\cos(\theta) = \frac{\mathbf{u} \cdot \mathbf{v}}{\\|\mathbf{u}\\|_2 \\|\mathbf{v}\\|_2} = \frac{\sum u_i v_i}{\sqrt{\sum u_i^2} \sqrt{\sum v_i^2}}$$	Evaluates angular alignment regardless of vector magnitude. Normalized vectors produce scores from -1.0 to 1.0 (1.0 = identical direction).
Dot Product (Inner Product)	$$\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^{N} u_i v_i$$	Cheapest computational profile. Identical to Cosine Similarity when embedding vectors are pre-normalized to unit length ($\\|\mathbf{u}\\| = 1$).
Euclidean Distance ($L_2$)	$$d(\mathbf{u}, \mathbf{v}) = \sqrt{\sum_{i=1}^{N} (u_i - v_i)^2}$$	Measures spatial distance between two vector tips. Lower is closer (0.0 = identical coordinates).

3. Text Chunking Architectures

Embedding an entire 100-page document into a single vector flattens granular information into an unusable average. Ingestion pipelines apply chunking strategies before computing vectors:

Strategy	Mechanism	Trade-off
Fixed Token Chunking	Slices text strictly every $N$ tokens (e.g., 512) with an overlap window (e.g., 64 tokens).	Simple and deterministic, but frequently cuts sentences or code blocks mid-thought.
Recursive Structural Chunking	Splits hierarchically by paragraph breaks (`\n\n`), lines (`\n`), then sentence boundaries.	Preserves complete semantic concepts and paragraphs within chunk bounds.
Document-Aware Chunking	Parses structural nodes (Markdown headers, Kotlin functions, JSON keys).	Highest retrieval relevance; requires custom parsers per file format.

4. Dual-Stack Implementations: Vector Similarity

Generating an embedding and executing a pure in-memory Cosine Similarity search over candidate vectors:

import numpy as np
from openai import OpenAI

client = OpenAI()

def compute_cosine_similarity(v1: list[float], v2: list[float]) -> float:
    a = np.array(v1)
    b = np.array(v2)
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

# 1. Generate Query & Document Embeddings
docs = [
    "Coroutines dispatch tasks across background threads using Dispatchers.IO.",
    "FastAPI provides asynchronous server endpoints using Python type hints.",
    "Jetpack Compose is a declarative UI toolkit for Android."
]

doc_res = client.embeddings.create(input=docs, model="text-embedding-3-small")
doc_vectors = [item.embedding for item in doc_res.data]

query = "How to run background threads on Android?"
query_vector = client.embeddings.create(input=[query], model="text-embedding-3-small").data[0].embedding

# 2. Rank by Cosine Similarity
scores = [compute_cosine_similarity(query_vector, dv) for dv in doc_vectors]
best_idx = int(np.argmax(scores))

print(f"Top Match (Score: {scores[best_idx]:.4f}): {docs[best_idx]}")

import kotlin.math.sqrt
import dev.langchain4j.model.openai.OpenAiEmbeddingModel

// Pure Kotlin Cosine Similarity without heavy native C dependencies
fun cosineSimilarity(v1: FloatArray, v2: FloatArray): Float {
    require(v1.size == v2.size) { "Vector dimensions must match" }
    var dotProduct = 0.0f
    var normA = 0.0f
    var normB = 0.0f

    for (i in v1.indices) {
        dotProduct += v1[i] * v2[i]
        normA += v1[i] * v1[i]
        normB += v2[i] * v2[i]
    }
    return dotProduct / (sqrt(normA) * sqrt(normB))
}

fun main() {
    val model = OpenAiEmbeddingModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("text-embedding-3-small")
        .build()

    val docs = listOf(
        "Coroutines dispatch tasks across background threads using Dispatchers.IO.",
        "FastAPI provides asynchronous server endpoints using Python type hints.",
        "Jetpack Compose is a declarative UI toolkit for Android."
    )

    // 1. Generate Embeddings via JVM Engine
    val docResponses = docs.map { model.embed(it).content().vector() }
    val queryVector = model.embed("How to run background threads on Android?").content().vector()

    // 2. Rank by Similarity
    val ranked = docs.indices.map { i ->
        docs[i] to cosineSimilarity(queryVector, docResponses[i])
    }.maxByOrNull { it.second }

    println("Top Match (Score: ${ranked?.second}): ${ranked?.first}")
}

5. Progressive Glossary

Term	Technical Definition	Mobile / Systems Analogy
Dense Vector	An array of continuous real numbers where almost all entries are non-zero.	Matrix transformation arrays or Canvas coordinate geometry arrays.
Dimensionality	The number of floating-point values in the embedding vector (e.g., 768, 1536).	Number of columns in an indexed database table schema.
Chunk Overlap	Number of shared boundary tokens between consecutive chunks to preserve sentence context.	Sliding window buffers or Paging 3 pre-fetch distance margins.
Approximate Nearest Neighbors (ANN)	Algorithmic heuristics (HNSW, ScaNN) that find top-k vectors in sub-linear time $O(\log N)$ instead of brute-force $O(N)$.	B-Tree index traversal replacing a full table scan in SQLite.

Sources & Reference Standards

Hugging Face: Dense Retrieval and Sentence Transformers Guide
OpenAI API Documentation: Vector Embeddings & Similarity Benchmarks
Malkov & Yashunin: Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World (HNSW) Graphs

← Module 03: Structured Outputs Proceed to Module 05: Production RAG →