Track 1: Foundation & Inference Mechanics
01. LLM API Protocols & Token Mechanics
Tokenization, context window budget sizing, temperature, top-p/top-k sampling, and asynchronous SSE response streams.
Flow<String> payloads.02. Prompt & Context Engineering
Deterministic prompt templates, role separation (System/User/Assistant), few-shot conditioning, and chain-of-thought prompting.
03. Structured Outputs & Schema Enforcement
Validating LLM outputs with Pydantic, JSON schemas, constrained grammar decoding, and self-correction loops.
04. Embeddings & Vector Search Foundations
High-dimensional semantic vectors, cosine similarity, Euclidean distance, chunking strategies, and token overlap.
Track 2: Retrieval & Relational Vectors
05. Production RAG Architecture
End-to-end ingestion, bi-encoder retrieval vs. cross-encoder reranking, and dynamic prompt envelope assembly.
06. Vector Databases: PostgreSQL + pgvector
Schema setup, HNSW vs. IVFFlat indexing trade-offs, distance operators, and hybrid metadata pre-filtering.
Track 3: Agentic Workflows & Serving Systems
07. Function Calling & Tool Execution
JSON schema tool declarations, multi-turn tool loops, sandboxed execution, and parallel tool dispatching.
08. Agentic Workflows & State Machines (LangGraph)
Cyclic agent graphs, state checkpointing, human-in-the-loop approvals, and conditional state reduction.
09. Asynchronous AI Serving with FastAPI & Cloud Run
ASGI event loops, SSE streaming protocols, client socket disconnect handling, and Cloud Run scaling.
Dispatchers.IO.
10. AI Evaluation & System Observability
The RAG triad (Faithfulness, Relevance, Precision), latency anatomy (TTFT vs. ITL), and LLM-as-a-Judge test gates.
Track 4: Edge AI & Systems Architecture
11. Edge AI & On-Device Small Language Models
Running quantized SLMs (Gemma 2B / Llama 3.2) locally via LiteRT & MediaPipe, NPU acceleration, and hybrid edge-cloud routing.
12. AI System Design, Semantic Caching & Cost Architecture
Semantic caching with vector distance thresholds, prompt KV-cache reuse, token bucket rate limiters, and PEFT/LoRA vs. RAG matrices.
Progressive Engineering Glossary
| AI Concept | System Definition | Mobile / Systems Equivalency |
|---|---|---|
| Token | Sub-word text chunks used by LLMs to compute probabilities (~0.75 words). | Byte buffer allocation or payload serialization packets. |
| Embedding | A dense vector representation capturing semantic meaning in N-dimensional space. | A deterministic hash code mapping semantic similarity rather than collision avoidance. |
| RAG | Retrieval-Augmented Generation: dynamic grounding of LLM prompts via external search. | Local database lookup before executing a remote backend fetch. |
| Quantization (INT4 / INT8) | Compressing model weights from FP32/FP16 floats to 4-bit/8-bit integers to fit in edge RAM. | Image bitmap subsampling (inSampleSize) and Proguard dead-code shrinking. |
| Semantic Cache | Caching query-response pairs keyed by vector cosine similarity rather than exact string hashing. | OkHttp cache interceptor with fuzzy semantic key evaluation. |
| Stateful Agent | An LLM looped within an iterative graph that mutates state across intermediate turns. | A reactive MVI State Machine managing UI and side-effects. |
Primary Sources & Standards
- Google LiteRT & MediaPipe: Edge AI Model Execution on Android
- Anthropic Protocol Guides: Prompt Engineering and Prompt Caching Specs
- Google DeepMind / Vertex AI Architecture Whitepapers: Gemini Technical Reports
- PostgreSQL pgvector Documentation: HNSW and IVFFlat Indexing Mechanics