03. Structured Outputs & Schema Enforcement

Android Architecture Mental Model: In mobile platforms, raw HTTP strings are deserialized into typed Kotlin data classes using kotlinx.serialization or Moshi. If the backend drops a key or changes a type, your parser throws a JsonDataException and crashes the view. In generative AI, natural language responses are inherently non-deterministic. Structured output generation acts as a compile-time type checker and serializer for LLM token emissions, guaranteeing that output payloads conform strictly to an exact schema before hitting downstream business logic.

1. The Non-Deterministic Parsing Problem

Historically, developers instructed models via prompts: "Respond in JSON format without markdown ticks". This frequently broke down in production:

Models prepended conversational fluff: "Sure! Here is the JSON: {...}".
Models hallucinated invalid trailing commas or unescaped nested quotes.
Key names dynamically mutated across inference passes (e.g., swapping userId for user_id or id).

Modern production systems achieve 100% syntactic schema compliance through two distinct architectural tiers:

flowchart TD subgraph EngineTier["Tier 1: Constrained Decoding (Grammar Masking)"] A["Next Token Prediction Logits"] --> B["JSON Grammar Engine / Masking"] B -->|Mask illegal tokens with -inf logit| C["Select ONLY Valid Syntax Tokens"] C --> D["Guaranteed Valid JSON Syntax Token"] end subgraph AppTier["Tier 2: Application Type Hydration"] D --> E["FastAPI Pydantic Validator"] E -->|Schema Validated| F["Domain Model / Kotlin Client DTO"] E -->|Validation Error| G["Reflection & Self-Correction Turn"] end

2. Constrained Decoding vs. Prompted Schemas

Mechanism	How It Operates	Reliability	Latency / Compute Impact
Prompted JSON Mode	Instructs model via System Prompt to output JSON. Uses soft penalty if output isn't parsable.	~85–92% (prone to schema drifts and invalid types on edge inputs).	Zero engine overhead; normal token generation speed.
JSON Schema / Strict Mode	Inference engine builds a Context-Free Grammar (CFG) / Finite State Machine (FSM) from the schema.	100% syntactic compliance; zero syntax errors possible.	Minor initial compilation overhead to compute the grammar FSM for the first token.

3. End-to-End Implementation: Dual-Stack Schema Enforcement

from typing import List, Literal
from pydantic import BaseModel, Field
from openai import OpenAI

client = OpenAI()

# 1. Enforced Pydantic Schema Specification
class FeatureFlagEvaluation(BaseModel):
    feature_key: str = Field(description="The unique identifier for the feature")
    enabled: bool = Field(description="Target rollout state for the user")
    variant: Literal["A", "B", "CONTROL"] = Field(description="Bucket allocation")
    reasons: List[str] = Field(description="Telemetry explanation for decision")

# 2. Strict Engine Parsing
response = client.beta.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a backend configuration allocator."},
        {"role": "user", "content": "Evaluate rollout for user_12948 on android_checkout_v2"}
    ],
    response_format=FeatureFlagEvaluation,
    temperature=0.0
)

# Guaranteed typed instance matching schema exactly
evaluation: FeatureFlagEvaluation = response.choices[0].message.parsed
print(f"Key: {evaluation.feature_key}, Enabled: {evaluation.enabled}, Variant: {evaluation.variant}")

import kotlinx.serialization.Serializable
import kotlinx.serialization.SerialName
import kotlinx.serialization.json.Json

// 1. Mirroring the contract in Kotlin
@Serializable
data class FeatureFlagEvaluationDto(
    @SerialName("feature_key") val featureKey: String,
    @SerialName("enabled") val enabled: Boolean,
    @SerialName("variant") val variant: VariantType,
    @SerialName("reasons") val reasons: List<String>
)

@Serializable
enum class VariantType {
    @SerialName("A") A,
    @SerialName("B") B,
    @SerialName("CONTROL") CONTROL
}

// 2. Safe Parsing from API Gateway Response
val jsonParser = Json { 
    ignoreUnknownKeys = false 
    coerceInputValues = false 
}

fun hydrateEvaluation(rawJson: String): FeatureFlagEvaluationDto {
    // Fails fast if the schema violates strict types
    return jsonParser.decodeFromString<FeatureFlagEvaluationDto>(rawJson)
}

4. The Self-Correction & Repair Pipeline

While constrained decoding guarantees syntactic validity (e.g., braces match and keys exist), it cannot guarantee semantic validity (e.g., an age property parsed as -50, or a non-existent category code). When validation fails, a structured error reflection loop repairs the object.

sequenceDiagram autonumber participant App as App Engine participant LLM as Model Inference participant Val as Pydantic / Schema Engine App->>LLM: Ingest User Request + Schema Spec LLM-->>App: Emits JSON Payload App->>Val: Validate against Schema Types alt Validation Succeeded Val-->>App: Valid DTO Object Emitted else Validation Failed (e.g., String in Integer Field) Val-->>App: ValidationError: field 'count' must be int App->>LLM: Follow-up Turn: Previous JSON failed with [ValidationError]. Fix and re-emit. LLM-->>App: Corrected Compliant JSON App->>Val: Re-validate Val-->>App: Valid DTO Object Emitted end

5. Progressive Glossary

Term	Technical Definition	Android / Systems Analogy
JSON Schema	A declarative standard defining structure, required fields, and data types for JSON objects.	OpenAPI / Swagger spec or Protobuf `.proto` contract definitions.
Constrained Decoding	An algorithmic mask applied during inference preventing tokens that violate grammar from being selected.	Input filters on Android `EditText` (e.g., `DigitsKeyListener` blocking non-numeric input).
Pydantic	Python data validation and parsing library that enforces strict type hints at runtime.	`kotlinx.serialization` or Moshi parsing with custom `JsonAdapter` validation.
Grammar Masking	Zeroing out unallowed token logits by setting their score to $-\infty$ during sampling forward passes.	State-driven UI button enabling/disabling via Jetpack Compose State.

Sources & Reference Standards

OpenAI Engineering: Introducing Structured Outputs in the API
Pydantic Documentation: Data Validation and Settings Management for Python
Outlines & Guidance Libraries: Efficient Guided Generation with Formal Grammars

← Module 02: Prompt Engineering Proceed to Module 04: Embeddings & Vector Search →