Topic 15 of 15 · Capstone

AI-assisted engineering workflows.

Every prior topic pointed an LLM at a user-facing product. This last one turns the same skills inward: use Claude, Gemini, or a Codex-style assistant to make your own engineering loop faster — PR summaries, generated architecture diagrams, test-case discovery, and debugging — while staying honest about what still needs a human reviewer.

The mental model

The reviewer's job doesn't disappear — it moves.

Every technique below produces a draft, not a decision: a draft summary, a draft diagram, a draft test list. Your job shifts from writing the first version to verifying the LLM's first version — the same shift code review already asks of you when reviewing a teammate's PR, just with a much faster, much less contextually-aware "teammate." Treat its output exactly that skeptically.

Section 0

Why This Is a Different Use Case

Topics 1-14This topic
Who consumes the outputAn end user of your productYou, or a teammate reviewing your PR
Cost of a wrong answerBounded by the product's guardrails (Topics 5, 8)Bounded by your own judgment before you hit merge
What "shipping" meansA deployed feature (Topic 13)A habit and maybe a small internal script
Everything here is lower-stakes to experiment with than a production feature — there's no end user to protect, just your own time to not waste. It's the right place to try ideas before deciding whether they're worth formalizing into the kind of product Topic 14 describes.
Section 1

PR Summaries That Are Actually Useful

A generic "summarize this diff" prompt produces generic output — restating what changed line by line, which a reviewer can already see in the diff. Topic 3's system-prompt discipline applies directly: be explicit about what a summary needs to add beyond the diff itself.

SYSTEM_PROMPT = """Summarize this pull request diff for a reviewer who hasn't
seen it. Don't restate the diff line by line — focus on:
1. What behavior actually changes from a user's perspective
2. Anything risky: new external calls, removed validation, changed error handling
3. What's NOT covered by tests in this diff, if anything

Keep it under 150 words. If the diff is purely mechanical (formatting,
renames), say so in one line instead of padding the summary."""

def summarize_pr(diff_text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[{"role": "system", "content": SYSTEM_PROMPT},
                  {"role": "user", "content": f"<diff>\n{diff_text}\n</diff>"}],
        temperature=0.2,
    )
    return response.choices[0].message.content
Section 2

Generating Architecture Diagrams

This is the exact pattern behind the CV line this whole series grew out of: use code changes to generate a Mermaid diagram of what actually changed structurally — new modules, new dependencies between them — so a reviewer gets a visual read on architectural impact instead of piecing it together from a file list.

DIAGRAM_PROMPT = """Given this diff, output ONLY a Mermaid flowchart (no
explanation, no markdown fences) showing the new or changed relationships
between modules/classes touched by this diff. Use short node labels.
If the diff doesn't meaningfully change structure (e.g. a bug fix inside
one function), output exactly: NO_STRUCTURAL_CHANGE"""

def generate_diagram(diff_text: str) -> str | None:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[{"role": "system", "content": DIAGRAM_PROMPT},
                  {"role": "user", "content": f"<diff>\n{diff_text}\n</diff>"}],
        temperature=0,
    )
    output = response.choices[0].message.content.strip()
    return None if output == "NO_STRUCTURAL_CHANGE" else output
flowchart LR
    Client -->|POST /extract-task| API[FastAPI service]
    API --> Extractor[Task extractor]
    Extractor -->|schema-constrained call| LLM[OpenAI]
    API --> DB[(pgvector store)]
Most Git hosting and IDE tooling renders Mermaid natively in a PR description or markdown preview — paste the generated block directly, no separate rendering step required. This one addition is what turned "read the diff" into "see the shape of the change" for the PR-review workflow this series' curriculum was built from.
Section 3

Test-Case Discovery

Reuse Topic 10's test-set mindset, aimed at your own code instead of a prompt: ask the assistant to propose edge cases for a function, then you decide which are worth writing — this is brainstorming, not delegation.

EDGE_CASE_PROMPT = """Given this function, list 5-8 edge cases a test suite
should cover, ranked by how likely they are to actually occur in production.
For each, state the input and the behavior you'd expect. Don't write the
test code — just the cases and expected behavior."""
Where this earns its keep: functions with non-obvious boundary conditions (off-by-one ranges, empty collections, concurrent access) — exactly the kind of case a tired reviewer skims past. It's weakest for business-logic correctness, where only you know the actual intended behavior.
Section 4

AI-Assisted Debugging

The highest-leverage debugging prompt isn't "fix this" — it's "here's the error, the relevant code, and what I've already ruled out; what haven't I considered?" This mirrors Topic 3's few-shot instinct: give the assistant the same context a senior engineer pairing with you would want before guessing.

DEBUG_PROMPT = """I'm seeing this error: {error}

Relevant code:
{code}

I've already ruled out: {ruled_out}

Don't repeat those. Suggest 2-3 specific things to check next, ranked by
likelihood, with a concrete way to verify each one."""
"Just fix it" promptsProduce a plausible-looking patch that may address the symptom without touching the root cause — exactly the failure mode Topic 10's evaluation discipline exists to catch in a product; here, it's your own judgment doing that job in real time.
Omitting what you've ruled outWithout that context, you'll get the same first three suggestions you already tried, wasting the round trip.
Section 5

Guardrails: Don't Trust, Verify

This entire series has argued for validating LLM output at every boundary — schemas (Topic 4), evaluation (Topic 10), injection defense (Topic 3). Applying that same skepticism to code and diagrams generated for your own workflow isn't optional just because the audience is now you.

OutputVerify before trusting
PR summaryRead the actual diff yourself before approving — the summary is a starting point for review, not a substitute for it
Generated diagramCheck it against the real module boundaries — a confidently wrong diagram is worse than no diagram
Suggested test casesConfirm each one is actually reachable and meaningful for your domain, not a generic list
Debugging suggestionsVerify the root cause before applying a fix — the same "confirm, don't guess" discipline as Topic 7's grounding
Section 6

Measuring Whether It Actually Helps

"Feels faster" is exactly the trap Topic 10 opened the series' evaluation topic warning against. If this is worth formalizing into a team workflow, measure it: PR review turnaround time before/after, or a simple survey of whether generated diagrams changed a reviewer's understanding. The CV line this topic mirrors — a 20% reduction in PR review friction — was a measured claim, not a felt one.

Section 7 · Checkpoint

Capstone: A PR Summary + Diagram CLI

A small command-line tool that reads a real git diff and produces both a review-focused summary and a Mermaid diagram when the change is structural — combining Sections 1 and 2 into one script you can actually run against your own repo.

import subprocess
from openai import OpenAI

client = OpenAI()

def get_diff(base: str = "main") -> str:
    result = subprocess.run(
        ["git", "diff", base, "--unified=3"],
        capture_output=True, text=True, check=True,
    )
    return result.stdout

def summarize_pr(diff_text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {"role": "system", "content":
                "Summarize this diff for a reviewer. Focus on behavior changes, "
                "risk (new external calls, removed validation), and test coverage "
                "gaps. Under 150 words. If purely mechanical, say so in one line."},
            {"role": "user", "content": f"<diff>\n{diff_text}\n</diff>"},
        ],
        temperature=0.2,
    )
    return response.choices[0].message.content

def generate_diagram(diff_text: str) -> str | None:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {"role": "system", "content":
                "Output ONLY a Mermaid flowchart (no explanation, no fences) "
                "showing new/changed relationships between modules touched by "
                "this diff. If nothing structural changed, output exactly: "
                "NO_STRUCTURAL_CHANGE"},
            {"role": "user", "content": f"<diff>\n{diff_text}\n</diff>"},
        ],
        temperature=0,
    )
    output = response.choices[0].message.content.strip()
    return None if output == "NO_STRUCTURAL_CHANGE" else output


if __name__ == "__main__":
    diff = get_diff()
    if not diff.strip():
        print("No changes against main.")
    else:
        print("## Summary\n")
        print(summarize_pr(diff))
        diagram = generate_diagram(diff)
        if diagram:
            print("\n## Architecture\n\n```mermaid")
            print(diagram)
            print("```")
Expected behavior: run from inside a git branch with real changes — prints a focused summary, and a Mermaid block only when the diff actually touches structural relationships between files, not for a one-line bug fix.
  • Ran the tool against a real branch in one of your own repos and judged whether the summary would have actually helped a reviewer
  • Ran it against a purely mechanical change (formatting, a rename) and confirmed the diagram step correctly outputs nothing
  • Deliberately fed it a large, multi-file diff and checked whether the summary still stayed useful and under 150 words, or started degrading — a live check of Topic 2's context-budget concerns
  • Read Section 5 again and, before trusting either output on a real PR, manually verified at least one summary or diagram against the actual diff
  • Next up

    Bonus: RAG on Android

    Take Topics 6, 7, 11, and 12 and put a Kotlin client in front of them — upload a document, ask questions, stream cited answers.