Shelf · AI

AI (capped)

Capped on purpose. LLM APIs, RAG, evals, agents and FastAPI serving: enough to build and to judge, no hype.

8 chapters · 48 cards · 0 chapters written · #3 in your reading order

1

LLM mental model

Planned
Difficulty
Priority
  1. Tokens and the finite context budgetEasyP3Talk about it
  2. Embeddings are similarity representationsEasyP3Talk about it
  3. Temperature and top-p have model limitsMediumP4Talk about it
  4. Why plausible output can be falseEasyP2Talk about it
  5. Model families and workload trade-offsMediumP4Talk about it
  6. When not to use an LLMEasyP2Talk about it
2

Prompt engineering as engineering

Planned
Difficulty
Priority
  1. Prompt engineering with versions and evalsMediumP3Talk about it
  2. Roles, structure and delimitersEasyP3Know cold
  3. Few-shot examples with negative casesEasyP3Talk about it
  4. Chain-of-thought caution and checkable outputsMediumP5Know it exists
  5. JSON requests versus schema contractsMediumP3Know cold
  6. Injection basics and adversarial prompt evalsMediumP3Talk about it
3

LLM APIs in Python

Planned
Difficulty
Priority
  1. LLM API integration for Claude and OpenAIMediumP3Know cold
  2. Streaming as an owned request lifecycleMediumP3Know cold
  3. Tool and function calls are proposalsMediumP3Know cold
  4. Structured outputs at the Python boundaryMediumP3Know cold
  5. Retry, timeout and rate-limit budgetsHardP3Know cold
  6. Token accounting, cost control and prompt cachingMediumP4Talk about it
4

RAG that actually works

Planned
Difficulty
Priority
  1. Retrieval-augmented generation and chunk ownershipMediumP3Talk about it
  2. Embedding and vector-store selectionMediumP3Talk about it
  3. Hybrid search for names and meaningMediumP3Talk about it
  4. Rerank candidates before spending contextMediumP4Talk about it
  5. Citations that actually support a claimMediumP3Talk about it
  6. Diagnose retrieval separately from generationHardP4Talk about it
5

Agents & tools

Planned
Difficulty
Priority
  1. Agent tool loops with bounded recoveryHardP3Talk about it
  2. Planning and durable task stateMediumP3Talk about it
  3. Tool contracts and executable guardrailsMediumP3Know cold
  4. MCP protocol, capabilities and trustMediumP3Talk about it
  5. Human approval tied to the exact actionMediumP3Talk about it
  6. Prove when multi-agent is justifiedHardP4Know it exists
6

Evaluation & reliability

Planned
Difficulty
Priority
  1. LLM evaluation and golden datasetsMediumP4Talk about it
  2. LLM-as-judge without treating it as truthHardP4Talk about it
  3. Faithfulness, groundedness and correctnessMediumP4Talk about it
  4. Prompt and model regression gatesMediumP4Talk about it
  5. Offline versus online evaluationMediumP4Talk about it
  6. Trace observable LLM behavior safelyMediumP3Talk about it
7

Productionising an AI feature

Planned
Difficulty
Priority
  1. Architecture, async and queued AI workHardP3Talk about it
  2. Stream a UI without exposing partial actionsMediumP3Talk about it
  3. Cache answers without crossing permissionsHardP3Talk about it
  4. Prompt-injection defense and tool permissionsHardP3Know cold
  5. PII, privacy and data governanceHardP2Talk about it
  6. Cost ceilings, fallbacks and reversible rolloutHardP3Talk about it
8

AI-assisted engineering, honestly

Planned
Difficulty
Priority
  1. AI-assisted engineering without surrendering judgmentMediumP2Talk about it
  2. Spec-first workflow and acceptance evidenceEasyP2Know cold
  3. Review generated code as accountable engineeringMediumP2Know cold
  4. Where AI made work measurably slowerMediumP5Know it exists
  5. Preserve learning and measure real outcomesEasyP5Know it exists
  6. What the 2026 market pays for: evidence limitsEasyP5Know it exists