Anthropic's Claude Engineering Masterclass: Prompting, Agent Architecture, Routines, and Memory (Full 4-Hour Session)

🎯 Two Scenarios & Why Evals Come First 0:00

The opening session reframes prompting as a debugging discipline, not a writing exercise. "We are rarely writing a prompt from scratch — we're often debugging an existing prompt." Margot frames the whole playbook around two scenarios every engineer hits: (1) maintaining a production prompt that's being migrated to a new model or architecture and has stopped working as well; and (2) building a brand-new agentic use case from zero to one.

The non-negotiable starting point for both is evaluations — "we need evaluations to provide that rigor, to understand whether a change to our prompt is actually correlating to an improvement." When you migrate models and performance drops, there are exactly two causes: either the new model is capable but behaving differently (fixable with prompting), or the new model is genuinely less capable (no amount of prompting will fix it). You can't tell which is which without an eval suite.

📋 The Meridian Mobile Test Suite 2:45

The worked example is a customer support bot for Meridian Mobile, a fictional telco — "a miniaturized example" representative of real prompts (no clear owner, multiple contributors, patches for old models mixed in, covering policy, tone, and process). The eval suite has just five test cases, but they're deliberately chosen to cover three key case types every suite needs:

The first eval run is a sobering baseline: only the control case passes, and the other four areas perform "pretty poorly." The rest of the session is systematically fixing those failure modes, one at a time.

🧹 General Hygiene: Structure & Role 6:15

Before targeting specific failures, apply "general prompting 101" hygiene. The sample prompt is a realistic mess: it tells the bot it's a human (untrue), contains text copied from a website (a stray "hero image" reference and a cookies clause), and lumps reasoning, role, policy, and tone into one giant undifferentiated paragraph.

The cleanup is structural: add XML tags to separate role, general guidelines, policy, and tone of voice, and rewrite the role description honestly. Just this cleanup — no behavioral changes — measurably improves the prepaid-plan scenario. The rule of thumb Margot returns to constantly:

📄 Output Contracts & Stop Sequences 11:00

The second hygiene step is an output contract — defining the format the model must return. For a conversational support bot this is low-stakes, but "if you're dealing with more complex output structures like nested JSONs," it becomes critical. The demo adds an XML output format to the prompt, then reinforces it at the harness level with a stop sequence that detects the closing XML tag and stops generation there.

The lesson is that the prompt isn't always the most reliable place to enforce consistency — the harness can do it more deterministically. For genuinely complex schemas, the recommendation is structured outputs, which enforce the format programmatically rather than hoping the model respects an instruction. This step doesn't visibly move the eval score (it wasn't the failure), but it's a best practice you should apply regardless.

🚫 Failure Mode 1: Withholding Information 13:24

The first targeted fix is the "hotspot" case: a customer asks how much hotspot data they have on an unlimited plan. The customer data clearly contains the answer (5 GB), but the model deflects — "since you're on a legacy plan, go check this yourself." Digging into the prompt reveals the culprit: an old instruction, "never give a customer the wrong plan details — instead point them to the URL," which reads exactly like a patch introduced for a previous, weaker model.

🔧 Failure Mode 2: Instructions Don't Add Capability 17:05

The proration case is the clearest single lesson of the whole session. The prompt already says "critical: always calculate prorated amounts correctly" — and the model still does sloppy mental math. The problem: "telling the model to do a good job isn't particularly helpful when we don't give the model the capability to actually do a good job."

The fix is to give the model a tool. Three steps: tell the model in the prompt to use a calculate_proration tool whenever it does calculations, define the tool schema in the API (what it does and when to use it), and implement the tool (the actual math). The eval flips to passing.

⚖️ Failure Mode 3: State Both Sides of the Trade-Off 20:00

The final failure is billing-error escalation. The model is supposed to escalate to a human, but instead it tries to diagnose the problem itself. The prompt says "avoid escalating unless absolutely necessary, as it costs ~$8 and counts against our fast-resolution metric" — which only gives the model one side of the story (the cost of escalating), so it over-fits to never escalating.

The fix is to give it both sides: escalating costs $8, but getting it wrong costs a refund and customer trust. The deeper point is about how models are evolving:

🏗️ Building From Scratch: The Scheduling Agent 23:00

The second scenario builds a new agent from zero: one that creates a week-long retail staff schedule for eight employees given availability and hard constraints. Because the rules are hard and checkable, the grader is a Python function that programmatically counts constraint violations — no LLM judge needed. This is the cleanest case for comparing approaches, because the eval is deterministic.

The hill-climb across model and prompt choices is the most instructive arc in the session:

The headline finding: a bigger model with adaptive thinking works, but a structured agentic loop on a smaller model beats it on cost and latency while still passing everything.

🔁 The Generate-Evaluate-Repair Loop 29:29

The winning architecture splits one monster prompt into three simple prompts running independently: a generator that drafts the schedule, an evaluator that reports specific violations (with evidence for every rule — an LLM checker, not a programmatic one), and a repairer that makes targeted fixes from the violation list.

The key benefit beyond efficiency is flexibility: because the evaluator is a prompt, you can inject soft constraints at runtime — "Harry doesn't like working with Sally, try to separate them" or "we need a third shift on Wednesday" — without touching the Python grader every time. The closing synthesis of the whole prompting playbook: general hygiene gives immediate uplift, evals make every change measurable, and isolating steps into separate prompt systems beats "one prompt to address everything."