Deep dives into working agents.

Technical walkthroughs with substance: architectures, prompting patterns, evals, and production lessons unpacked until complex agent systems become reusable design material.

Page 1 of 4 · showing 1–24 of 78

Show the outdated contentHistory

Reverse Engineer Anything: 5 Use Cases for GitHub's #1 Trending Skill

AI LABS walks through five genuine use cases for REA (Reverse Engineer Anything, morluto/rea — 274,081 stars), the MCP server that lets Claude Code and Codex reverse-engineer apps and websites by tracing clicks back to the exact code and labeling every finding seen/guessed/unknown. The five: copy a feature from another app (Obsidian graph + Notion board), research all competitors at once, find bugs in your own app (a pricing bug that code review would have missed), run a site-wide performance checkup, and audit security before shipping (exposed OpenAI/Stripe keys). Firecrawl is the sponsor.

deep-dives

AI LABS

Published2026-10-16

Pi Durable: Proactive, Multi-Client Agents That Never Stop

IndyDevDan's 35-minute walkthrough of Pi Durable, the durable-agent harness shipped alongside Pi 1.0 by Mario Zechner and the Earendil team. Pi Durable makes agents persistent across failures, clients, and change: background compaction, a shared application-state object readable by every session on a device, phased durable jobs that resume from any step, detached agents any client (TUI, browser, CLI, curl, mobile) can attach to via a device→workspace→agent hierarchy, live tool changes mid-execution (read-only mode, Jev damage control blocking rm -rf), forking, non-replayable one-shot operations, and long-running agents that survive crashes and restarts. The demo: a Jev classifier watches an event stream, detects an incident, and triggers a proactive Pi Durable agent that investigates, fixes a null-check bug, and deploys — all for ~1.5¢. Verified: Pi 1.0 + Pi Durable shipped together (Earendil announcements), earendil-works/pi (114,638★, MIT, TypeScript), and @earendil-works/pi-durable (npm v1.2.0, MIT). Pi Durable is explicitly experimental; 'proactive agents are the future' is the speaker's opinion, and he promotes his own courses/products.

deep-dives

IndyDevDan

Published2026-10-13

Turn Claude Code Into a Software Factory: The Four-Pillar Orchestration Layer

Patrick Ellis (CTO of an AI-native startup) argues the 10x-100x leverage isn't coming from smarter models but from the orchestration layer around them — and that Anthropic and OpenAI are both converging on long-horizon autonomous work (Greg Brockman's Codex vision: tasks that run in the cloud for hours, days, even weeks). His framework has four pillars: context (company/product goals, design language, adjacent codebases, APIs), tools (grep, Playwright MCP, gh CLI, Sentry, Terraform, gcloud), workflows/SOPs (engineering/design/security reviews — his favorite being gstack, Garry Tan's YC skill library with /office-hours, /engineering-review, and an adversarial review that invokes a CLI Codex from Claude Code and vice-versa), and verification (Playwright screenshots vs the UI mock plus deterministic linters/unit tests/dead-code detection). Two tools worth the look: gstack (135k stars, MIT) and Steve Yegge's Beads (27.8k stars) — a dependency-aware task graph that keeps long-running agents from hallucinating completed checkmarks. The proof case is an analytics platform (Pub/Sub, Cloud Run, BigQuery) shipped in a week, with a six-hour overnight Codex run and a human-review gate on core infra changes. The strategic takeaway: document and aggregate your company's context into one orchestration layer, because models will get better but they'll never know your specific workflows unless you write them down. Ellis's startup pitch and custom skills are flagged as self-promotion; gstack and Beads were verified live via the GitHub API.

deep-dives

Patrick Ellis

Published2026-10-10

I Built an AI Software Factory: Parallel Agents That Actually Work

A software factory, concretely defined: an operating loop where issues come in from the backlog, a coding agent writes the fix, an independent system verifies it, and a human merges. Prompt Engineering's build enumerates the five ways parallel agents break — shared-folder collisions (one commit touching six files), double-booked issues (fix: a claim label), the self-grading trap (an agent always reports "tests pass"), stale training cutoffs (fix: live docs), and PR flood (fix: merge stays human) — then runs a five-stage pipeline (triage → worker → verifier → reviewer → integrator) where each stage is a fresh cloud sandbox booted from a shared snapshot. The two architectural details worth stealing: typed object handoff (stages pass structured JSON, not chat history) and live-docs-via-proxy-key-injection (the agent reads docs but never sees the API key). Two hard-won findings: passing tests does not mean the code is right (a reviewer caught a same-millisecond ordering bug every test missed), and parallel agents that share a test file produce merge conflicts (7 of 9 PRs) — hence the integrator stage. Result: 10 issues in 4.5 minutes, 9 hidden-test-passing PRs plus one vague ticket correctly skipped, for ~$3.67 in tokens and ~$0.01 in sandbox time. Upstash Box (and Context7) are the video's paid sponsors — flagged; the five-stage architecture is the vendor-neutral takeaway. Source code is published at github.com/PromtEngineer/software-factory.

deep-dives

Prompt Engineering

Published2026-10-09

Claude Dashboards: Live BI and Observability for Your Business

Anthropic launched Claude Dashboards (with Claude Motion) in public beta on Oct 8, 2026: connect a data warehouse or connectors, ask a plain-language question, and Claude writes the queries and builds a live, self-updating dashboard. The core distinction from artifacts — artifacts were static HTML snapshots; dashboards query live from a database. Mansel Scheffel's walkthrough covers both headline use cases: business intelligence (where's your cash, what's overdue, where you're burning money) and observability into your own Claude/AI infrastructure via OpenTelemetry. His recommended pattern is a central Supabase middle layer as the single source of truth — connector (MCP) plus plugin (schema/security best-practice skills), with an edge function ingesting metrics — and scheduled tasks to keep the data fresh, since a dashboard only knows what's been pulled into the database. The interactive dashboard adds a comment/chat function that explains any row (what it is, why it's the biggest problem, the fix) and lets you inspect the underlying SQL. Beta glitches are noted (queries reporting success while empty). The video's "business rescue" skill (manifest/evidence/forecast/rescue/report) and his community are flagged as self-promotion. Feature launch and Supabase details verified against Anthropic's docs and help center.

deep-dives

Mansel Scheffel

Published2026-10-09

From Vibes to Production: Evaluating and Shipping AI Agents That Work

Laurie Voss (Head of Developer Relations at Arize AI, co-founder of npm) runs a two-hour AI Engineer workshop on the most under-served part of agent development: proving your agent actually works. The core thesis: evals are testing for AI, traces are logging for AI, and the most popular shipping method today — run a few queries, eyeball the output, ship — is the vibes problem. The talk walks the full arc: three kinds of evals (deterministic code evals, LLM-as-judge with a rubric, human review plus meta-evaluation); why agents are harder (cascading failures like the Tesla-vs-Tesla mixup, the τ²-bench 'creatively correct' flight-reschedule); reading traces before writing evals (open vs axial coding, severity × frequency, the Swiss cheese model); the two-line OpenTelemetry/OpenInference instrumentation; a correctness eval that scored 0/13 because the judge's training data stopped in January while the agent reported live 2026 data; a faithfulness eval revealing 7-of-13 reports hallucinated; the four-part custom rubric and its anti-patterns (the god evaluator, treating evals like code); meta-evaluation with golden data sets and judge biases (position, length, confidence, self-preference); and closing the loop by handing judge explanations to Claude Code to rewrite prompts and verifying with experiments. It's vendor-agnostic theory plus an Arize AX walkthrough, flagged as such. Speaker identity and tooling are verified; the demo's numbers are the talk's own.

deep-dives

Laurie Voss

Published2026-10-07

Hidden Claude Code Settings That Change How You Work

A curated tour of the Claude Code settings, commands, and shortcuts most daily users never touch, grouped by what they solve. Setup that learns how you work: an interactive /init mode (ask questions, then write a CLAUDE.md that reflects your habits plus starter skills and hooks) and /import to migrate your Codex or Cursor setup. Keeping setup fresh: a prompt-audit command that inspects your whole config with subagents and reports outdated instructions, plus skill-doctor to find unused skills by usage and token cost. Context and permission tuning: disableClaudeAIConnectors, enableArtifacts:false, auto-mode setup, and a fewer-permission-prompts allowlist. Evals: /claude-api build-eval and hillclimb (Anthropic's claude-api skill, added Sept 28 2026). Plan and output: showClearContextOnPlanAccept, plansDirectory, /output-style (five styles), showThinkingSummaries. Multi-session: focus mode, /branch, /fork, /subtask, list-agents (sessions messaging each other), askUserQuestionTimeout, local notifications. Model routing: opus-plan and CLAUDE_CODE_SUBAGENT_MODEL(_FORCE). Plus prompt-editing shortcuts (Ctrl+S/Ctrl+Y/Ctrl+_) and /powerup. Settings keys and the eval workflows are verified against the official docs; the sponsor segment is noted and omitted.

deep-dives

AI LABS

Published2026-10-07

DeepSeek Harness Just Got a Web Agent

DeepSeek Harness (dsh), one of the fastest-growing open-source agentic coding harnesses, ships a desktop version — and the real story is the architecture underneath. Built on Cordis's 'everything is a plugin' runtime, the harness is just a host: model routing, web search, the terminal, and the whole agentic loop are plugin packages wired through a per-profile patch file. The video walks through the UI (full agent trajectories, per-tool-call token counts, and a cache-hit-rate readout no other harness shows) and the four modes, then demonstrates the general technique for extending it: an MCP client plugin pointing at TinyFish's MCP server to add a full web agent (search, browser-rendered fetch, a Mako-driven web agent, and a Playwright cloud browser) on top of the harness's read-only built-in web tools. The DeepSeek Harness side is verified against the official repo; the TinyFish half is a sponsored partnership and is flagged as such, with sponsor links omitted. The Hugging Face ≤30B demo — where a 29B-named model is really 31B and a 27B-named model is really 7B — is a genuinely useful lesson in making web-agent prompts explicit.

deep-dives

Prompt Engineering

Published2026-10-06

Context Language Models: Your Agent Doesn't Need Compaction

A new paper from Meta and the University of Washington flips context management on its head. Instead of having the harness summarize at a fixed threshold — which throws away whatever didn't look important at the time — a Context Language Model treats its context as an editable file and rewrites it itself each turn: keep, shrink, delete, or rewrite anything. The video walks the paper (arXiv 2609.37725), demos the Pi extension, and runs its own tests: Pi's built-in summarization hallucinated 14 of 18 seal codes, while CLM got all 18 exact. It also catalogues the gotchas — parallel tool calls defeat self-editing, prefix-cache placement drives cost, it's not automatically cheaper, and a 9B model needed RL to beat summaries. Plus the security caveat the paper itself raises: if the model can edit its memory, so can a prompt injection.

deep-dives

Prompt Engineering

Published2026-10-06

Skills v1.3: Matt Pocock's /pr, /implement-spec, and /retro

Matt Pocock ships version 1.3 of his skills repo — the most-starred agent-skills collection on GitHub — with three additions central to his workflow. implement-spec turns a spec and its tickets into one PR by orchestrating sub-agents that can now spawn sub-agents. PR is a body template that borrows Dex Horthy's Show Me skill to make human review cheap, using before/after evidence, the one-way vs two-way door merge-danger test, and blast radius. retro reads real coding-agent sessions afterward and mercilessly lists what to fix — missing CI, context loss between compactions, token waste, even the release Claude cut without asking. The one rule: keep retro human-in-the-loop, or it loops on its own false positives.

deep-dives

Matt Pocock

Published2026-10-05

Why Software Factories Fail: Harness Engineering Is Not Enough

Dex Horthy (HumanLayer, author of 12-factor-agents) argues that the lights-out software factory — where nobody reads the code and you just route everything into the agent queue — does not work, and that the reason is not a skill issue or a harness problem but a model-training problem. Coding models are reinforced on a single binary signal: did the test pass without breaking another. Nothing in that reward penalizes bad architecture, whose cost only shows up months later, so models get better at passing tests and no better at keeping a codebase maintainable. That, he argues, is why Claude Code went from nothing to billions while tools with the same read/write/edit commands did not: it was the first model trained against the harness it ships in. His fix is to turn the lights back on and plan up front — product review, system architecture, the underrated step of program design down to types and call graphs, then vertical slices — so that thirty minutes of alignment saves hours of review and a good PR becomes a joy to read instead of slop to untangle.

deep-dives

AI Engineer

Published2026-10-05

OpenRouter State of Models: Jev, Open-Weights, and Tokenomics

IndyDevDan uses OpenRouter's public data to break down the state of LLMs entering Q4 2026, through five trends every engineer should understand. Trend one: token usage is climbing nearly exponentially — roughly 146 trillion tokens last week versus 4.5 trillion a year ago — and that growth is the strongest argument against the AI-bubble narrative, even as Anthropic targets a $2 trillion IPO while losing nearly $50 billion. Trend two: DeepSeek, OpenAI, and Google hold the top three OpenRouter market-share spots all year, but napkin math on Anthropic's direct enterprise traffic suggests it may be processing 1.5x OpenRouter's entire weekly volume. Trend three: Jev by TypeSafe brings the zero-shot classifier back as a new species of decision model, already 12th on OpenRouter on a fraction of the tokens — and not trivially cloneable. Trend four: free stealth models like SpaceBunny Alpha come with the 'if it's free, you're the product' catch. Trend five: the Pi agent is steadily closing on Claude Code in OpenRouter usage, a signal that engineers want to own their agent harness rather than rent it. The through-line is tokenomics: combine compute, don't select it — throw state-of-the-art models at new problems, then scale down to cheaper models once the work repeats.

deep-dives

IndyDevDan

Published2026-10-05

Claude Mods: How They Work and Why They're a Big Deal

Claude Code mods are JavaScript or TypeScript functions that run inside Claude Code itself, not outside it like hooks, skills, and MCP servers. A mod is loaded once and stays alive for the whole session, so it can watch, change, or stop a tool call before it runs and draw its own live panels in the terminal and the desktop app. When Claude is about to act, the mod receives the event, a way to talk back, and a next function — it can let the action through, rewrite it, or answer instead. Because Anthropic is already building parts of Claude Code itself as mods, the diff view and the agent are mods too, the whole experience is now customizable. The video shows hot reload, installing a mod from a marketplace folder or repo, a guard that stops rm -rf, and a session-audit prompt that mines your last 20 sessions for the mods you actually need. The caveat is security: a mod runs on your machine with your permissions, so validate it with claude plugin validate before installing.

deep-dives

Prompt Engineering

Published2026-10-03

Loop Engineering: Getting Claude Code to Build on Its Own

Loop engineering is the structured workflow behind agents that build on their own for hours: an agent makes a change, a separate check scores it, and the change is kept only if the score improves. AI LABS rebuilt Karpathy's AutoResearch loop as a Claude Code workflow and found its one blind spot. The setup is a stack of skills and agents — a project context skill as the app's memory, a build skill that drives the loop, a write checks skill that writes checks before any feature is built, an approve checks program that locks the checks where the agent cannot edit them, a fresh feature builder agent per feature, and a results file recording every round. The blind spot is habits: a fresh agent with the same instructions repeats the same mistakes, so they add a second loop that reads the results and rewrites the how-to-work instructions for the next run. It caught what scores missed — a database that passed all 10 checks while saving nothing, and a mentions feature that clashed with legacy code.

deep-dives

AI LABS

Published2026-10-03

Fixing the PR Bottleneck: Three Brakes for Agent-Generated Code

Matt Pocock's talk from AI Engineer Paris 2026 argues that the pull request is still the bottleneck — and agents have made it worse. Agents make it trivial to open a PR and only slightly easier to open one worth reviewing, so as software factories accelerate, you get a 'slop cannon' unless you add brakes. His answer is a three-layer cake: cheap deterministic automated checks, then automated review by a separate sub-agent, then a human pass that only looks at the one-way-door PRs. The talk is dense with concrete mechanics — tautological tests and over-mocked tests that cannot fail, deep modules from Ousterhout's design philosophy, why coding standards must live in a separate coding-standards.md rather than the implementer's context, why a review agent should commit fixes instead of commenting, and a retro skill that turns every human review into better automated checks for the next PR. The through-line: code is the environment your agent works in, and agent experience (AX) deserves the same rigor developer experience got.

deep-dives

Matt Pocock

Published2026-09-30

10 Levels of Jev: Where the System-One Model Fits in Your Agentic Stack

IndyDevDan lays out ten concrete levels of using Jev, TypeSafe's System One decision model, inside real agentic engineering — from a smart if-statement and support triage, through bash-tool gating and in-agent guardrails, all the way to a Pi coding agent that reaches for Jev on its own. The through-line is the reframe: Jev is not an LLM and not a replacement for one — it's a third class of model for narrow, programmable decisions, and the win is speed and cost. One Jev call is roughly 600x cheaper than a frontier model, which turns prompt-injection checks, force-push gates, and repo-wide file scouting from 'too expensive to run at scale' into a fraction of a cent. The takeaway for agentic engineers: stop reaching for the biggest model by default — define the decision, pick the right tool, and inspect the confidence.

deep-dives

IndyDevDan

Published2026-09-29

Mine Your Coding Agent's Conversation History: From JSONL Goldmine to Better Rules, Hooks, and Skills

Every conversation with your coding agent is already saved on your machine as JSONL files — every prompt, tool call, and failure. Cole Medin shows how to turn that overlooked goldmine into concrete improvements. The simple version is one prompt that has the agent scan its own history and list the top 10 opportunities to become more efficient or reliable. The full version loads the transcripts into structured tables (sessions, turns, tool calls) in Databricks, using the built-in Genie agent to parse the messy nested schemas with Spark — so querying becomes token-efficient instead of reading hundreds of thousands of tokens per analysis. The result: real changes to his AI layer, including 38 lines of new global rules ('never guess a path'), better Git permissions, and a session-tree hook that injects the live repo layout into every new conversation.

deep-dives

Cole Medin

Published2026-09-14

Needle: The Free, On-Device LLM You Can Actually Build With

Andrew Warner interviews Cactus Compute's Henry Ndubuaku about Needle — a free, 8–29MB on-device LLM that turns natural language into structured tool calls without an internet connection or external API. The core message is reliability through restriction: scope Needle's tools tightly, enumerate the options, and gate execution on its confidence score, and it 'virtually doesn't make mistakes' — because a limited model can't suddenly become your therapist, control your computer, or write code it wasn't asked to. It integrates into apps via llm.txt (paste it into Claude Code), runs on hardware like robot vacuums, phones, and even AirPods, and — unlike Jev — is open source and built to embed in your own software. The verdict: same safety ethos as Jev, different market (on-device vs cloud), and 'better because you can build with it.'

deep-dives

The Next New Thing

Published2026-09-14

A Model Router with Jev & OpenJevs: Routing, PII Gating, and Difficulty Scoring in Milliseconds

Sam Witteveen builds a local model router with Jev (TypeSafe's decision model) and swaps in SemIf — an open, local clone — so the whole router runs offline. The router asks three questions in one parallel request: choice (which lane — chitchat, code, image), score (how hard is it — can the local 2B model handle it or escalate), and null (is there PII — force local). A Next.js UI talks to a FastAPI server that decides, routes, and streams back from the winner, logging everything to SQLite. Results: 75% of requests answered locally, a fake API key caught at 0.96 confidence, decisions in 331ms (cloud Jev) or 88ms (local SemIf), and the whole day's demos costing under a cent.

deep-dives

Sam Witteveen

Published2026-09-14

Self-Compacting Agents: Let the Agent Decide When to Compact Its Own Context

IndyDevDan builds a self-compacting Pi coding agent — a context-aware agent with its own tool to decide when to compact, what to preserve, and how to continue. Instead of accepting the default fixed-percentage auto-compaction in Claude Code and Codex, the agent gets three thresholds (notice, warning, force) tuned around the 270K point where GPT pricing doubles, a live context bar, and full control over the compaction prompts — something Pi allows and Claude Code doesn't. Then the same hand-written plan is handed to three harnesses: GLM 5.2 chokes at 98% context, Fable 5.1 (Claude Code) ships in 50 minutes on 500K tokens, and GPT-6 Astra (Codex) ships in 21 minutes on 136K. The core lesson: master context, model, prompt, and tools — and you master the agent.

deep-dives

IndyDevDan

Published2026-09-14

Jev: The Decision Model That's 100× Faster Than an LLM — Three Real Prototypes

Jev, from TypeSafe, is a different kind of model: it doesn't write text — it classifies, scores, ranks, and routes, returning calibrated probabilities instead of sentences. Moritz's tutorial shows what that unlocks with three working prototypes: a voice-controlled browser where Jev answers a batch of multiple-choice questions about each spoken fragment in a few hundred milliseconds (letting plain code act on the probabilities), an AI memory-retrieval system that cut token usage 80–98% by making Jev the judgment engine while code does the counting and chunking, and a YouTube topic scorer that featurizes 600 videos and ranks candidate titles. The honest trade-off: Jev gives up writing, explanation, and step-by-step reasoning for ~100× speed — so the winning pattern is Jev deciding and an LLM reasoning when deeper intelligence is needed.

deep-dives

Moritz | AI Systems

Published2026-09-14

The 5 Benchmarks That Beat the Index: Ranking GPT-6 Astra, Claude Fable 5.1, and Open-Weights

The Artificial Analysis Index is a proxy of a proxy — mash ten benchmarks into one number and you lose the only thing that matters: which model to run for YOUR work. IndyDevDan picks his top five agentic benchmarks (Terminal-Bench v4.0, APEX Agents, AutomationBench, AA-Omniscience, DeepSWE v1.1) and ranks GPT-6 Astra, Claude Fable 5.1, and the open-weights field against each. The throughline: model selection is a three-dimensional problem (performance, cost, speed), Astra wins on cost-per-task by roughly 4x over Fable, Gemini 3.8 Flash owns the workhorse cost sweet spot, and saturated benchmarks — flat lines — are dead weight. The move is a model stack, not a model.

deep-dives

IndyDevDan

Published2026-09-14

Pstack Is Agent Overkill. Use It Anyway!

Cursor engineer Lauren Tan (poteto) used her personal skill stack 10,000 times in one week inside Cursor's engineering team, then open-sourced it as pstack. Rob Shocks breaks down the /poteto-mode router and its 23 playbooks, the verification-first skills (create/maintain verification, interrogate), and the 20+ engineering principles — laziness protocol, redesign-from-first-principles, minimize-reader-load — that turn AI agents from slop machines into rigorous engineers. The verdict: it's overkill, until quality is the constraint.

deep-dives

Rob Shocks

Published2026-09-11

Tencent's Free Tool Syncs Your Agent Skills Across Your Whole Team

TeamAI (teamai-cli) configures 11 AI coding agents — Claude Code, Codex, Cursor, OpenCode, OpenClaw, Hermes, DeepSeek Harness, Coder, Zcode, CodeBuddy, WorkBuddy — all pulling skills, rules, hooks, and MCP servers from one shared Git repo. The answer to config drift at scale: 'put it in git.' The sync loop is four arrows: teamai push → merge request → reviewer merges → a SessionStart hook runs teamai pull, deterministically, outside the model's control loop — the model can't forget to sync. Repo layout includes culture.md (team values injected into every session), roles/tags/subscriptions (a 'package registry for institutional knowledge'), and a friction-scoring Stop hook that turns interruptions, denied tool calls, and failed retries into team learnings — 'high friction is high information.' Honest gaps: Team Context and Improvement are beta, Models not implemented, 5 of 11 agents fully supported. The irony: we didn't escape process — we taught it to the machines.

deep-dives

Signal Coders

Published2026-09-09