Don't Build More AI Agents Until You Watch This

Don't Build More AI Agents Until You Watch This

Nate B Jones · ~18 min · Deep Dive Document
Video thumbnail — Don't Build More AI Agents Until You Watch This
⏱ ~18 min 🎤 Nate B Jones 🏷 Agent Maintenance · Harness Design · Vercel · Codex · Claude Code

Overview

Nate B Jones makes the case that agent maintenance — not agent building — is the defining skill of 2026. Using Vercel's sales agent as the anchor story (they improved their agent by deleting 80% of its tools), he lays out four durable principles: models improve and can break harnesses that were designed for weaker versions; agents inherit all the organizational drift around them; frontier labs like OpenAI and Anthropic already invest more in harness maintenance than raw model capability; and every AI user — from engineers to writers to product leaders — already has a harness, whether they realize it or not. The video closes with a five-point maintenance checklist and a strong recommendation for Stewart Brand's The Maintenance of Everything as essential reading for anyone working with agents.

1 Vercel's Counterintuitive Lesson

▶ 0:00

Vercel made its sales agent better by deleting 80% of its tools. Not by adding more context, more integrations, or more memory — by removing them. And this wasn't a context-window optimization; it was a deliberate harness decision.

The backstory: Vercel's sales inbox was the usual mess — real leads, spam, support questions dressed as sales questions, small companies, big accounts, messages needing quick replies, others needing research, others needing routing. They did something smart:

  • Studied one of their best reps — watched the actual workflow closely enough to turn pieces of it into an agent
  • Mapped the real decisions — what the rep ignored, what they answered, what made a lead real, what research happened before a reply, when a message was actually a support issue
  • Built around the observed workflow — not the paper workflow, not the idealized process, but what actually happened

The resulting agent filtered inbound messages, qualified leads, researched companies, drafted responses, and routed support questions away from sales. A human still reviewed everything — the goal wasn't autonomous operation, it was making a repeatable workflow from a strong employee run fast.

💡 "The agent did not get better when the team kept piling on tools. It got better when they took away tools." — This is the sentence the entire video orbits around.

2 The Beginner Instinct vs. The Maintenance Instinct

▶ 1:57

Most people build agents in the wrong direction. The pattern is instantly recognizable:

  • Start with one task → add a tool → add another tool → add a memory file → add a Slack integration → add a browser → add a CRM action → add another exception
  • After a while, the agent looks powerful and muscled up, but it becomes harder to trust

Nate frames this as two competing instincts:

  • The beginner instinct — to add. More tools, more skills, more integrations, more access
  • The maintenance instinct — to ask what should be removed

This applies equally to skills in Codex or Claude — if you've got a pile of accumulated skills, the maintenance question is which ones are helping and which ones are creating confusion.

The harness defined

Nate introduces the central concept: the harness (or "workbench"). The agent is the worker; the harness is the workbench. Specifically:

  • What the agent reads
  • What it remembers
  • What tools it can touch
  • What it's allowed to change
  • What proof it has to bring back
  • What stops it when work gets risky
💡 "The real agent story of 2026 is not 'can you build an agent?' — of course you can build an agent. The harder question is whether you can keep the setup around the agent healthy as the work changes and the model evolves."

3 Principle 1 — Models Improve, Agents Break

▶ 4:34

The model underneath the agent is not stable. It's getting better at tool use, reasoning across steps, understanding messy instructions, reading files, remembering what matters, and moving through work without needing every step spelled out. This sounds purely good — but it also means yesterday's harness can become wrong at the price of an update.

Nate gives concrete examples of both directions of breakage:

Harness too tight (designed for a weak model)

  • A tool that helped a weaker model can confuse a stronger one
  • A rule that protected against an unreliable model's mistakes can trap a better model
  • A workflow that forced structure around a clumsy agent becomes a drag when the model can handle much more
  • Example: you told the agent "don't infer, don't create records, don't recommend — just summarize." That was right for the old model. The new model can compare sources, tell weak signals from patterns, and draft useful next steps — but your harness still treats it like the old model

Harness too loose (designed for a weak model with human catch-all)

  • You gave the old model broad access because you knew a human would catch everything
  • The new model can take 20 plausible actions in a few minutes
  • They look real, they look organized — they create work that a human has to unwind
💡 "We are used to software breaking when it gets worse. Agents can also break when the model gets better — and that is a different and new thing."

4 Principle 2 — Agents Inherit Organizational Drift

▶ 5:35

Every company already has drift problems — they existed before AI. The product wiki is wrong. The CRM field means something different than it used to. The dashboard still says "activation" but the team changed what activation means. The support tags evolved. The roadmap moved. The owner changed. The docs didn't.

With normal software, this is vaguely annoying. With agents, it's dangerous. Because agents don't just sit — they produce work. They summarize, recommend, draft, route, update, and act. That's their value. So:

  • Stale wiki → agent reads and ingests stale truth
  • Changed process → agent follows old process unless you update the docs
  • Outdated prompt → agent serves last quarter's company
  • Incorrect dashboard definition → agent makes the wrong number feel very convincing
💡 "This is not a model failure. The agent did its job. It's the old maintenance problem with a machine that can now produce work from that mess — work that is sometimes very convincing."

The Sailboat Metaphor

Nate introduces Stewart Brand's The Maintenance of Everything as the right frame for thinking about agents. Brand writes about sailboats, vehicles, weapons, manuals, corrosion — the work that keeps important systems alive after the launch moment is over.

Agents are much more like sailboats than apps: you don't launch them and walk away. The weather changes, the lines loosen, salt gets into everything. A sailboat is not maintained because it was badly designed — it is maintained because it lives in motion. Agents live in motion too: the model changes inside them, the world changes around them.

5 Principle 3 — Frontier Labs Already Know This

▶ 8:14

The biggest AI companies aren't just betting that their models will get better. They're betting that better models can help ship and evolve the harness faster. This is the strategic context behind products like Codex and Claude Code.

Codex as a maintained workbench

Codex is strong not just because the model is strong — it's strong because OpenAI keeps maintaining the harness around the model so it feels intuitive and native as the model and world evolve. It has become an operating surface for work:

  • Terminal, desktop app, IDE, browser, computer use
  • Files, plugins, memory, automations
  • Approvals, sandboxing, network controls
  • Keychain storage, managed configs, logs

This is way beyond a chat box with a smarter brain — it's a carefully maintained workbench around machine work.

Why each tool in the workbench matters

  • Terminal — real work lives in commands, repos, files, tests, and local tools (Claude Code's original insight)
  • Browser — real work happens on interfaces that humans see
  • Computer use — not every tool has a clean API
  • Plugins — work lives in other systems (GitHub, Google Drive, Jira, Slack)
  • Memory — preferences and corrections shouldn't have to be rebuilt every day
  • Approvals and sandboxing — a capable agent still needs boundaries
  • Logs — when an agent does something weird, someone needs to know what happened
💡 "There are really two teams in the world building good harnesses right now: the Anthropic team and the OpenAI team."

6 The Harness–Capability Flywheel

▶ 10:18

Here Nate describes the compounding loop that makes the frontier lab bet so powerful:

  1. Better agents help build more effective harnesses
  2. Better harnesses make the agents more useful
  3. More useful agents help rebuild the harness once more

If the model can help you ship the harness, test the harness, refactor the harness, observe the harness, and train the harness, then capability gains start to compound — a flywheel ignites.

This reframes the Vercel story as not just a quirky sales automation anecdote. It's a pattern: the companies that win are the ones that keep rebuilding the wrapper as the model and the work change.

What this means for everyone else

If you're building your own agent setup, you're now making a choice:

  • Light custom harness — a clean set of instructions, memory, source folders, and repeatable methods around Codex or Claude. "Here are the sources. Here's the job. Here's what you can't touch. Here's the proof I need. Here's when a human decides." This can be enough
  • Deep custom harness — data feeds, review screens, permission levels, logs, model choice, escalation paths, approval rules, and a plan for what happens when the model changes. Very worth it for some workflows, but now you're not just building an agent — you're investing in the long-term maintenance of an agent and harness system
💡 "The more custom the harness, the more you own the upkeep." — This is the build-vs-buy question reframed for the agent era.

7 What Is Your Harness?

▶ 13:30

The fourth principle: everyone who uses an AI tool already has a harness, whether they call it that or not. It's the setup that makes a model useful for your real work.

Harness by role

  • Individual user — project folders, memory, prompts, source docs, approval habits, browser access, file rules, tools, verification loop, proof-asking habits
  • Product leader — the sources the agent reads before planning
  • Sales — CRM fields, call notes, routing rules, human approval steps
  • Support — policy store, escalation paths, refund rules
  • Writer — drafts, transcripts, voice notes, editorial rules, archive, the instruction that the agent must show where an idea came from
  • Engineer — repo, tests, terminal, permissions, work trees, logs, review rules (the most mature harness example we have today)

Nate's personal harness

Even for Nate's own delegation model, the harness is much more than a prompt — it's the whole way the agent meets his files:

  • Where should it look first?
  • Which folders are source of truth?
  • What should it ignore?
  • What should it ask about before touching?
  • What should it remember? What should it forget?
  • When it searches memory, is that right?
  • When does it actually go read the file?

And crucially: this has changed because the agents have changed, because the models have updated.

💡 "The more useful question is: can I maintain this sailboat over a voyage? What harness does this agent need? Those are the same question."

8 The Five-Point Agent Health Check

▶ 15:52

For any serious agent, Nate recommends checking these five things regularly:

1. What's it eating?

What's the agent reading? Are the sources current? Did the workflow move? Did a new source become important? Did an old source become misleading?

2. Test its reach

What can it touch? Can it only read, or can it draft, create tickets, post in Slack, update records, spend money, publish? A permission that was harmless for a weaker model may be too broad for a strong one. A restriction that made sense for an unreliable model may hold back a better one.

3. Check its job

Is this still a summary agent? Should it be a planning agent now that models are better? Is it supposed to find themes? Recommend tradeoffs? Route work? Do not let the job change silently — change it on purpose if you're going to do it at all.

4. Check the proof

The agent shouldn't just say "customers are frustrated with onboarding." It needs to:

  • Link to tickets
  • Link to sales notes
  • Quote customer language
  • Say which sources it checked and where
  • Say which sources it could not access

The proof is not just the agent saying it — the proof is a linkable trail a human can inspect.

5. Check the value

Does anyone read the output? Does it change the work? Does it save time after review? Does it create another pile of work? Is it duplicating a report? Has the model improved enough that the agent ought to be rebuilt? Has the business changed enough that the agent should be retired?

💡 "Agents, unlike almost anything else, break in two directions. They break because the world around them drifts, and they break because the model inside them improves. And maintenance is the work of keeping the harness fit between those two moving things."

🎯 Key Takeaways

🔑 Key Takeaways

  • Less can be more — Vercel improved its agent by deleting 80% of its tools, not by adding new ones; the beginner instinct is to add, the maintenance instinct is to ask what should be removed
  • The harness is the real product — what the agent reads, remembers, can touch, must prove, and what stops it when work gets risky — this setup matters more than raw model capability
  • Agents break when models improve — a harness designed for a weaker model can either trap a better one (too tight) or let it create convincing chaos (too loose)
  • Organizational drift is agent poison — stale wikis, outdated SOPs, changed processes, and wrong dashboard definitions become actively dangerous when an agent uses them to produce work
  • Agents are sailboats, not apps — they live in motion, with the model changing inside and the world changing around them; they need continuous maintenance, not one-time configuration
  • Frontier labs compete on harness quality — OpenAI (Codex) and Anthropic (Claude Code) invest heavily in the workbench around the model, not just the model itself
  • The harness–capability flywheel compounds — better agents build better harnesses, better harnesses make agents more useful, creating a compounding loop
  • Everyone already has a harness — whether you're an engineer, writer, product leader, or sales rep, your setup of folders, memory, prompts, rules, and verification habits is your harness
  • Custom harness = owned maintenance — the deeper your custom setup, the more long-term maintenance you commit to; light harnesses around existing platforms trade capability for reduced upkeep
  • Five-point health check — regularly audit what the agent reads (sources), what it can do (reach), what its job is (role), what proof it provides (evidence), and whether its output matters (value)
  • Deletion is a maintenance superpower — the really mature question is "what part of this harness will I need to delete later?" — simplicity is key to maintainability
  • Silent failure is the real danger — an unmaintained agent doesn't fail loudly, it keeps working on stale truth until it costs you; all it has to do is keep working to start haunting your business

🔗 Resources & Links

Timestamp Index

▶ 0:00 Vercel made its agent better by deleting 80% of its tools
▶ 0:27 Vercel studied its best sales rep's workflow
▶ 1:39 What happened after the agent existed
▶ 1:57 The beginner instinct vs. the maintenance instinct
▶ 2:41 The harness / workbench concept defined
▶ 4:34 Principle 1: The model changes inside the agent
▶ 4:51 Harness too tight vs. too loose examples
▶ 5:35 Principle 2: Systems drift — wikis, dashboards, SOPs
▶ 7:18 Stewart Brand's The Maintenance of Everything
▶ 7:57 Agents are sailboats, not apps
▶ 8:14 Principle 3: Frontier labs invest in harness
▶ 8:56 Codex as a maintained workbench
▶ 9:22 Claude Code team doing the same
▶ 10:18 The harness–capability compounding flywheel
▶ 11:46 Light vs. deep custom harnesses
▶ 13:30 Principle 4: What is your harness?
▶ 13:55 Harness by role — product, sales, support, writer, engineer
▶ 14:30 Nate's personal harness maintenance
▶ 15:52 Five-point agent health check begins
▶ 16:05 Check 2: Test the agent's reach
▶ 16:21 Check 3: Is the job still right?
▶ 16:41 Check 4: Proof and linkable trails
▶ 17:03 Check 5: Does the agent's output matter?
▶ 18:03 Read The Maintenance of Everything — closing