🧭 The Three Kinds of Harnesses 0:56
Sam opens by untangling a word that's caused a lot of the confusion: "agent harness." When most people hear it, they think of coding harnesses. But there are really three distinct categories, and they solve different problems:
| Kind | Examples | What it gives you | The trade-off |
|---|---|---|---|
| Coding harness | Claude Code, Codex, Pi, OpenCode | Model + filesystem + shell + tools, runs the loop | Built for one job: writing code |
| Assemble-yourself framework | LangGraph, Deep Agents, Agent SDK / ADK | All the pieces | You wire the loop, state, and approvals yourself — tedious at scale |
| Hosted runtime harness | Claude / Gemini managed agents | Provider runs session, tool calls, sandboxes, approvals | You're tied to their models and their hosting |
The third category — what people now call managed agents — is genuinely nice to use: you define an agent, give it a model (almost certainly the provider's), tools, and a system prompt, and the company runs the loop while you just hit an API. The catch, until very recently, is that "that category really meant you only had one choice — it has to be hosted with the actual provider," and you're tied to models that may not fit your use case.
🆓 TrueForge: The Open-Source Managed Agent 3:43
TrueForge 🔗 is TrueFoundry's open-source answer to that third category. The README's framing is the whole pitch in one line: "Building an agent is easy, but running one well is not."
A transparency note worth stating up front: this video is sponsored by TrueFoundry, the creators of TrueForge. But Sam is explicit that everything shown is in the open-source repo, there's no paid tier in what you'll see, and the only ask is a GitHub star if you find it useful. (He's worked with the team before on their gateway platform, so he's not a stranger to the company.)
💰 The Harness Matters More Than the Model 4:22
The core insight of the video is a mental-model correction. Most people — Sam includes himself — think of agent cost as "which model, and how much per token." But at volume, the harness is usually the thing driving your bill, because it's the harness that decides how many tool calls you make, in what order, and what goes into the context window.
"Really, what we're paying for is whatever's in the context window." Tool bloat — too many tool definitions loaded in without progressive disclosure — runs the bill up fast, and that's a harness decision, not a model decision.
📊 The Benchmark: Same Model, 30% Cheaper 5:29
The TrueFoundry team put the harness thesis to the test. They took 14 tasks from DevRev's enterprise bench — the kind where an agent joins data across a CRM, an issue tracker, a document store, and multiple MCP servers, all with the same prompt — ran three trials each, and graded with a blind LLM judge. Crucially, they didn't stack the deck with cheaper models: it ran on Opus through three different harnesses.
| Harness | Tokens per run | Relative cost |
|---|---|---|
| Claude managed agents (Opus) | ~10 million | baseline |
| TrueForge (Opus, same model) | ~3.8 million | −30%, identical accuracy |
| TrueForge (GLM 5.2) | — | −75% vs Claude managed agents |
The conclusion is stark and specific: "same model, same answers, for roughly a third of the tokens — that was all harness." And because TrueForge is open source, you can then take the further step of dropping to an open model like GLM 5.2 and cut the whole thing by 75% against the original Claude managed-agents baseline. The harness is the lever, and the model choice on top of it is the second lever.
🔓 Vendor Lock-In & Data Retention 7:21
For Sam, the cost savings are the headline but lock-in is the bigger deal — "probably a bigger deal than a lot of people are watching just on the park sweep." The concrete pain points stack up fast:
He also points to the structural change making lock-in unnecessary: open models are now so good that high-level intelligence is democratized, and "there are providers popping up everywhere running the same weights with the same API endpoints." For people building inside larger companies, there's often a hard mandate to run on-prem or in their own cloud account — "a hosted agent system just isn't going to get through procurement, no matter how good it is" — which historically forced them to write the whole loop themselves, "an inefficient and expensive option." TrueForge slots into that gap, and it even integrates with Ollama and LM Studio for fully local models.
🏗️ Architecture & Runtime 9:22
TrueForge is an MIT-licensed TypeScript repo you spin up with a single npx command. In local mode it provisions a SQLite database — all your data stays local, running on localhost. Hosted modes (Postgres, Redis, Docker) exist too, but the local mode is "a very deliberate step on TrueFoundry's part to make this as easy as possible to get started."
The chat UI is a trap for the unwary: it's not the product. "Don't be fooled into thinking this is another chatbot — it's the test bench," where you poke at the agent before saving it. The actual runtime is exposed three ways:
🧩 Models, Connectors, Skills, Sandboxes 11:03
The configuration surface is four things in settings. Models: off-the-shelf Claude, Gemini, and a bunch of open providers, or a totally private endpoint — Sam configures his own API endpoint to run DeepSeek Flash 4.1 (his old favorite was DeepSeek V4 Flash, so he's curious about the new one). Connectors: Tavily for search, DeepWiki, Notion, Bright Data for scraping, or a custom MCP server running locally or in the cloud.
Skills: you import a folder containing a SKILL.md — Sam drops in the Tavily research skill — and it becomes available to the agent. That's the same skill pattern now showing up across Claude Code, Hermes, and the wider ecosystem. Sandbox providers: he flags this as the currently light area ("TrueFoundry's got some interesting things around sandboxes coming in the near future").
The demo is an "AI news" agent built in minutes: a system prompt pointing at VentureBeat's AI category, the Tavily connector doing the searches, DeepSeek Flash 4.1 running the whole thing, with toggles for clarifying questions, dynamic sub-agents, and generative UI (he turns those off for a simple ping-and-return flow). The output comes back with the stories, a summary, and a source URL for each.
🐍 Using Python & Shipping 17:33
Using the agent from code is deliberately minimal: a uv project with a single Python file that calls the localhost endpoint where the agent is hosted. uv run main.py sets everything up, runs the task, and streams results back — or you can have it write an output file or save to a database at the end.
The portability story is the point: agents you create are stored, so you can prototype locally, then install TrueForge in the cloud and import them there. Sam expects TrueFoundry to eventually offer their own managed version — "prototype everything locally and then ship it to their cloud to run regularly at scale" — but the open-source path means that's an option, not a requirement.
🗺️ Roadmap & Verdict 18:50
The roadmap addresses his biggest gap. Gateway integration (a natural fit, given TrueFoundry's existing gateway product for models, MCPs, and policy enforcement), and — the one he wants most — local sandbox execution, ideally lockable with Docker sandboxes "to make it fully safe and secure." The rest he describes as "very sensible stuff… perhaps stuff that is in the locked-down versions of managed agents."
Sam's closing framing is the right one to end on: this is "all the advantages of a managed solution without the vendor lock-in and without the downsides." Whether it's production-ready for you depends on how much you need those sandboxes — but the direction is clear.
💡 Key Takeaways
- There are three kinds of harnesses, not one. Coding harnesses, assemble-yourself frameworks, and hosted runtimes (managed agents) — each solves a different problem.
- TrueForge is the open-source managed agent. Clone an MIT repo, run it on any model, on your own hardware, no runtime bill.
- The harness, not the model, usually drives your agent costs. It decides tool-call count, ordering, and what's in the context window — and tool bloat is a harness decision.
- Same model, same answers, a third of the tokens. Opus through TrueForge used 3.8M tokens/run vs 10M on Claude managed agents — a 30% cut, 75% on an open model.
- Lock-in is the bigger deal than cost. Zero-day data retention and on-prem mandates rule out hosted agents entirely; TrueForge keeps the agent definition on your side.
- The chat UI is a test bench, not the product. The real runtime is the HTTP endpoint and TypeScript SDK, bakeable into Slack or any app.
- Watch the sandbox providers. It's the least mature piece — local sandbox execution is on the roadmap and is the thing to verify before production use.
🔗 Resources & Links
- 📦 github.com/truefoundry/trueforge — the repo (MIT, TypeScript)
- 📖 trueforge.dev/introduction — docs
- 🌐 truefoundry.com/trueforge — product page
Source video: youtube.com/watch?v=3AdW345Jskk