This Open Source Is Quietly Eliminating Agent Hallucinations

Overview

DevOps Toolbox reviews Babysitter, an open-source agent orchestration framework by A5C AI that aims to eliminate hallucinations not through magic, but through deterministic quality gates, enforced TDD workflows, and approval loops. The tool wraps around existing AI coding agents (Codex, Cursor, Pi, Gemini, Copilot, OpenCode) and enforces obedience — making sure the agent does exactly what you asked, with tests first, quality gates at each step, and a clear definition of done. The trade-off: it's slower, but the results are dramatically more precise and reliable.

1 The Hallucination Problem

▶ 0:00

The host opens with a blunt admission: as a $200/month OpenAI subscriber who has tried every model (Anthropic, OpenRouter, Quirks, OpenCode, Zen, Go, Black) — AI hallucinates all the time regardless of your tier or spend.

The standard approach — guardrails and rules — doesn't really work because AIs don't follow rules reliably. What if instead of rules, you had an actual system that orchestrates tasks with deterministic outcomes?

• Not a magic wand, not a trick
• An actual orchestration layer that enforces obedience
• Open source — nobody's selling it
• Aims for hallucination-free output

2 What Is Babysitter?

▶ 1:00

Babysitter by A5C AI is an open-source framework built "by developers for developers." The core premise identifies three problems with current AI agents:

1. Everything is a black box — no real audit of what the agent is doing or why
2. Context window is strangling you — agents lose track of what matters as context fills up
3. The agent decides when something is "done" — not you

Babysitter's solution: quality gates + loops until criteria are met + a real definition of done that isn't a metaphor.

Key properties

• Open source — no data sent anywhere, fully inspectable
• Model agnostic — works with any LLM provider
• Agent wrapper or standalone harness — can wrap Codex, Cursor, Pi, Gemini, Copilot, OpenCode, OR run independently in CI/ephemeral environments
• Four execution modes: call (standard), plan-only, yolo (autonomous), and forever mode

3 Installation & Execution Modes

▶ 2:43

Babysitter can be installed as an npm package or as an extension for your existing agent:

npm install @a5c-ai/babysitter-sdk

For Pi users, it's even simpler — Babysitter is listed as an optional extension:

pi install

The four modes

• Call — standard mode: you provide a prompt, Babysitter orchestrates the agent through quality gates with approval checkpoints
• Plan-only — generates a plan without executing, useful for reviewing the approach first
• YOLO — autonomous mode: the agent runs with minimal human intervention. "Let's see what happens"
• Forever — continuous mode for long-running processes

The orchestrator can be set to internal (Babysitter's own harness), claude, or other options. You specify a workspace directory and a process (actual JavaScript/TypeScript code, not markdown).

4 User & Project Onboarding

▶ 5:05

Before any work begins, Babysitter goes through a two-phase onboarding: first it learns about you, then about your project.

Phase 1: User profile

Run babysitter user install to create a markdown profile with:

• Name, specialties, experience level
• Preferences (verbosity, style — "for the love of God, emoji free")
• GitHub scan to detect language preferences (e.g., Go and Shell vs JavaScript)

The host keeps the context short and specific — more detail isn't always better for agent context.

Phase 2: Project onboarding

Run babysitter project install and let it explore on its own. It reads:

• Source code, dependencies, git history
• Asks what you're optimizing for: security, production reliability, or specific concerns it detected (e.g., data integrations)
• Asks about your workflow: PR-based? Trunk-based development?

This takes about 15 minutes. The context window fills up significantly. Afterward, you can run babysitter doctor to get a report of what needs attention.

Built-in methodologies

During setup, Babysitter reveals the methodologies cemented into its brain — including debugging, testing, and GSD (Getting Stuff Done, a productivity method). The docs list processes for TDD, web/mobile apps, DevOps, security, and even business and science domains.

5 The TDD Workflow in Action

▶ 8:06

The host builds a small REST API to demonstrate Babysitter's workflow. The prompt requests a task API with all CRUD methods, using TDD (Test-Driven Development) with a quality target to pass.

Why TDD matters for agents

The host makes a strong observation: whenever he asks any agent with any model to use TDD, he gets better results. It's an N=1 observation, but he's not alone in noticing it. His personal irony: "I was always taught to use TDD but never actually done it properly. But now we have little coding minions and they can't refuse yet."

The execution sequence

1. Orchestrator loads — calls skill instructions
2. Plan printed — lists endpoints and quality gates
3. Process code generated — Babysitter produces its own TypeScript orchestration code (not markdown instructions)
4. Approval gate — asks for human approval before proceeding
5. Tests first — literally the first piece of generated code is the test file
6. Implementation follows — code written to pass the tests
7. Report — endpoints in place, tests run with coverage, spec code verified

The precision factor

The result is extremely precise — almost to a fault. The API has zero logs because they weren't requested. The host notes this "can either be good or problematic and requires verbosity from your side when planning."

Testing the live API: GET /tasks returns empty array ✅, POST with empty data returns proper validation error ✅, POST with valid JSON creates the task ✅, DELETE works ✅.

6 YOLO Mode — The Contrast

▶ 7:40

In parallel, the host ran a second task in YOLO mode with a generic, less specific prompt — "sent into oblivion." The results contrast sharply:

• It produced something — login, auth, and actions
• "No real structure, but fine"
• Running it eventually works, but produces unexplained errors
• The host doesn't bother debugging: "I still don't know what's up here"

The lesson: YOLO mode without precise instructions produces the typical AI agent experience — functional-ish but messy. The call mode with TDD and quality gates produces the precise, correct result.

7 Verdict — The Philosophy of Obedience

▶ 11:10

The host's final assessment is genuinely positive:

• Babysitter is not a magician — it's an orchestrator. It won't turn a bad prompt into a good result
• It's not OpenClaw — not "running around drinking tokens and chatting to other agents for fun"
• If you're a pro, it'll be a pro employee — but the user is a huge part of the process. You need to be precise in what you ask for
• Speed is the trade-off — running GPT 5.5 on high/xHigh with Babysitter on top means "you don't get the Usain Bolt of AI agents." He dropped to medium reasoning just to move faster

He commits to running it for the next few weeks/months and reporting back.

🎯 Key Takeaways

🔗 Resources & Links

• Babysitter on GitHub — the open-source agent orchestration framework by A5C AI

⏱ Timestamp Index