I Replaced Hermes Agent And OpenClaw With This

I Replaced Hermes Agent And OpenClaw With This

๐ŸŽค Owain Lewis โฑ 22:13 ๐Ÿ“บ 14.2K subscribers Deep Dive

1 Introduction โ€” Why I Switched

โฑ 0:00

Owain Lewis spent months living with Hermes Agent as his daily AI assistant. The core promise resonated with him deeply โ€” an AI system that learns about you over time, builds skills, and operates autonomously across platforms. The concept was excellent.

But friction accumulated. Automatic memory creation kept stuffing random context into his profile. Skills generated on the fly sometimes produced unreliable or stale instructions. Instead of fighting the system, he decided to build something smaller and more opinionated that works with the coding agents he already trusts.

๐Ÿ’ก This isn't an anti-Hermes video. Owain is explicit: both Hermes and OpenClaw are incredible projects. He simply wanted a thinner layer between himself and the agents doing the actual work.

2 How Hermes Agent Architecture Works

โฑ 2:00

Before explaining his alternative, Owain gives a fair breakdown of what Hermes Agent does well. The architecture revolves around a persistent identity layer built from several components:

  • memory.md / user.md โ€” persistent files that store facts about the user, preferences, and context across sessions
  • Automatic skill creation โ€” when Hermes solves a complex task, it can save the approach as a reusable skill
  • Multi-platform connectivity โ€” Telegram, CLI, and other interfaces
  • Cron jobs โ€” scheduled autonomous tasks
  • Tool system โ€” extensible tool calls for file operations, web browsing, code execution
Hermes Agent Architecture
Telegram
CLI
iMessage
โ†’
memory.md
user.md
skills/
โ†’
Hermes Agent Core
โ†’
Browser
Terminal
File I/O
Cron

The identity layer โ€” where the agent knows who you are and how you like things done โ€” is, in Owain's words, what Hermes genuinely nails. This is the hardest problem in personal AI, and Hermes takes a strong swing at it.

3 Problems with Automatic Memory & Skills

โฑ 5:00

Here's where the friction built up. The automatic memory system โ€” designed to learn about you organically โ€” had a tendency to stuff random garbage into the memory and user files:

  • One-off facts from a single conversation persisted as if they were permanent preferences
  • Outdated context never got cleaned up, leading to stale facts that confused future sessions
  • The "soul" file accumulated contradictory or irrelevant personality traits

Skills: Theory vs. Practice

Automatic skill creation sounds incredible on paper. In practice, Owain found that skills generated on the fly were often:

  • Unreliable โ€” instructions that worked once but weren't generalizable
  • Overly specific โ€” tied to a particular environment or context
  • Hard to audit โ€” you don't know what skills exist until they interfere with something
โš ๏ธ Owain ended up hacking Hermes itself โ€” adding gates to memory creation and modifying core files. When you're modifying the tool to stop it from doing what it's designed to do, that's a signal the tool's philosophy doesn't match yours.

4 OpenClaw's Approach & Its Issues

โฑ 8:00

Owain also evaluated OpenClaw โ€” a different agent framework with its own architecture. After 100+ hours and over $1,000 in testing costs, he encountered significant problems:

  • 200+ vulnerabilities discovered during security assessment
  • Malicious packages found in the dependency chain
  • Similar issues with autonomous agent behavior โ€” agents acting without sufficient guardrails

The core issue wasn't unique to OpenClaw. It's a pattern across full-featured agent frameworks: the more autonomous the system, the more opportunities for things to go sideways without the user's knowledge.

๐Ÿ”’ Security is an under-discussed dimension of agent frameworks. When an agent can install packages, modify files, and make network calls autonomously, the attack surface becomes enormous.

5 Building Push โ€” The Simpler Alternative

โฑ 11:00

Rather than building another full agent framework, Owain built Push โ€” a deliberately small gateway that connects messaging platforms to the coding agents you already trust.

What Push Is

  • A thin gateway โ€” not a framework, not an agent, just a routing layer
  • Connects Telegram and iMessage to agents like Codex, Claude Code, or Pi Agent
  • Lets you message your assistant from anywhere โ€” phone, desktop, on the go
  • Supports voice messages natively
  • Runs scheduled jobs without complex cron infrastructure

What Push Is Not

  • Not a memory system โ€” no automatic fact storage
  • Not a skill system โ€” no auto-generated procedures
  • Not trying to be an "identity layer" โ€” that's the coding agent's job
โœ… Push's philosophy: your coding agent (Codex, Claude Code) already handles the hard AI work. Push just makes it accessible from anywhere.

6 How Push Works โ€” Architecture

โฑ 14:00
Push Architecture
Telegram
iMessage
๐ŸŽ™ Voice
โ†’
Push Gateway
polling ยท routing ยท scheduling
โ†’
Codex
Claude Code
Pi Agent
โ†’
Todoist
Gmail
GitHub

Key Design Decisions

  • Telegram polling โ€” Push polls for messages instead of requiring inbound webhook ports. Simpler deployment, no port exposure, better security.
  • Message routing โ€” incoming messages are structured with context and routed to the appropriate coding agent
  • Personality configuration โ€” lightweight config file, not an AI-managed "soul"
  • Scheduled jobs โ€” simple cron-like scheduling without a full orchestration layer

Task List as Control Plane

This is perhaps the most opinionated design choice in Push. Instead of letting the AI decide what to do next, Owain uses his task list (Todoist) as the control plane. The AI works through tasks you've explicitly defined, giving you oversight and control over what gets delegated.

๐ŸŽฏ The task list pattern inverts the autonomy model: instead of the AI deciding what's important, you decide and the AI executes. This is a deliberate trade-off โ€” less autonomous, more predictable.

7 Real-World Usage โ€” Email, Tasks, Research

โฑ 17:00

Owain walks through his actual daily workflow with Push. The system handles practical, everyday tasks:

Email Workflow

  • Assistant reviews incoming email via Gmail integration
  • Creates Todoist tasks from important messages
  • Drafts replies for review before sending

Task Management

  • Todoist serves as the shared workspace between human and AI
  • Tasks are created, prioritized, and completed through the assistant
  • Full visibility into what the AI is working on at any moment

Research & Development

  • Research queries sent via Telegram message or voice
  • GitHub integration for code-related tasks
  • Results delivered back through the messaging platform
๐Ÿ“ฑ The killer feature isn't any single integration โ€” it's that everything is accessible from your phone via Telegram. No SSH, no terminal, no laptop required for routine delegation.

8 Why Smaller & Opinionated Wins

โฑ 20:00

Owain closes with a philosophy that frames the entire video: composability over monolithic agent systems.

The Composability Argument

  • Use the best tool for each job โ€” Codex for coding, Gmail for email, Todoist for tasks. Don't make one system do everything.
  • The gateway pattern โ€” a thin routing layer that connects existing best-in-class tools is more maintainable than a monolithic agent.
  • Reduced complexity โ€” fewer moving parts means fewer failure modes and easier debugging.

What This Means for Agent Frameworks

This isn't a rejection of agent frameworks โ€” it's a different bet on where the value is. Hermes and OpenClaw bet that the agent itself should be smart, stateful, and autonomous. Push bets that the agents already exist (Codex, Claude Code) and what's missing is just a way to reach them from anywhere.

๐Ÿงฉ The best personal AI assistant might not be the one with the most features. It might be the smallest one that connects everything you already use.
Philosophy Comparison
Hermes / OpenClaw

Full agent framework
Memory + Skills + Identity
Maximum autonomy
One system for everything

Push

Thin gateway layer
Routes to existing agents
Human-controlled task list
Composable architecture

๐ŸŽฏ Key Takeaways

1 Hermes Agent's identity layer (memory.md/user.md) is genuinely good โ€” but automatic memory creation often adds stale or wrong facts
2 Automatic skill creation sounds great in theory but produces unreliable instructions in practice
3 Push is a thin gateway, not a full agent framework โ€” it routes messages to agents you already trust (Codex, Claude Code)
4 Telegram polling avoids exposing inbound ports โ€” simpler and more secure
5 Treating your task list as the control plane gives you oversight over delegated AI work
6 Composability beats monolithic โ€” use the best tool for each job rather than one system for everything
7 Voice message support and multi-platform connectivity don't require a heavy framework
8 The best personal AI assistant is opinionated and minimal, not feature-complete
Push Hermes Agent OpenClaw Codex Claude Code Personal AI Telegram Composability