📋 Overview
On April 16, 2026, OpenAI transformed Codex from a developer-focused CLI tool into a full desktop agent that can see your screen, click and type across any macOS application, and run multiple agents in parallel — all in the background while you keep working. Nate B Jones walks through what Codex is now, why its computer use implementation is meaningfully ahead of Claude's, the team that made it possible (the creators of Apple Shortcuts), and the deeper strategic divergence between OpenAI and Anthropic: OpenAI is building a body that drives any GUI, while Anthropic is building a body that requires the ecosystem to ship agent-ready interfaces. Both are racing toward persistent, ambient, event-driven agents — but through fundamentally different mechanisms.
1 What Codex Is Now — The Desktop Agent Evolution
When OpenAI launched the original Codex command line in April 2025, it was a straightforward developer tool: you typed prompts, it wrote code. A year later, Codex has become a full desktop application with capabilities that go far beyond code generation:
- Screen vision — sees your screen and can interact with any macOS application
- Click and type — operates GUIs like a person, across the entire desktop
- Image generation — built-in capability within the agent
- Web browsing — has its own built-in browser
- Memory — remembers what you've been working on across sessions
- Self-scheduling — can wake itself up for long-running tasks
- Parallel agents — runs multiple agents simultaneously without hijacking your cursor
- 90+ plugins — specialized capabilities in one unified app
The Transformation Timeline
- Feb 2, 2026: macOS desktop app launches
- Mar 4: Windows support added
- Mar 5: GPT-5.4 folds Codex's coding capability into the general-purpose model line
- Mar 19: OpenAI confirms the super app consolidation plan
- Apr 16: The pivotal release — computer use, in-app browser, image generation, memory, 90+ plugins
2 Speed & Reliability — Codex vs Claude Computer Use
Jones has been running Codex and Claude side-by-side on the same workflows. His assessment is that the gap is not subtle:
Speed
- Codex moves through workflows at close to the speed of a person who already knows the software
- Claude is slower — the cursor hesitates, the agent rethinks, sometimes retries
- Tasks that finish in 2 minutes on Codex can take 5-6 minutes on Claude
Scope
- Claude prefers Chrome — it's limited in where it can go on the desktop
- Codex can touch anything on macOS — any app, any window
Reliability
The historical failure mode for computer use products has been very specific: the agent gets partway through a task, hits an unexpected modal dialog or other UI issue, fumbles, and you have to take over. Codex's implementation generally doesn't fumble — it backs up when it needs to, recovers, and finishes quickly.
The Model Underneath
GPT-5.4 is the first general-purpose OpenAI model with native computer use capabilities baked in. It benchmarks in the mid-70s on OS World, which puts it above the human baseline for GUI control.
3 Real-World Workflows — Beyond Demos
Jones highlights early user reports showing the range of what's actually working with Codex computer use — notably, none of these are demos, they're workflows people ran and then decided to keep running:
- Mass Slack triage — reading and clearing hundreds of unread bot messages and daily digests so the inbox starts clean
- Spotify playlist building — driving the desktop Spotify app from a verbal description
- Front-end visual regression testing — walking through a web app to catch visual bugs while the developer keeps shipping the next feature
- Bug reproduction + PR creation — reproducing a browser bug, screenshotting the reproduction, pasting the image straight into a pull request description
- End-to-end testing + self-fixing — running E2E tests on the app you just built, then auto-fixing the issues it finds
- Legacy dashboard automation — driving internal dashboards that never had an API, "exactly the software category automation has given up on for years"
- Daily recap pipeline — pulling Git commits, issue tracker updates, and calendar events → writing meeting minutes to Notion → dropping prioritized to-dos into Apple Reminders
- Daily login routines — playing login sequences in the background automatically
- Posture monitoring — detecting webcam slouching and popping up a stretching video
4 Two Labs, Two Bodies — OpenAI vs Anthropic Strategy
Both labs have left the "coding tool" box, both built desktop apps — but they moved in fundamentally different directions. Jones frames the divergence through the metaphor Greg Brockman used: "models have gone from being the product to being part of the product" — the brain is built, now the work is on the body.
Anthropic's Body: Structured Interfaces
Anthropic pivoted toward knowledge work — synthesis, research, writing, analysis. Their body is built from:
- Co-work — point Claude at a folder, describe an outcome, it does the intellectual labor
- MCP servers — 3,000+ integrations in the Claude ecosystem
- Plugins & connectors — maintained by the ecosystem
- Conway — an always-on event-driven agent environment (leaked in early April)
The architecture is excellent, but it depends on the ecosystem cooperating. Every MCP server has to be built by someone. Every connector has to be maintained. Every webhook has to be configured. How far the body can reach depends entirely on how fast the world ships agent-ready interfaces.
OpenAI's Body: Computer Use
OpenAI pivoted toward something broader than knowledge work — what Brockman calls "computer work," which encompasses ops, creative work, research, and automation. Their body is computer use itself:
- Legacy enterprise software — works
- Internal tools — works
- SaaS that will never build an MCP server — works
- The vendor portal your ops team tolerates — works
- The app your company built in 2019 and never touched again — works
5 Explicit vs Implicit — The Design Philosophy Split
The two apps reflect their companies' philosophies at the product design level:
Claude: Explicit by Design
- Every mode has a scope and permission model
- Co-work asks you to point at a folder before it does anything
- The friction is the product choice — it communicates what's about to happen so you can be deliberate
- Explicit scopes, explicit permissions, explicit interfaces
Codex: Implicit by Design
- Built like a tool that assumes you don't want to think about modes
- You describe the outcome — the agent decides whether to open files, drive a GUI, run a plugin, browse the web, or write code
- Computer use is the escape hatch when nothing else works
- Plugins are invoked without explicit selection
- Memory keeps context flowing between tasks automatically
6 The Team — Workflow → Shortcuts → Sky → Codex
Codex's computer use capability didn't appear from nowhere. In October 2025, OpenAI acquired a 12-person company called Software Applications Incorporated — the team behind Sky, an unreleased macOS-native AI interface doing essentially what Codex's computer use does now.
The Lineage
- Ari Weinstein & Conrad Kramer — previously built Workflow, the iOS automation app Apple acquired in 2017 and turned into Shortcuts
- Shortcuts is now the system-level automation layer across iOS, iPadOS, and macOS
- Kim Beverett — spent 10 years at Apple as a senior program manager working on Safari, WebKit, privacy, Messages, Mail, Phone, FaceTime, and SharePlay
This accumulates into a very specific skillset: deep macOS integration, accessibility and screen recording permission handling, motion paths that don't feel robotic — the kind of OS-level work that takes years of Apple experience to do well.
Weinstein publicly credited his team by name for the cursor motion work — the specific detail that makes background computer use feel like a co-worker instead of malware.
The Broader Pattern
Both labs are making the same move: acquiring narrow, hard-to-replicate human expertise — not for the IP, often not for the product, but for the team:
- Anthropic acquired a research team that shipped Claude's Windows desktop control within 4 weeks of joining
- OpenAI acquired IO (Jony Ive's hardware startup) for several billion dollars
7 Chronicle — Ambient Memory as Training Signal
On April 20, four days after the main Codex release, OpenAI shipped Chronicle as a research preview for ChatGPT Pro users on macOS. What it does:
- Captures your screen periodically
- Processes the images on OpenAI servers
- Writes local Markdown files that Codex pulls into future sessions as context
Thibault (Codex team lead) called it "early and token hungry" but said it had changed how he and many people at OpenAI work. Altman described it as "telepathy-like."
The Deeper Read
The shallow read is ambient memory. The deeper read is that Chronicle is the training signal for computer use. If your agent is going to drive the same GUI you drive, it benefits enormously from watching how you drive it. Your workflows, your app preferences, your muscle memory patterns — they all become context the agent inherits.
Privacy Tradeoff
Screen captures are sent to OpenAI servers, with unencrypted local memories. Not available in the EU, UK, or Switzerland for that reason. That's a cost OpenAI accepted because they believe the future works better with it and because the data makes the agent better at driving your software over time.
8 Conway — Anthropic's Event-Driven Bet
On April 1st, Anthropic accidentally published the Claude Code source code (~500,000 lines of TypeScript) when a packaging error pushed it to a public registry. Buried inside was Conway:
- Always-on event-driven agent environment with its own sidebar UI
- Three panels: Search, Chat, and System
- Webhook-triggered invocation — events fire the agent, not manual prompts
- Proprietary extension format — Anthropic's own connector standard
- Browser control via Claude in Chrome
- Deep connector integration with the MCP ecosystem
When Does Conway Win?
Anthropic wins this bet if:
- MCP adoption accelerates significantly
- Enterprise software ships clean integrations
- Standards mature quickly enough for Conway to have a rich substrate
MCP is already spreading fast and has become a de facto standard. The Claude Code developer ecosystem has tremendous momentum. But it requires the ecosystem to move, and enterprise software ecosystems tend to move slowly. The long tail of internal tools is not getting agent interfaces on a timeline that matters.
9 OpenAI's Strategic Vectors & What They Cut
In the Ashley Vance interview, Brockman and Altman described three strategic vectors that define OpenAI's entire roadmap:
- The agentic platform — ChatGPT as conversation hub, Codex as agent, Atlas as browser
- Computer work — specifically, not just knowledge work
- Personal AGI — that does things for you in the real world
What Got Cut
The clearest signal of how serious OpenAI is: what they killed:
- Sora got shut down — video generation didn't ladder up to the three vectors
- A drug discovery effort got pulled — interesting but off-strategy
Jones notes this is "an unusually disciplined cut for a company that's been accused of sprawl for a while" — leadership is willing to kill popular products to stay focused.
Compute as Profit Center
Compute is explicitly a profit center now, not just a constraint. The super app consolidation plan (ChatGPT + Codex + Atlas) is a bet on routing users into a platform where compute gets billed at margins OpenAI chooses. That changes the financial shape of what they're building.
10 What To Use & When — Practical Recommendations
Use Codex When:
- Driving software without good APIs — legacy tools, internal dashboards, vendor portals
- Cross-tool triage — Slack + email + calendar + issue trackers at the same time
- Front-end visual testing — catching visual regressions across app states
- Bug reproduction — screenshotting repros and filing them automatically
- Parallel background agents — queuing 3-4 tasks before stepping out has become a real workflow
- Ambient work that doesn't fit into a single mode
- Refactoring complex codebases & long-running development tasks
Use Claude When:
- Scoped, bounded knowledge work — Co-work's "point at a folder and describe the outcome" is cleaner for tasks where you want explicit control
- Developer-friendly coding workflows — Claude Code remains well-loved for its ergonomics and multi-agent workflows
- Work with proper agent integrations — if the software already has MCP servers or will get them soon, Claude's structured approach is cleaner and more reliable
What To Watch Next
- Conway launch: If Anthropic announces it publicly → event-driven agents become a real category. If they delay or kill it → they've lost confidence in the ecosystem-cooperates bet
- MCP adoption velocity: If a major wave of enterprise vendors ships MCP servers in H2 2026 (Salesforce was an early hint) → Anthropic's architecture looks better and better. If they don't, or ship thin wrappers → driving the UI directly stays the better path
🎯 Key Takeaways
🔑 Key Takeaways
- Codex is no longer a coding tool — It's a full desktop agent that can see your screen, click and type across any macOS app, and run multiple agents in the background simultaneously
- Computer use gap is wide — Tasks that take 2 minutes on Codex take 5-6 on Claude; Codex can touch any app while Claude prefers Chrome; GPT-5.4 scores above human baseline on OS World
- The body, not the brain, is the product now — Models are converging; competitive advantage comes from the harness around them — how agents connect to the world
- OpenAI's body = computer use — Works with any GUI software, no vendor cooperation needed, no APIs required, no ecosystem buildout necessary
- Anthropic's body = structured interfaces — MCP servers, connectors, Conway — excellent architecture but dependent on the ecosystem shipping agent-ready integrations
- The Apple Shortcuts team made Codex work — OpenAI acquired Software Applications Inc. (creators of Workflow → Apple Shortcuts), and their deep macOS expertise is the reason Codex's computer use feels like a co-worker, not malware
- Chronicle is a training signal, not just memory — Screen captures teach the agent how you use your software, making computer use smarter over time — but not available in the EU/UK
- Conway is Anthropic's counter-bet — An always-on event-driven agent environment found in the Claude Code source leak, betting the world will build agent-native interfaces fast enough
- OpenAI is killing products to focus — Sora and drug discovery got cut because they don't ladder up to the three strategic vectors: agentic platform, computer work, personal AGI
- Acquisition pattern > benchmarks — Both labs are buying scarce human teams, not IP. The teams they acquire tell you more about where advantage will live than any model comparison
- If it has a screen, it's automatable now — The surface of what's possible widened dramatically; legacy enterprise software that was outside the automation conversation for years is now fair game
🔗 Resources & Links
- Full Open Source Summit Conversation — Referenced summit discussion linked in the video description