WWDC26 — Run Local Agentic AI on the Mac Using MLX

📋 Overview

At WWDC26, Apple engineer Angelos from the MLX team demonstrates how to build and run fully local agentic AI workflows on the Mac — no cloud, no API keys, just your hardware. The session introduces a four-layer open-source stack (MLX → MLX-LM → MLX-LM Server → Agent), shows how the M5 chip's Neural Accelerators deliver 4× faster prompt processing, demonstrates continuous batching for parallel subagents, and walks through distributed inference across multiple Macs over Thunderbolt RDMA. Live demos include an agent building a SwiftUI drawing app from scratch in 2 minutes and fixing bugs directly in Xcode — all powered by a locally-running model.

1 Chat vs Agent — The Agentic Loop

▶ 0:21

Angelos begins by contrasting the traditional chat experience with the agentic paradigm:

Traditional Chat

• You send a prompt → model sends a response
• If you need to act on the response (run a command, check a file, fix an error) — that's on you

The Agentic Loop

• User → Agent: You describe what you want
• Agent → Model: The agent consults the LLM to decide what to do
• Agent → Tools: The agent calls tools — running commands, reading files, hitting APIs
• Observe → Loop: The agent reads tool results, goes back to the model for the next step
• This cycle keeps repeating until the task is done

2 Live Demo — Local Agent Summarizing PRs

▶ 1:24

The first demo shows an agent running entirely locally on a Mac. The screen shows two panes:

• Left: MLX running the model
• Right: The OpenCode agent interface

Angelos asks the agent to fetch recent pull requests from the MLX GitHub repository, summarize the changes, and identify anything that needs his attention. The agent:

1. Reasons about the request
2. Calls the GitHub CLI to fetch PR data
3. Reads through the diffs
4. Produces a concise summary

Everything runs locally — the model runs on his hardware and only the git commands reach the network. The LLM inference, reasoning, and summarization all happen on-device.

3 The Four-Layer Local Agentic AI Stack

▶ 2:19

The stack that powers local agentic AI on the Mac has four layers, each building on the one below:

Layer 1: MLX (Foundation)

Apple's open-source array framework purpose-built for Apple Silicon. Handles all low-level computation, Metal acceleration, and memory management. Everything else is built on this.

Layer 2: MLX-LM (Language Models)

Provides everything needed to load, run, quantize, and fine-tune large language models. Supports thousands of models from HuggingFace. Offers both CLI tools and a Python API.

Layer 3: MLX-LM Server (Serving Layer)

An OpenAI-compatible HTTP server that exposes your local model through a standard API. Key capabilities:

• Structured tool calling — the model can invoke functions reliably
• Reasoning models — step-by-step analysis before responding
• Drop-in replacement for any cloud LLM API

Layer 4: Agent (Application Layer)

Any framework or tool that speaks the OpenAI chat completions protocol: Xcode, OpenCode, Pi agent, custom scripts, or anything else. Because MLX-LM Server provides a standard interface, any agent framework works out of the box.

The Broader Ecosystem

It's not just Apple building on this stack. Several popular tools build on MLX and MLX-LM: Ollama, LM Studio, and vLLM are among the most popular. The ecosystem is broad and growing — if you're using one of these tools, chances are you're already running on MLX.

4 Three-Step Setup — Zero to Local Agent

▶ 4:36

Getting a fully local agentic workflow running takes only three steps:

Step 1: Install MLX-LM

Step 2: Start the Server

Start with a small model to test your setup. The server loads the model and accepts requests on localhost.

Step 3: Point Your Agent at the Local Server

In most agent frameworks, you just set the base URL to your local server's address and you're done. The agent doesn't know or care that the model is running on your Mac rather than in the cloud.

Concrete Example: OpenCode Configuration

Define a local provider, set the URL to localhost, set the model name the server expects, and tell OpenCode to use the local model for everything. That's it — every interaction now runs through your local model.

5 Challenge 1 — Prompt Processing & M5 Neural Accelerators

▶ 5:48

The first key challenge for local agentic AI is prompt processing speed. In an agentic workflow:

• Every time the model receives tool output, it must process all that new context before it can reason about the next step
• This happens over and over throughout the agentic loop and adds up fast
• Agentic sessions typically comprise hundreds of thousands of tokens, and most of those are input tokens (not generated)

M5 Neural Accelerators

The M5 chip introduces dedicated Neural Accelerators, and MLX can target them for exactly this kind of work:

• Neural Accelerators make matrix multiplication 4× faster on M5 compared to M4
• With specialized multiplication and attention kernels in MLX, this translates almost exactly to prompt processing speedup
• Agents can read your codebase or process tool results almost 4× faster

6 Challenge 2 — Concurrency & Continuous Batching

▶ 6:57

In practice, agents rarely work alone. A common pattern is for an agent to spawn several subagents, each tackling a different part of the problem in parallel:

• One might be reading documentation
• Another searching code
• A third writing tests
• All at the same time — meaning multiple requests hit your local model simultaneously

Continuous Batching in MLX-LM Server

Instead of processing requests one at a time, MLX-LM Server handles this with continuous batching:

• Dynamically groups incoming requests into batches and processes them together on the GPU
• New requests can join a batch in progress without waiting for the current one to finish
• The result: subagents don't stall waiting in a queue — they all get served concurrently, keeping the entire agentic workflow moving

7 Challenge 3 — Distributed Inference Across Macs

▶ 7:46

Sometimes a single machine — even one with 512GB of RAM — isn't enough. The most recent DeepSeek model, for example, has 1.6 trillion parameters and requires more than 800GB of memory just for the weights.

MLX Distributed Support

MLX lets you spread a model across multiple Macs connected over Thunderbolt or Ethernet. For agents, this is powerful in two ways:

• Run much larger, more capable models that wouldn't fit on a single machine
• Parallelize prompt processing across devices, directly speeding up the agentic loop

Setup

Launch the server with mlx.launch and a hostfile containing node information and connection types. The model is automatically sharded across all available devices.

Thunderbolt RDMA (macOS 26.2)

Starting with macOS 26.2, Apple introduces support for Thunderbolt RDMA — low-latency, high-bandwidth communication over Thunderbolt. This delivers significant speed-ups for distributed inference: up to 3× with four nodes.

8 Demo — Building a SwiftUI App from Scratch

▶ 9:29

In the second live demo, Angelos asks the local agent to build a SwiftUI drawing app for iPad from a blank Xcode project. The agent's workflow:

1. Inspects the current directory to find the existing project structure
2. Makes a plan to guide its implementation
3. Writes the code — no manual copying required
4. Builds the app using xcodebuild
5. Fixes build errors it encounters along the way

The result: a fully functional drawing app created in ~2 minutes. Angelos then iterates on it, asking the agent to add rounded end caps to the drawing strokes. The agent edits the code, recompiles until it compiles without errors, and the updated app is ready to test.

9 Demo — Bug Fixing in Xcode with Local AI

▶ 11:39

The final demo shows integration with Xcode itself. The setup:

1. Open Xcode Settings → Intelligence tab
2. Click Add Chat Provider → Locally Hosted
3. Set the port to 8080 (or whichever port the MLX server is using)
4. Done — Xcode can now talk to your local model

Angelos introduces a bug into the previously working drawing app, then asks the local model to fix it. Within seconds, the model:

• Identifies the bug
• Inspects the surrounding code
• Writes a targeted fix

The app builds and runs successfully. This demonstrates how a locally running agent can integrate with your existing Xcode development workflow — reading project files, understanding build errors, and making targeted fixes — with your code never leaving your Mac.

🎯 Key Takeaways

🔗 Resources & Links

• MLX on GitHub — Apple's open-source array framework for Apple Silicon
• MLX-LM on GitHub — Language model tools: load, run, quantize, fine-tune, and serve LLMs
• mlx-lm on PyPI — Install with pip install mlx-lm
• WWDC26 Sessions — Full session catalog including "Explore distributed inference and training with MLX"

⏱ Timestamp Index