Google WikiSkill Teaches Itself Your Company's Rules (No Fine-Tuning!)

🧠 The New-Employee Analogy

Google Research recently published a paper called WikiSkill that's doing the rounds in the industry. Despite the name, it's not a "skill" in the agentic sense — it's a framework. When you use it in your agentic applications, it gets the AI agent to learn an application the way a new employee learns a new job: make mistakes, correct them, take notes so they don't happen again — without going back to college to relearn everything.

The LLM driving the agent is not retrained and not fine-tuned. That's why the buzz. Picture a finance new hire who knows spreadsheets cold but still gets the first month wrong — because the mistakes come from company rules that are never written anywhere. A "case" means six units instead of the usual twelve; a product return goes in as a negative number. Nobody wrote these down, so no new hire — however smart — can learn them except by getting corrected. Now think of the AI agent as that new hire, except every time it gets a rule wrong, a human has to go fix the instructions. WikiSkill removes that human.

🏗️ The Four-Role Architecture

The paper's diagram is simpler than it looks: three co-workers and one strict manager, with a checker scoring each run. The worker is the agent doing the actual work — reading the messy files, writing itself a small script, running it, and saving a clean file. A checker compares the clean file against the correct numbers and gives it a score. Then the note-taker steps in: whenever the worker gets something wrong, it compares the wrong answers with the right ones and writes down what the rule must have been.

The playbook writer takes those notes and turns them into one change to the instruction file — a plain document that says "this is how you do this job at this company." Finally, the strict manager tests that change on ten spreadsheets the worker has never seen before and keeps it only if the score goes up; otherwise it's thrown out. Surya built a working version of this on his laptop using Google's Agent Development Kit (ADK), and the demo lives in the ADK web chat window.

🧾 The Messy Data & the Unwritten Spec

The demo dataset is a realistic monthly product-sales sheet: date, region, store, order ID, product type, units, and revenue. The dates are in several different formats, regions are spelled inconsistently, and units are cryptic — "3 CS" means three cases, but "case" itself is the trap (six units, not twelve). Returns are written as negative numbers, but only someone who knows the convention would recognize them as returns rather than errors.

The finance team's spec tells the agent to sort by date, then region, then order ID, and to "apply the standard finance conventions for dates, units, returns, duplicate orders, etc." — but nobody has written down what "standard" means. It's not a rule on paper; it's in someone's head. The only way to learn it is to do the job once, get verified, and be told what's right and wrong — which is exactly where the strict manager comes in.

❌ Running Without Skills

First, Surya asks the agent to clean the August file without the skills loaded — behaving like it's day one, with no playbook. The result: it identifies several rows with missing region values and flags key failures, but it can't figure out what actually needs to be done. No rules have been seen, so it just reads — and stalls on the ambiguities that the unwritten conventions would resolve.

That's the baseline. In a normal workflow, this is the point where a human would step in, spot the errors, and manually edit the prompt or instructions — the exact loop WikiSkill is designed to automate away.

🔀 RAG vs Fine-Tuning vs Memory

The obvious objection is "isn't this just RAG, fine-tuning, or memory?" The distinctions matter. RAG needs a document to look up — but here the rules aren't in any document, they're in someone's head. Fine-tuning takes days and you can't read what the model learned. Memory remembers, but nobody checks whether it remembered the right thing.

📈 The Results Dashboard

Surya gave the agent 20 spreadsheets: it practiced on 10 and was tested on the other 10 it had never seen. The scoreboard tells the story: the baseline learning process sat at 70%, then the 11th file scored 40% — rejected. The 12th was also rejected. (Iterations 3–6 failed for a mundane reason: Surya ran out of account usage — his setup, not the framework.)

Then on iteration seven it got 10 out of 10, and the manager accepted the change. The result is a skill file written in plain English that you can actually read — when to apply a rule, when not to, and the finance conventions, including rules the agent figured out by itself after learning and practicing on the unseen sheets. The agent is writing the instruction file every iteration and keeping it only once it converges on a passing score.

✅ Running With Skills

Same file, same model — but this time with the learned skill loaded. The result now passes: the agent correctly understands which entries are returns (negative revenue) and which are legitimate positives, and all rows match the expected ground truth perfectly. The skill it wrote in the previous step is what closes the gap — no human edited the instructions in between.

That before/after on identical input is the whole point: the framework didn't make the model smarter, it gave the model the company's rules, learned automatically and validated against unseen data.

🎯 Why It Matters & Who Should Use It

The distinction Surya draws: RAG and fine-tuning pre-prepare the model before it touches your company, while WikiSkill lets the agent learn the company's rules automatically, in place, with no external help. It takes a bit of time, but it's the most automated route to "minimum issues in the long run" — which makes it a strong fit for long-running agents.

Who should use it? Any agent doing repetitive work where the mistakes are about your company, not the world: month-end close where every store's export has its own quirks; invoice scoring where the right cost center is something only the accounts-payable team knows. On the build side, Surya described the experiment to AntiGravity (Google's agentic IDE) in plain English, uploaded the paper, and had it scaffold the app on ADK. A community MIT implementation already exists.

✅ Key Takeaways

1. WikiSkill is a framework, not a "skill." It teaches an agent a company's unwritten rules the way a new employee learns them — no retraining, no fine-tuning.
2. Four roles do the work. Worker (executes), note-taker (records what went wrong), playbook writer (compiles lessons into the instruction file), and a strict manager (keeps changes only if they score better on unseen data).
3. It co-evolves a wiki with the skills. The paper separates raw execution experience from the persistent, readable knowledge that survives across iterations.
4. It targets rules that live in someone's head. "Case = 6 units," "returns are negative" — conventions no document contains, so RAG can't find them.
5. Readable, verifiable, gated. Unlike fine-tuning (opaque) or memory (unchecked), the learned skill is plain English you can read, and it's kept only if it actually helps.
6. The demo converged. Baseline 70% → rejected iterations → iteration 7 hit 10/10, and the agent even derived rules on its own.
7. Best for repetitive, company-specific work. Month-end close quirks, invoice cost-center scoring — and long-running agents.
8. It was built with Google's own stack. AntiGravity scaffolded the ADK app from the paper; an MIT community implementation exists.

🔗 Resources & Links

• 📄 WikiSkill paper — "WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution" (Liyan Tang et al., Google Research)
• 🧪 Community implementation (MIT)
• 🛠️ Google ADK — the Agent Development Kit the demo was built on
• 🧭 Google AntiGravity — the agentic IDE that scaffolded the app
• ▶️ AI with Surya — the channel

Framework, paper title, and authors verified against the arXiv record; AntiGravity and ADK verified against Google's own pages. The caption track renders a few names loosely ("Fang" → FAANG, "anti-gravity" → AntiGravity, "rag" → RAG).

🕒 Timestamp Index