WikiSkill: Evolving Skills for LLM Agents

🎯 The Problem

Published Aug 27, 2026 (Google Research). The core problem: valuable lessons are lost over time. Current approaches scatter error logs across messy run records, so programs forget important lessons whenever an experiment fails. 0:00

The main finding: a shared notebook boosts AI skills. "Maintaining a permanent reference library of past mistakes boosted accuracy, allowing smaller models to consistently beat much larger rivals."

🔗 arxiv.org/pdf/2608.27454 (paper) · github.com/ashutoshsinghpr7/wikiskill (reference implementation — Hermes Agent backend, MIT, pip install wikiskill)

🏗️ The Architecture

A four-step evolutionary loop with a workspace divided into three tiers. 1:15

Tier	What it holds
Raw execution traces	Immutable history
Persistent wiki	Consolidated patterns + failure causes
Executable skills	Validated procedures

The four-step loop: (1) inference agent executes training tasks → (2) wiki maintainer analyzes to consolidate patterns and failure causes → (3) skill proposer drafts targeted updates → (4) validation check adopts or rolls back without discarding the accumulated wiki knowledge.

📊 Results

Across five inference models and benchmark tasks, WikiSkill achieved the highest average accuracy — outperforming the strongest baseline by 3–12 percentage points (EvoSkill, SkillOpt). The yellow curve shows larger models benefit even more, widening the advantage on Gemini 3.5 Flash. 1:45

Smaller models with evolved skills can outperform larger baseline models without skills — "structured procedural knowledge significantly enhances capability without retraining."

🔀 Cross-Model Transfer

Table 2 tested whether skills evolved on one model can boost another. 2:10

The surprise: skills generated by stronger models often outperform self-evolved skills — e.g. Qwen 3.6 27B's skills boosted Qwen 3.5 9B on the spreadsheet benchmark from 34% to over 50%. Skills transfer across model families.

🔬 Ablation

Table 3 tested the role of persistent knowledge on Gemini 3.5 Flash. 2:38

Configuration	Average accuracy
Wiki access only for the skill proposer (default)	~64%
Wiki access disabled from the proposer	under 49%

Confirms: cumulative knowledge retention is essential for evolving robust procedural skills. The wiki must live in the proposer's loop.

📖 Case Study (ALF World)

Figure 3 shows wiki-guided evolution: the persistent wiki records rejected proposals, chronological logs, and recurring failure patterns; the active skill layer uses that history to draft a concrete skill called break-repetition-loop. 3:06

The payoff: because the proposer learns directly from prior failed attempts, it synthesizes an effective rule that prevents infinite action cycles.

💡 Key Takeaways

Compiling raw agent experience into persistent knowledge enables continuous skill evolution — without updating model weights.
Three tiers (traces / wiki / skills) with a validation-gated rollback that keeps failed knowledge.
Beats the strongest baseline by 3–12 points across five models and benchmarks.
Skills transfer across model families — and stronger models' skills often beat self-evolved ones.
Agents build compounding practical knowledge without retraining.

📍 Timestamp Index

0:00 Intro + the problem

1:15 Architecture + loop

1:45 Results (Table 1)

2:10 Cross-model transfer (Table 2)

2:38 Ablation (Table 3)

3:06 Case study (ALF World)

3:33 Wrap-up