Microsoft Built a New Charting Language for AI Agents

Flint: Splitting Meaning From Geometry in AI-Generated Charts

Chart generation is secretly two jobs. Language models are good at one and bad at the other, which is why one in five generated charts arrives broken. Microsoft Research's answer is to stop asking the model to do the part it cannot do.

Video thumbnail
πŸ“Ί Better Stack ⏱️ 5:45 πŸ“… 27 July 2026
Flint Microsoft Research Vega-Lite MCP Intermediate representation Data visualisation

πŸ“‰ The one-in-five problem 0:00

The opening observation is one most people have hit without naming it. An agent will build an entire backend in minutes, then be asked for a single waterfall chart and hand back a hundred lines of Vega-Lite that renders a blank page. The instinct is to blame the model.

Microsoft Research's bet is that this is misdiagnosed. It is not a model problem, it is a language problem β€” and the fix is a new intermediate layer rather than a better prompt or a bigger model.

The number that motivated the project: the Flint team built their own analytics agent and measured roughly 80% success on good-looking charts. That sounds respectable in isolation. With real users on the other end, it means one chart in five arrives broken β€” which is an unshippable product, not a good score.

🧩 Meaning and geometry 0:35

The insight that makes the whole thing click: chart generation is secretly two jobs, and models are asymmetrically good at them.

JobWhat it involvesLLMs at it
MeaningThis column is a month, that one is revenue, that one is a percentage changeThey nail it
GeometryAxes, step sizes, scale domains, label spacing, colour rampsUnreliable β€” and this is where charts die

So Flint splits the work. The model writes about ten lines of JSON describing only meaning, using semantic types β€” quarter, price, rank, percentage change, and 70-plus others. Then a deterministic compiler takes over, makes every geometric decision, and emits a real Vega-Lite, ECharts or Chart.js specification.

"The model does meaning, the compiler does math. That's the whole trick."

Worth pausing on why that division is principled rather than convenient. Semantic classification is a judgement task with fuzzy inputs β€” exactly what language models are built for. Axis scaling is arithmetic with a correct answer, and a compiler that gets it right once gets it right every time. Flint routes each task to the machinery suited to it.

βš™οΈ Setup and first chart 1:45

Installation is a single line β€” claude mcp add flint. Pure TypeScript on Node 18, and the presenter notes nothing unusual on Apple silicon.

The demo: paste a CSV of monthly signups, ask for new users by month per game as a heatmap. Claude writes a ten-line Flint spec, calls render_chart, and the finished PNG lands in the chat within seconds β€” rendered locally, so the data never leaves the machine.

The part the presenter did not expect: a separate tool called create_chart_view opens an interactive panel. No re-prompting, no additional tokens. The model has finished its job and the chart remains editable β€” which is a meaningfully different interaction model from "the agent hands you an image and you ask it to try again".

🎨 What semantic types buy you 2:27

Everything hangs on the semantic types, and the demonstration is well chosen. Change one word β€” the column tagged quantity becomes percentage change β€” and the compiler:

  • Flips the entire colour scheme to a diverging palette
  • Rewrites the number formatting
  • Rescales the axis

One word in, a dozen correct decisions out. The framing the presenter offers is the one to remember: you tell Flint what the data is, never where the pixels go.

The second consequence is portability. Because the output is a genuine Vega-Lite, ECharts or Chart.js spec, the same input works with whichever backend you already use. There is no Microsoft rendering island β€” the compiled spec is yours to hand-tune. Flint positions itself as an intermediate language that gets you 95% of the way, not a replacement for anything.

A diverging palette is the right call for percentage change because the data has a meaningful zero point and readers need to see direction, not just magnitude. That is a real visualisation convention, correctly encoded β€” the example is not a toy.

πŸ†š Why not just Vega-Lite? 3:12

The obvious objection from anyone in data science, and the video meets it directly: Flint is not fighting Vega-Lite. It compiles to it.

The argument is about where the abstraction leaks. Vega-Lite feels high-level right up until you want the chart to look good β€” at which point a waterfall balloons past a hundred lines of coupled parameters. The same waterfall in Flint is about ten lines, because the compiler owns all the awkward parts.

AlternativeThe video's position
Vega-LiteNot a competitor β€” Flint's compile target. ~100 lines vs ~10 for a waterfall
MermaidDifferent job β€” flowcharts. "You don't ship Mermaid charts to customers"
Chart MCP servers26 separate tools, and they render your data on someone else's server
FlintFive tools, one schema, all local

⚠️ The caveats 3:56

To the video's credit, roughly a third of its running time is spent here rather than on the demo.

It is a research project, not a product

Around two months old, version 0.2.8 at time of filming, 8 contributors. They were still catching bugs that month where the same spec rendered differently in Vega-Lite and in ECharts β€” which, for a tool whose entire pitch is backend portability, is the bug that matters most.

The viral number has a cut in it

The claim doing the rounds β€” 100,000 tokens down to 200 per chart β€” is the author's own figure. Zero benchmarks, and at the time of filming the paper was still listed as coming soon.

The sharpest objection

Frontier models already one-shot simple charts perfectly well. If you are a power user with a big model, Flint solves a problem you do not have.

That is the honest framing, and it narrows the case considerably. What is left: small or cheap models, composite charts such as waterfalls and sunbursts, and any product where an end user sees the chart and 80% is not good enough.

Missing pieces

No maps. No 3D. No network graphs. No layering. Accessibility described in the video as an empty GitHub issue. And for Python developers, no package yet β€” so do not go looking for one.

🎯 Who it is actually for 4:30

Two adoption signals are offered, and they are the strongest evidence in the video precisely because they are not the author's own numbers.

First, over 5,000 npm downloads a week for a two-month-old research drop. Second β€” and more telling β€” Flint is already used by Data Formulator, Microsoft's own analytics tool. Which means Flint has a customer inside the building keeping it alive.

Why the internal-customer signal matters more than the download count: downloads measure curiosity, and a Microsoft Research release gets curiosity for free. A dependency in a shipping product from the same organisation means someone is obliged to keep it working. That is what usually separates research code that survives from research code that is archived in a year.

πŸ”­ The pattern underneath 5:13

The closing point is the one worth carrying away, and it is not about charts at all. The most-agreed-on reading was that agentic systems in general are converging on exactly this shape:

The LLM emits a tiny, validatable intermediate representation, and a deterministic layer does everything else.

The justification is a single clean sentence: you can validate ten lines of JSON before it even renders. You cannot validate a hundred lines of generated D3.

That generalises well beyond visualisation. Any domain where a model currently emits a large, fragile artefact directly is a candidate for the same split β€” have the model produce a small semantic description, and let deterministic code expand it. The error surface shrinks to something you can actually check.

πŸ” Claims checked

The video makes several checkable claims. Each was verified against the GitHub API, the npm registry, PyPI and arXiv:

ClaimResult
Microsoft Research, open source, TypeScriptConfirmed β€” microsoft/flint-chart, MIT, TypeScript
Five MCP toolsExactly five: render_chart, compile_chart, create_chart_view, validate_chart, list_chart_types
Over 70 semantic typesREADME states "70+"
Compiles to Vega-Lite, ECharts, Chart.jsConfirmed in the paper abstract
Used by Microsoft's Data FormulatorConfirmed β€” "flint-chart": "^0.2.1" in its dependencies
No Python package yetpypi/flint-chart returns 404 (the unrelated flint maths package is a different project)
Over 5,000 npm downloads a weekUnderstated β€” 11,309 last week
About two months old, version 0.2.8, 8 contributorsDrifted β€” created 13 May 2026, now at 0.4.1 with 9 contributors
Accessibility is an empty GitHub issueNot empty β€” issue #48 has a 579-character body, plus #62 on compiler-generated chart descriptions
Zero benchmarks; the paper is coming soonOutdated β€” the paper is published: arXiv 2607.20775, 22 July 2026, five authors
The substantive correction. The video says the paper is still coming. It is out β€” "Flint: A Semantics-Driven Data Visualization Intermediate Language", published 22 July 2026. That said, the underlying point partly survives: the abstract claims Flint "simplifies the authoring process without compromising on visual quality" but reports no percentages, no success rates and no comparative measurements. So the paper exists; a published benchmark of the 80%-to-something improvement still does not.

Version numbers, download counts and star counts move. Figures verified 28 July 2026.

πŸ’‘ Key takeaways

  1. Chart generation is two jobs, not one. Meaning β€” which column is a date, which is a percentage β€” and geometry: axes, scales, spacing, colour. Models are reliable at the first and unreliable at the second.
  2. 80% success is a failing grade in production. One broken chart in five is fine in a demo and unshippable in a product with real users on the other end.
  3. The split is the whole design. The model emits ~10 lines of semantic JSON; a deterministic compiler makes every geometric decision. Each task goes to the machinery suited to it.
  4. You declare what the data is, never where the pixels go. Retagging one column from quantity to percentage change flips the palette, the formatting and the axis in one move.
  5. It compiles to what you already use. Genuine Vega-Lite, ECharts or Chart.js output β€” no proprietary rendering island, and the compiled spec is yours to hand-tune.
  6. It is not competing with Vega-Lite. A waterfall is ~100 lines of coupled parameters there and ~10 in Flint, because the compiler owns the awkward parts.
  7. Rendering is local. Five tools and one schema, versus chart MCP servers with 26 tools that render your data on someone else's machine.
  8. Frontier models already one-shot simple charts. If you are a power user on a large model, this solves a problem you do not have. The real case is small models, composite charts, and end-user-facing output.
  9. The paper is out, but the benchmark is not. Published 22 July 2026 β€” yet the abstract carries no success rates or comparative numbers. Mechanism documented, effect still unquantified.
  10. An internal customer beats a download count. Microsoft's own Data Formulator depends on flint-chart. Downloads measure curiosity; a shipping dependency creates an obligation to keep it working.
  11. Know what is missing before adopting. No maps, no 3D, no network graphs, no layering, no Python package, and accessibility still open.
  12. The generalisable idea has nothing to do with charts. Have the model emit a small validatable intermediate representation and let deterministic code expand it. You can check ten lines of JSON before it renders; you cannot check a hundred lines of generated D3.

πŸ”— Resources & links

πŸ• Timestamp index

0:00The blank-page waterfall chart
0:12Not a model problem β€” a language problem
0:35Chart generation is secretly two jobs
0:49Geometry is where generated charts die
1:04Their own agent measured 80% success
1:17The split β€” 10 lines of semantic JSON
1:34The model does meaning, the compiler does math
1:45Setup β€” claude mcp add flint
1:56CSV to heatmap, rendered locally
2:13create_chart_view β€” editable, no extra tokens
2:27Change one word, get a dozen decisions
2:47What the data is, never where the pixels go
3:01No Microsoft rendering island
3:12Don't we already have Vega-Lite?
3:38Mermaid and the 26-tool MCP servers
3:56It's a research project, not a product
4:11The viral token claim has a cut in it
4:24The sharpest objection β€” frontier models
4:30Where the narrow case actually is
4:46No maps, no 3D, no Python package
4:58Adoption β€” npm and Data Formulator
5:13The pattern all agentic systems converge on
5:27You can't validate 100 lines of generated D3