0:12Freddy Lecue — Wells Fargo's Head of Frontier AI Models — walks through the bank's first foundation model. The twist: it's not language-based. Financial services run on tabular data — customer transactions — and Lecue's argument is that dumping all of that into a language model isn't the best way to use it. So Wells Fargo trained a transaction foundation model: one backbone, pretrained on the raw stream of customer transactions, fine-tunable for fraud detection, credit decisions, anti-money laundering, or marketing.
Why transactions, not words
3:22The playbook mirrors language modeling — but the "base" is transactions, not text. A language model learns representations by predicting the next token; the transaction model learns representations by predicting the next transaction, using every channel (debit, credit, digital, check). Predicting the next transaction is never the goal — it's the pretext. Once the representation is learned, you fine-tune it for downstream tasks, exactly like you'd post-train a language model for classification. The stated ambition: move from "many siloed models across organizations" to one shared backbone that every team reuses because it already encodes customer and transaction behavior at scale.
Why not XGBoost
2:14The obvious objection — "it's tabular data, why not just use XGBoost?" — is the one they already lived. XGBoost is the production fraud model today. The new model beats it on both latency and quality: higher detection rate and lower false positives. The missing piece Lecue points to is relationships — an XGBoost model sees a transaction as a row of features and loses its relationship to all the customer's past transactions unless you hand-engineer that context in. A transformer's attention, by contrast, attends from a transaction to the history of past transactions — and that's what he credits with pushing the detection rate up.
The data
2:47Scale: billions of transactions monthly, millions of accounts, hundreds of transactions per client per month. Lecue frames each transaction on five dimensions — who, what, how, when, where: who's asking and who's providing (customer vs. merchant), what's being bought, which channel, and the time and location. 5:58The existing production model leans on heavy hand-crafted feature engineering. The shift: step back and find the raw features — the real information units — rather than pre-built engineered ones, and let the transformer figure out the relationships between them. That distilled down to roughly 100 raw features (from thousands).
Tokenization
7:59Lecue is blunt about where the real work was: "the most complex part was the tokenization." Transaction features are heterogeneous — numerical (transaction amount, delta-time between transactions), categorical with wildly different cardinalities (some with 8 types, some far larger), and structured (IP address, postal/zip code). Do you encode every zip code as its own token? Too huge a vocabulary makes training messy; too coarse loses signal. The answer is a value-to-token design that groups zip codes by distribution, decides what information belongs in a single token, then maps transaction + characteristics → token ID → embedding lookup. He calls the recipe's secret sauce "how you do the tokenization, and then how you do the post-training" — the architecture itself is "nothing fancy."
Architecture & scale
9:42It's a decoder-only transformer — predict the next transaction, standard causal language modeling. They used Chinchilla scaling math to size it, and deliberately did not chase a trillion parameters: 1B parameters was the smallest model that gave the highest boost, and a 11B version is in the works. On data/token ratio, they started assuming ~20× parameters in tokens (Chinchilla's guidance) and ended up needing roughly 10× — "much more tokens per parameter" — found through experimentation.
From scratch
4:45Lecue stresses this because it's commonly confused: pretrained from scratch — zero open weights, only the architecture borrowed — trained on Wells Fargo's GPU cluster over a couple of weeks. 12:10The training recipe is standard best practice: mixed precision, distributed data parallel, maximized micro-batch for faster convergence, RoPE, FlashAttention, activation checkpointing — "nothing really new, just advice from the people who built much bigger models." Evaluation is perplexity plus accuracy dashboards on downstream tasks.
Results vs XGBoost
13:44The downstream task shown is fraud classification: given a sequence of a customer's transactions, is this next one fraudulent against their history? Two metrics matter, and both map directly to dollars: detection rate and false positive rate. Wells Fargo's result: more fraudulent transactions flagged with fewer false positives than the production XGBoost model — a false decline means a real customer's card gets blocked, they call, and there's an operational cost plus a lost transaction. Lecue calls it "millions of dollars in savings." Latency is the hard constraint — a card decline must complete in under 250 ms, and the model ran ~10× faster than that. (No public model or paper — this is proprietary bank work.)
What's next
16:47The one-backbone pitch is the point: the same pretrained foundation is already being used for fraud detection, and Wells Fargo is now investigating credit-line increase, anti-money laundering, and marketing — each a fine-tune on the shared representation rather than a new from-scratch model. The takeaway for enterprise AI: for tabular domains, the foundation-model playbook ports over, but the moat isn't the architecture — it's the tokenization and the post-training.
Claims checked
The speaker and his role are real. Freddy Lecue is Managing Director and Head of Frontier AI Model Methodology at Wells Fargo. The talk is "Wells Fargo Foundation Model: Application to Fraud Detection at Scale" from AI Engineer NYC 2026.
There is no open model, paper, or repo to independently verify the numbers against — the 1B-parameter size, the "millions in savings," and the fraud-rate improvement are the speaker's own reported results, not independently audited. Wells Fargo's proprietary data means the detection-rate and false-positive figures can't be checked externally.
The "the moat is tokenization, not architecture" framing is this summary's synthesis of Lecue's emphasis, not his words.