Why Retail AI Fails at the Vocabulary Layer

Every retail AI pitch starts the same way. Demand forecasting. Personalized recommendations. Dynamic pricing. The models are getting better, the compute is getting cheaper, and the demos look incredible.

Then you plug in real catalog data from 2,000+ stores across 15 channels, and the whole thing collapses. Not because the models are weak, but because nobody agrees on what anything is called.

A "slim fit navy chino" in one brand's catalog is a "navy skinny trouser" in another. A third calls it a "chino pant, dark blue." Same product. Three descriptions. Three taxonomies. Three sets of attributes that don't map to each other. Before you can forecast demand, recommend alternatives, or allocate inventory, you need to solve this: a shared vocabulary across brands, channels, and regions. This is the unsexy foundation nobody wants to build. It's also the one that determines whether everything downstream works or doesn't.

In this post:

1. The problem nobody talks about: why vocabulary fragmentation kills AI before it starts
2. What fragmentation actually looks like: concrete examples across naming, sizing, color, and category
3. Why this kills AI at scale: garbage taxonomy in, garbage recommendations out
4. This is a context engineering problem: the same patterns apply
5. What actually works: extraction, normalization, governance
6. The universal pattern: every domain with fragmented naming has this problem
7. The vocabulary layer is the product: why this deserves investment, not workarounds

---

The problem nobody talks about

Search for "retail AI" and you'll find hundreds of posts about recommendation engines, demand forecasting models, and pricing optimization algorithms. You'll find almost nothing about catalog normalization.

This is a blind spot, and it's a costly one.

The retail AI stack has an implicit assumption baked into every layer: that the data feeding the models uses consistent, structured vocabulary. That a "trouser" in Brand A's system means the same thing as a "trouser" in Brand B's system. That "navy" in one channel maps to "navy" in another. That size "M" is the same "M" everywhere.

None of that is true. Not at one brand, not across brands, and definitely not across channels and regions.

I work on a retail platform that handles 2,000+ stores across 15 channels. Each brand onboards with its own catalog structure, its own naming conventions, its own attribute schema. Some have rich, structured product data. Others have a title field, a description field, and nothing else. The variance isn't an edge case. It's the default state.

When you try to build AI on top of this, the models don't fail dramatically. They fail quietly. Recommendations look plausible but miss because "slim fit" in one brand doesn't map to "slim fit" in another. Demand forecasts are wrong because the same product category has four names. Inventory allocation breaks because the system can't tell that two SKUs are actually the same item in different packaging.

The model isn't the bottleneck. The vocabulary is.

---

What fragmentation actually looks like

This isn't abstract. Here's what vocabulary fragmentation looks like in practice, across four dimensions that every multi-brand retailer deals with.

Product naming

The same physical product, described differently by three brands:

Brand	Product title	Category	Key attributes
Brand A	"Slim Fit Navy Chino"	Men > Trousers > Chinos	fit: slim, color: navy, type: chino
Brand B	"Navy Skinny Trouser"	Menswear > Bottoms	fit: skinny, color: navy, type: trouser
Brand C	"Chino Pant - Dark Blue"	Men's > Pants	color: dark blue, type: chino pant

Three brands. Three category hierarchies. "Slim" vs. "skinny" vs. no fit attribute at all. "Navy" vs. "dark blue." "Chinos" vs. "trousers" vs. "pants." A human can see these are roughly the same product. An algorithm cannot, unless you build the translation layer.

Size systems

Region	Notation	What it means
US	32x30	Waist 32 inches, inseam 30 inches
UK	32R	Waist 32 inches, regular length
EU	48	Continental sizing (maps roughly to US 32)
Brand-specific	"M"	Varies by brand. Could be 30-34 waist

A customer searching for "size 32 navy chinos" expects to find all matching products regardless of brand. Without a normalization layer that maps US 32 to UK 32R to EU 48, your search and recommendation engines are blind to 60-70% of matching inventory.

Color naming

This one is deceptively hard:

• Navy, dark navy, deep navy, navy blue, midnight navy: five names for what is often the same color
• Stone, sand, khaki, beige, oatmeal, tan: six names for a spectrum that most customers think of as two or three colors
• Burgundy, wine, maroon, oxblood, berry, plum: overlapping terms with no industry standard mapping

Colors have emotional and brand-specific meanings. A premium brand calls it "midnight." A fast-fashion brand calls it "dark blue." Both are valid descriptions. Neither is useful for cross-brand inventory visibility or recommendation.

Category taxonomies

Every brand builds its own category tree. Some are three levels deep. Others are seven. The mapping problem is combinatorial:

• Brand A: Men > Trousers > Chinos > Slim Fit
• Brand B: Menswear > Bottoms > Casual > Trouser
• Brand C: Clothing > Men's > Pants
• Brand D: Apparel > Male > Lower Body > Chino Pant

Four different hierarchies, four different levels of granularity, four different naming conventions. Every downstream system that needs to aggregate across brands (search, recommendations, analytics, inventory) has to pick one or build a mapping for all of them.

---

Why this kills AI at scale

The vocabulary problem doesn't just make data messy. It makes AI unreliable in specific, predictable ways.

Recommendations break. A recommendation engine trained on Brand A's catalog learns that "slim fit chinos" and "tailored trousers" are related. But when it encounters Brand B's catalog where the same products are called "skinny pants" and "dress bottoms," those learned associations don't transfer. The model either recommends nothing (low recall) or recommends wrong items (low precision). Both outcomes destroy customer trust.

Demand forecasting hallucinates. If "navy chinos" appear under three different names across three brands, the forecasting model sees three separate products with low individual demand instead of one product with high aggregate demand. It under-orders for all three. The result: stockouts on a product that actually has strong demand, invisible to the system because the vocabulary fragmented it.

Search becomes unreliable. A customer searches "blue trousers." Brand A has them indexed as "navy chinos." Brand B has them as "indigo pants." Brand C has them as "blue trousers" exactly. Only Brand C shows up. The other two might as well not exist.

Analytics lie. Category performance reports show "chinos" growing 15% and "trousers" declining 8%. In reality, they're measuring the same products categorized differently across brands. Strategic decisions based on these reports allocate investment to the wrong categories.

The pattern is consistent: models trained on inconsistent labels produce inconsistent output. This isn't a model quality problem. It's a data quality problem that no amount of model improvement can fix. You can upgrade from a basic collaborative filter to a transformer-based recommendation engine, and it will still fail if "slim fit" doesn't reliably map to "slim fit" across your catalog.

---

This is a context engineering problem

If this sounds familiar, it should. The vocabulary problem in retail is structurally identical to the context engineering problem in AI development.

When I write about context engineering, the core argument is: the quality of AI output depends on the quality of context you provide. Not the prompt. Not the model. The context: the structured understanding of your world that the system operates within.

Retail vocabulary normalization is the same problem at the data layer:

Context engineering (AI dev)	Vocabulary engineering (retail)
Teach the model your domain, constraints, and conventions	Teach the system your product taxonomy, naming rules, and attribute schemas
Encode failures as guardrails in CLAUDE.md	Encode edge cases as mapping rules in the normalization layer
Without context, models produce generic output	Without vocabulary, models produce fragmented output
The spec defines what "correct" looks like	The canonical taxonomy defines what "same product" means
Skills encode repeatable expertise	Extraction models encode repeatable attribute parsing

The parallel runs deeper. In context engineering, I treat constraints as first-class ingredients because they shape the solution space. In vocabulary engineering, mapping rules are first-class ingredients because they shape what the AI can see. Without them, the AI operates on a fractured view of reality. With them, it operates on a unified one.

This is also why I think of vocabulary normalization as a spec-driven problem. You need to precisely define what "correct" means: what the canonical terms are, what the acceptable mappings are, what the hierarchy looks like. Ambiguity in the spec produces ambiguity in the output. That's true whether you're specifying software behavior or product taxonomy.

---

What actually works

After working through this problem across multiple brands and channels, here's what I've found actually moves the needle. None of it is glamorous. All of it is essential.

1. Attribute extraction pipelines

The first step is extracting structured attributes from unstructured product descriptions. A title like "Slim Fit Navy Chino in Stretch Cotton" needs to be parsed into:

• Fit: slim
• Color: navy
• Type: chino
• Material: stretch cotton

This used to require massive regex libraries or manual mapping tables. Now, LLMs handle the extraction with high accuracy if you give them the right context (there's that word again). The trick is providing the model with your canonical attribute list so it maps to known values instead of inventing new ones.

What works:

• Few-shot examples of product titles mapped to attributes
• A canonical list of valid values for each attribute (not open-ended extraction)
• Confidence thresholds: high-confidence extractions go through automatically, low-confidence ones queue for human review
• Continuous feedback: human corrections retrain the extraction model

What doesn't work:

• Rule-based extraction alone (too brittle for natural language variation)
• Open-ended extraction without canonical constraints (produces new synonyms faster than you can map them)
• One-time batch jobs (catalog data changes constantly)

2. Normalization layers

Extraction gives you structured attributes. Normalization maps those attributes to canonical terms. This is the translation dictionary:

• slim, skinny, narrow, fitted → slim_fit
• navy, navy blue, dark navy, midnight navy → navy_blue
• chino, chinos, chino pant, chino trouser → chinos

The normalization layer is a living system, not a static mapping file. New brands bring new terms. Seasonal collections introduce new colors. The layer needs to grow without breaking existing mappings.

Key design choices:

• Canonical IDs, not strings. Never depend on display names internally. Use canonical identifiers (color:navy_blue) and map display names separately per channel and locale.
• Hierarchical normalization. Normalize at multiple levels: "midnight navy" → navy_blue (specific) → blue (general). Different downstream systems need different levels of granularity.
• Confidence scoring. Not all mappings are certain. "Burgundy" to red is a judgment call. Score the mapping confidence and let downstream systems decide their tolerance.

3. Taxonomy governance

This is the part that requires organizational discipline, not just technology. Someone has to own the canonical taxonomy. Someone has to decide whether "wine" maps to burgundy or red. Someone has to resolve conflicts when two brands use the same term for different things.

The governance model that works:

• A canonical taxonomy with explicit rules for addition, modification, and deprecation
• Version control on the taxonomy so changes are traceable and reversible
• Automated conflict detection that flags when a new brand's terms create ambiguous mappings
• Regular review cycles (quarterly) where the taxonomy team audits drift and resolves accumulated edge cases

Without governance, the normalization layer accumulates technical debt. Mappings contradict each other. Edge cases pile up. Within a year, the normalization layer is as fragmented as the original catalogs.

---

The universal pattern

This isn't just a retail problem. Every domain with fragmented naming conventions has the same structural challenge. The specifics change, the pattern doesn't.

Domain	The vocabulary problem	What breaks without normalization
Healthcare	ICD-10 codes vs. clinical notes vs. insurance claims. "Type 2 diabetes" has 20+ documentation variants.	Clinical decision support, billing accuracy, population health analytics
Finance	Instrument classifications across exchanges: same equity listed under different tickers, different naming conventions, different regulatory categories	Portfolio analytics, risk aggregation, compliance reporting
Logistics	SKU systems across warehouses, carriers, and customs systems. Same item, different identifiers at every handoff	Inventory visibility, shipment tracking, demand planning
Manufacturing	Bill-of-materials naming. "Stainless steel 304" vs. "SS304" vs. "AISI 304" vs. "18/8 stainless"	Procurement aggregation, quality traceability, supplier matching
E-commerce	Marketplace listings across Amazon, Shopify, and direct channels: different title formats, different attribute schemas, different category trees	Cross-channel analytics, unified inventory, consistent customer experience

The underlying issue is always the same: multiple systems describing the same reality with different vocabularies, and downstream AI systems that can't function without a shared one.

This is why I think of vocabulary normalization as a context engineering problem, not a data engineering problem. Data engineering gives you clean pipes. Context engineering gives you shared understanding. The vocabulary layer is about building shared understanding across systems that were never designed to talk to each other.

If you're interested in how this connects to the broader challenge of running 2,000+ stores as a system rather than a collection of point solutions, the vocabulary layer is the connective tissue. Without it, every store is an island with its own dialect. With it, you have a platform that can reason across the entire network.

---

The vocabulary layer is the product

Here's the reframe that changed how I think about this problem: the vocabulary layer isn't infrastructure. It's the product.

Every AI capability you want to build on top of retail data (recommendations, forecasting, allocation, search, analytics) depends on vocabulary quality. The recommendation engine is only as good as the product graph it operates on. The forecasting model is only as accurate as the demand signals it aggregates. The search index is only as complete as the attribute coverage across your catalog.

Investing in vocabulary normalization doesn't feel like building a product. It feels like cleaning up someone else's mess. There's no demo that makes a taxonomy mapping pipeline look exciting. Nobody puts "we normalized 40,000 product attributes across 15 brands" on a conference slide.

But teams that skip this work and go straight to the flashy AI layer discover the same thing within six months: the models don't work reliably, and no amount of model tuning fixes it. The bottleneck was never the model. It was the vocabulary.

If you're building AI for any domain with fragmented naming, start here. Build the extraction pipeline. Build the normalization layer. Invest in taxonomy governance. Treat the vocabulary as a product with its own roadmap, its own quality metrics, and its own team.

The models will get better on their own. The vocabulary won't build itself.

---

This post connects to context engineering patterns (the same techniques apply at the data layer), spec-driven engineering (encoding intent precisely for taxonomy rules), and evals for AI agents (measuring normalization quality).