The Semantic ID Layer : Rebuilding RecSys

Item embeddings are hitting real limits: embedding tables that dominate the parameter budget, brittle cold start, and representations that force models to memorize instead of generalize. This talk traces the shift to Semantic IDs — short, hierarchical sequences of discrete codes derived from content and learned with residual quantization.

This talk will briefly talk about how they're built, then make the case that the interesting move isn't tokenizing items, it's building two trees: one over content, one over users, sharing a vocabulary. From that foundation, seven retrieval sources fall out as tree walks — user-to-user neighborhoods, history-prefix expansion, a tunable exploration dial, prefix-frequency interest profiles, streaming trend detection, cold-start substitution, and prefix-level policy blocklists. Downstream, the same codes serve as ranking features and as generation targets.

The talk concludes with a pragmatic production architecture and an honest assessment of quantization loss, code collapse, semantic–behavior misalignment, identifier churn, privacy, and constrained-decoding latency. Semantic IDs do not eliminate dense embeddings; they turn embedding geometry into reusable structure.