A production-grade multistage recommender system deployed on Kubernetes, combining two-tower retrieval, DLRM ranking, multimodal item embeddings, and a real-time behavioral personalization loop.
-
Towards Data Science: Deploying a Multistage Multimodal Recommender System on Amazon EKS featuring Bloom Filters, Feature Caching, and Contextual Recommendations
-
Medium article: Deploying a Multistage Multimodal Recommender System on Amazon EKS featuring Bloom Filters, Feature Caching, and Contextual Recommendations
Click to watch the Demo
A user request triggers a 14-stage ensemble served by NVIDIA Triton Inference Server on EKS:
- Feast user lookup — fetches user features (age, gender,
top_category) from a Valkey-backed Feast online store - NVT transforms — applies the same NVTabular preprocessing workflow used during training to user, item, and context features
- Two-Tower retrieval — encodes the user query and searches a FAISS index of item embeddings to retrieve the top-N candidates
- Bloom filter — removes items the user has already seen using a Valkey Bloom filter
- Feast item lookup — resolves item features (category, price, gender) from a numpy in-memory cache loaded at startup (~0.5ms vs ~195ms for a live Feast round trip)
- Multimodal embedding lookup — attaches CLIP image and sentence-transformer text embeddings (PCA-reduced to 64-dim each) to each candidate
- DLRM ranking — scores the filtered candidates; a reranker reranks and samples from the scored candidates and results are returned to the caller enriched with DynamoDB item metadata
top_category — the user's dominant item category over the past 24 hours — is updated in near real-time without retraining:
- When a user interacts with an item, the serving Lambda adds it to a Valkey sorted set (
user:{id}:recent_items) and enqueues an SQS message - The
recsys-feature-computationLambda triggers on that message, recomputestop_categoryfrom the sorted set, and writes it to both the Feast online store (Valkey) for immediate serving and the S3 offline store (Parquet) for the next incremental training run - The incremental training pipeline reads the S3 offline store to override stale Feast historical features, eliminating training/serving skew for
top_category
| Layer | Technology |
|---|---|
| Model serving | NVIDIA Triton Inference Server |
| Orchestration | Amazon EKS, Karpenter, Kubernetes HPA |
| Feature store | Feast (Valkey online store, S3 offline store) |
| Item metadata | Amazon DynamoDB |
| Serving Lambda | AWS Lambda + Function URL |
| Real-time features update | AWS SQS → Lambda → Feast |
| Training pipeline | Kubeflow Pipelines on EKS |
| Preprocessing and Training | NVIDIA NVTabular, Merlin-Tensorflow |
Full deployment instructions: Docs/documentation.md
@article{momoh2026multistage,
title={Deploying a Multistage Multimodal Recommender System on Amazon Elastic Kubernetes Service},
author={Momoh, Mustapha Unubi},
platform={Towards Data Science},
year={2026},
month={May},
url={https://towardsdatascience.com/deploying-a-multistage-multimodal-recommender-system-on-amazon-eks-featuring-bloom-filters-feature-caching-and-contextual-recommendations}
}- Ziyou "Eugene" Yan, System Design for Recommendations and Search, eugeneyan.com, Jun. 2021.
- Benedikt Schifferer, Solving the Cold-Start Problem Using Two-Tower Neural Networks for NVIDIA's E-Mail Recommender Systems, NVIDIA Merlin (Medium), Jan. 2023.
- Justin Wortz, Justin Totten, Scaling Deep Retrieval with TensorFlow Recommenders and Vertex AI Matching Engine, Google Cloud Blog, Apr. 19, 2023.
- Sam Partee, Tyler Hutcherson, Nathan Stephens, Offline to Online: Feature Storage for Real-time Recommendation Systems with NVIDIA Merlin, NVIDIA Technical Blog, Mar. 1, 2023.
- Jacopo Tagliabue, Hugo Bowne-Anderson, Ronay Ak, Gabriel de Souza Moreira, Sara Rabhi, NVIDIA Merlin Meets the MLOps Ecosystem: Building a Production-Ready RecSys Pipeline on Cloud, NVIDIA Merlin (Medium), Feb. 2023.