Large-Scale Recommendation and NLP Systems

Production recommendation at scale is a systems problem as much as a modeling one: raw user events must become reliable features, retrieval must be fast over large catalogs, ranking must model user interest dynamics, and everything must be served under latency budgets and iterated on continuously.

AI engineer owning workflows end to end — data collection and labeling, feature engineering, model training, deployment, and iterative optimization.

1. 01User Events — raw behavioral and content signals collected and labeled.
2. 02Feature Pipeline — filtering, preprocessing, and feature construction, including user-profile labels backed by a graph database.
3. 03Retrieval — two-tower models with vector search for fast candidate generation.
4. 04Ranking — interest-evolution models (DIN / DIEN) plus tree models and contrastive learning.
5. 05Recommendation API — served with CUDA / ONNX / TensorRT acceleration.

• Two-tower retrieval for scalability

  Decoupling user and item encoders lets item embeddings be precomputed and indexed, so retrieval scales to large catalogs via approximate nearest neighbor search.

• DIN / DIEN for the ranking stage

  Attention over behavior history (DIN) and interest evolution (DIEN) capture that user intent is sequential and context-dependent, which flat features miss.

• ONNX / TensorRT for inference

  Training frameworks are not serving runtimes. Exporting to ONNX and compiling with TensorRT plus CUDA cut inference latency to fit serving budgets.

• Graph-database-backed user profiles

  Modeling users and entities as a graph made label extraction, storage, querying, and regular updates tractable and directly consumable by the recommender.

Offline metrics for retrieval and ranking quality, plus production monitoring of click-through and retention as the optimization targets. Iterative optimization was driven by these signals across feature, model, and serving layers.

• Two-tower retrieval can miss long-tail items whose embeddings sit in sparse regions; hybrid retrieval and periodic re-indexing help.
• Feature/serving skew between offline training and online inference is a persistent risk that requires disciplined pipeline parity.
• Aggressive inference optimization (quantization, fusion) can shift outputs; numerical validation against the reference model is required before rollout.

• PyTorch
• DIN
• DIEN
• BERT
• CUDA
• ONNX
• TensorRT
• Knowledge Graphs