DATA Pill #045 - When Netflix and Google know you better than you know yourself - Recommendation Systems.

ARTICLES

Slack has developed a unified framework called the Recommend API, which allows the user to bootstrap new recommendation use cases behind an API, which is easily accessible to engineers at Slack.

Behind the scenes, these recommenders reuse a common set of infrastructure for every part of the recommendation engine, such as data processing, model training, candidate generation, and monitoring. This has allowed it to deliver a number of different recommendation models across the product, driving improved customer experience in a variety of contexts.

Netflix developed a new machine learning algorithm based on reinforcement learning to create an optimal list of recommendations considering a finite time budget for the user. In a recommendation use case, often the factor of finite time to make a decision is ignored; Netflix added this dimension to its recommendation system and in general in decision problems, in addition to the relevance of the recommendations.

How do you select the right query pins from the user’s profile? We added a new component to the Query Selection layer called Query Reward. Query Reward consists of a workflow that computes the engagement rate of each query, which we store and retrieve for use in future query selection. Therefore, we can build a feedback loop to reward the queries with downstream engagement.

How Pinterest improved the Homefeed engagement volume from a machine learning model design perspective — by leveraging real time user action features in the Homefeed recommender system.

An insight into analyzing and predicting "out of memory" or OOM kills on the Netflix App. The article explains the features used for training the model, including device specifications and user interactions with the application. It also discusses the challenges encountered during the model deployment and how they address them, such as offline evaluation and A/B testing.

How leveraging can graph information be broadly useful? Devin explains and discusses Airbnb's approach to implementing graph machine learning. This one will help you to understand how to leverage graph information to improve your models.

Well introduced RecSysOps with a set of best practices and lessons that Netflix Tech Team learned. It consists of four components:

• issue detection,
• issue prediction,
• issue diagnosis,
• issue resolution.

Look at patterns that are useful for anyone operating a real-world recommendation system to keep it performing well and improve it over time.

TUTORIAL

• How can TikTok recommend videos to you that were uploaded minutes ago?
• How can YouTube pick up on your brand-new interest immediately after you watched one video about it?
• How can Amazon recommend products based on what you currently have in your shopping cart?

From the pen of Jake who worked on the YouTube recommendation system:
An overview of online recommendation systems, the various approaches for building different subcomponents and offer some guidance to help you reduce costs, manage complexity and enable the team to ship ideas.

WHITE PAPER

Let’s explore the recommendation systems definition, how they work, how you can implement them, what benefits you can get from recommendations and how you should measure the performance and business value of recommender systems. At the end you can read about GID ML Framework – how it can help you in developing your recommendation systems.

VIDEOS

This video discusses a recommendation system, why it is valuable, and the challenges you may encounter when you build one. It will also introduce a few Google open source products related to recommendation systems, TF Recommenders, ScaNN, TF Ranking, and TFLite on-device recommendation model.

Watch one of essential Siddhardhan's projects and learn how to build a Movie Recommendation system using Machine Learning with Python.

PODCASTS

David Sweet, author of “Tuning Up: From A/B testing to Bayesian optimization”, introduces Dan and Chris to system tuning, and takes them from A/B testing to response surface methodology, contextual bandit, and finally bayesian optimization. Along the way, we get fascinating insights into recommender systems and high-frequency trading!

Deepak and Miguel discuss how recommendation systems are becoming increasingly important and can be used to increase revenue or click-through rates. It also covers algorithm advancements and how large-scale recommendations are implemented in real-time. This one is a great way to let you know more about the importance of personalized recommendations for companies, the growth of this field, and how companies can reinvest in their recommendation systems to improve their user experience and revenue.

CONFS EVENTS AND MEETUPS

Join the Gartner Data and Analytics Summit, where Gartner will provide solutions to your most significant challenges. This event will equip you to create and empower the innovative and adaptable organizations of the future.

• Implement strategies and innovations backed by data, analytics, and data science to navigate disruption.
• Lead purposefully by balancing trust, accountability, governance, and security with adaptability and responsiveness.
• Scale your purpose beyond organizational silos across value chains and ecosystems to foster societal perseverance.
• Leverage cloud-based data management to optimize costs in the current economic environment.