Strategic Shift of Amazon’s Recommendations | AI/ML for Product Managers
Every Product Manager should know this
In the early days of e-commerce, the goal was clear:
Provide customers with personalized product recommendations.
The initial approach was user-based collaborative filtering.
User-to-User Collaborative Filtering
This method sought to identify users with similar purchase histories to predict what a given customer might like.
The logic was straightforward
If person A and person B have similar tastes, and person A buys product X, then person B might also be interested in product X.
However, this approach quickly hit a wall. There are primarily 3 reasons
Computational Bottleneck: Comparing every user’s purchase history with every other user’s became computationally infeasible.
The sheer volume of Amazon’s customer base made real-time recommendations extremely slow. This was because finding the group of customers whose purchase histories most closely resembled a given visitor’s would require comparing purchase histories across Amazon’s entire customer database.Data Update Issues: Customer purchase histories change frequently, sometimes daily. This meant that any similarity index based on user-to-user comparisons would become outdated very quickly, requiring constant and computationally intensive updates.
Scalability Issues: As Amazon’s customer base and product catalogue grew, the user-based approach became increasingly unsustainable. It could not scale to meet the demands of a rapidly expanding business.
User-to-User → Item-to-Item Collaborative Filtering
Faced with the limitations of user-based collaborative filtering, Amazon made a pivotal decision.
They turned the problem on its head and pioneered item-to-item collaborative filtering.
This approach focused on the relationships between products rather than the relationships between customers.
The core idea was that if customers who buy product A are also likely to buy product B, then product B should be recommended to customers who buy product A.
How it works:
The algorithm reviews a customer’s purchase history and, for each purchase, identifies a list of related items. The algorithm then flags items that repeatedly appear across those lists as potential recommendations. Items that appear repeatedly across these lists are candidates for recommendation to the visitor, weighted based on their relatedness to the visitor’s prior purchases.
This approach was much more computationally efficient.
It required far fewer lookups to identify the customers who bought a particular product than to identify customers who were similar to a given user. It was computationally feasible to update lists of related items daily.
Improved Recommendation Quality: Analyzing purchase histories at the item level, instead of the customer level, yielded better recommendations.
Further Refinement for Item-to-Item
The initial item-to-item approach, while a major improvement, still had room for refinement.
Simply counting how often items were bought together wasn’t enough. It led to the most popular products being recommended universally.
Amazon addressed this by creating a more nuanced metric for measuring “relatedness”:
Differential Probabilities: They calculated the probability of a customer buying product B given that they had already purchased product A, compared to the average customer’s likelihood of buying product B. If purchasers of A were more likely to buy B than the average customer, the items were considered related.
Discounting Heavy Buyers: The team realized they had to assess the increased likelihood of buying product B with any given purchase. This meant discounting the tendency of heavy buyers to purchase everything more frequently.
Learn about AI and Machine Learning (Don’t fall behind )
To understand such Tech Concepts and Machine Learning Case Studies. Download our Tech for Product Manager having Concepts and Case Studies about Artifical Intelligence
Technomanagers
More about PM Interview questions and Mock Interviews | YouTube | Website | Tech & Strategy Newsletter