Q17Big Data Analytics
Question
What are Recommender systems?
Answer
A critical review of Recommender Systems. Explains how Netflix and Amazon utilize massive Big Data architectures to mathematically predict user preferences through Content-Based filtering and the highly complex matrix factorization of Collaborative Filtering.
Recommender Systems are the absolute financial engine of the modern internet. Amazon, Netflix, and YouTube do not wait for users to search; they use massive Big Data Machine Learning algorithms to mathematically predict exactly what the user wants to consume next, driving billions of dollars in revenue.
1. Content-Based Filtering
This algorithm mathematically analyzes the strict physical attributes (metadata) of the items the user has previously interacted with.
- Mechanism: If User A watches 5 movies that are tagged with the attributes
["Action", "Tom Cruise", "Explosions"], the algorithm mathematically creates a User Profile vector. It then scans a database of 10,000 unviewed movies and calculates the Cosine Similarity between the User Vector and the Movie Vectors, recommending the closest match. - Demerit: The "Filter Bubble." It will violently ONLY recommend Action movies. It physically cannot discover that the user might also like a Romantic Comedy.
2. Collaborative Filtering
This algorithm completely ignores the metadata of the movie. It strictly analyzes massive matrices of User-Item interactions (ratings, clicks).
- Mechanism: If User A likes Movies 1, 2, and 3. And User B likes Movies 1, 2, and 3. The algorithm mathematically determines User A and User B are a "Neighborhood." If User B suddenly watches and likes Movie 4, the algorithm violently forces Movie 4 into User A's recommendation feed, even if they have completely different metadata.
- Demerit: The "Cold Start" problem. If a brand new movie is uploaded, it has exactly zero user ratings. The Collaborative algorithm is mathematically blind to it and cannot recommend it to anyone until someone randomly clicks it.