Recommending Top N Movies Using Content-Based Filtering and Collaborative Filtering with Hadoop and Hive Framework

Abstract

Nowadays, the recommender system plays an important role in the real world by which we can recommend the most useful and perfect movies to the users from a large set of movies list and their ratings based on different users. Since the number of users and the movies are increasing day by day, computing the recommended movies list in a single node machine takes a very large time. Hence to reduce the computation time, we are using Hadoop framework to work in a distributed manner. Further, we have proposed a hybrid approach to recommend movies to the users by combining both the filtering techniques, i.e., user-based collaborative filtering and content-based filtering to overcome the problems of these techniques. In content-based filtering, we recommend items that are similar to the previous items which are highly rated by that user. Whereas in case of user-based collaborative filtering technique, we find out the most similar users with respect to the current user based on their cosine similarity and centered cosine similarity, and based on best similarity values, top N movies are recommended to the user by predicting the ratings of the movies. Further, to reduce the computation complexity, Hive database for Hadoop framework is used for developing SQL type scripts to perform MapReduce operations.