Abstract
The rapid development of data has greatly impacted the storage, analysis, and processing performance of data on systems that use machine learning algorithms. Therefore, the aim of this research is to examine commonly used big data platforms and related machine learning libraries as well as classify recommendation systems. From there, we experimentally evaluate the accuracy of the recommendation system based on model-based collaborative filtering using Alternating Least Squares—ALS in Spark's Machine Learning Library (MLlib) for the 1 M MovieLens dataset. The experiment produces good results when evaluating the accuracy of the recommendation system based on Spark’s MLlib. Additionally, the report also examines changes in the precision of the recommender engine based on adjusting the parameters of the ALS algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bhatnagar A, Chittapur R, Rachakonda AR (2022)A large scale content ranking platform as applied to e-commerce store fronts. In: 2022 17th Annual system of systems engineering conference (SOSE), Rochester, NY, USA, pp 97–104. https://doi.org/10.1109/SOSE55472.2022.9812691
Yanwu Y, Kang Z, Daniel DZ, Bernard JJ (2022) Time-varying effects of search engine advertising on sales–An empirical investigation in E-commerce. Decis Support Syst 163:113843. https://doi.org/10.1016/j.dss.2022.113843. ISSN 0167-9236
Yanling L, Yejun X, Jing H, Ju W, Enrique HV (2022) Social network clustering and consensus-based distrust behaviors management for large-scale group decision making with incomplete hesitant fuzzy preference relations. Appl Soft Comput 117:108373. https://doi.org/10.1016/j.asoc.2021.108373
Alazzam H, AbuAlghanam O, Sharieh A (2022) Best path in mountain environment based on parallel A* algorithm and Apache Spark. J Supercomput 78:5075–5094. https://doi.org/10.1007/s11227-021-04072-0
Azeroual O, Nikiforova A (2022) Apache spark and MLlib-based intrusion detection system or how the big data technologies can secure the data. Information 13:58. https://doi.org/10.3390/info13020058
Ivan L, Krešimir P, Siniša S, Marin V (2022) A distributed geospatial publish/subscribe system on Apache Spark. Futur Gener Comput Syst 132:282–298
Apache Hadoop. https://hadoop.apache.org/. Last accessed 20 Feb 2023
Japec L et al (2015) Big data in survey research. Public Opin Q 79. https://doi.org/10.1093/poq/nfv039
Landset S, Khoshgoftaar TM, Richter AN, Hasanin T (2015) A survey of open source tools for machine learning with big data in the Hadoop ecosystem. J. Big Data 2(1):24. https://doi.org/10.1186/s40537-015-0032-1
Callegaro M, Yang Y (2017) The role of surveys in the era of ‘big data. Palgrave Handb Surv Res, pp 175–192. https://doi.org/10.1007/978-3-319-54395-6_23
Ngoan TT, Cang PA, Cang PT (2020) An approach to improve the recursive Join algorithm. In: Fundamental and applied information technology-FAIR, Vietnam
Sankaran SR (2017) Is spark better than hadoop map reduce? https://bigdata-madesimple.com/is-spark-better-than-hadoop-map-reduce/. Last accessed 28 Feb 2023
Deitel P, Deitel H (2019) Intro to python for computer science and data science. In: Learning to program with AI, big data and the cloud. Pearson
ProjectPro (2021) Spark MLlib for scalable machine learning with spark. https://www.dezyre.com/article/spark-mllib-for-scalable-machine-learning-with-spark/339. Last accessed 4 Apr 2023
Gosh S, Nahar N, Wahab MA, Shahadat M, Andersson K, Recommendation system for e-commerce using alternating least squares ( ALS ) on apache
Mankayarkarasi (2020) Introduction to spark MLlib for big data and machine learning. https://www.analyticsvidhya.com/blog/2020/11/introduction-to-spark-mllib-for-big-data-and-machine-learning/. Last accessed 20 Dec 2022
Nga HTT, Cuong ND (2017) Collaborative filtering recommender system and movielens dataset used for simulating the user-based nearest neighborhood algorithm. In: Conference on information and communication technology—ICT, Vietnam
Nghe NT (2016) Recommender systems: techniques and applications. Can Tho University Publisher, Vietnam
Chen R, Hua Q, Chang YS, Wang B, Zhang L, Kong X (2018) A survey of collaborative filtering-based recommender systems: from traditional methods to hybrid methods based on social networks. IEEE Access 6(c):64301–64320. https://doi.org/10.1109/ACCESS.2018.2877208
Collaborative Filtering. https://spark.apache.org/docs/latest/ml-collaborative-filtering.html. Last accessed 4 Feb 2023
McDade JE (2009) Matrix factorization techniques for recommender systems. IEEE Comput Soc. https://doi.org/10.1080/00220671.1937.10880693
Zadeh R (2015) Matrix completion via alternating least square (ALS). In: Databricks and Stanford. https://stanford.edu/~rezab/classes/cme323/S15/notes/lec14.pdf. Last accessed 10 Feb 2023
MovieLens. https://grouplens.org/datasets/movielens/. Last accessed 20 Mar 2023
Le Thi T, Tran TK, Phan TT (2023) Deep learning using context vectors to identify implicit aspects. IEEE Access 11:39385–39393. https://doi.org/10.1109/ACCESS.2023.3268243
Tran TK, Phan TT (2020) Capturing contextual factors in sentiment classification: an ensemble approach. IEEE Access 8:116856–116865. https://doi.org/10.1109/ACCESS.2020.3004180
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Nga, H.T.T., Thuy, A.N.T. (2024). Recommender System with Apache Spark. In: Swaroop, A., Polkowski, Z., Correia, S.D., Virdee, B. (eds) Proceedings of Data Analytics and Management. ICDAM 2023. Lecture Notes in Networks and Systems, vol 786. Springer, Singapore. https://doi.org/10.1007/978-981-99-6547-2_37
Download citation
DOI: https://doi.org/10.1007/978-981-99-6547-2_37
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6546-5
Online ISBN: 978-981-99-6547-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)