Abstract
The ratings of TV programs are irregular, and most of the viewers do not score every program they have watched, which leads to uneven distribution of user-program ratings matrix and low sparseness of the matrix in the recommendation system. For the recommendation algorithm, the sparsity of the input matrix has a great impact on the accuracy of the recommendation algorithm. Especially in the face of big data sets, the problems are enlarged. Aiming at these problems, a program recommendation algorithm based on LDA topic model and improved ALS collaborative filtering is proposed. This algorithm combines the program similarity matrix obtained from program features, preprocesses the score matrix to get the prescore matrix as input, and then dynamically weights the user and program features to improve the collaborative filtering algorithm to achieve the effect of “stable recommendation” and “multiple recommendation.” The simulation results show that the parallel operation of Spark MLlib algorithm library not only solves the problem of low timeliness of big data sets but also stabilizes the average RMES of hybrid recommendation algorithm at about 0.52. Compared with the traditional ALS collaborative filtering recommendation algorithm, the effect is significantly improved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li, Z., Lin, Y., Zhang, X.: Hybrid employment recommendation algorithm based on spark. J. Phys. Conf. Ser. 887, 012045 (2017)
Guo, C., Lu, M., Wei, W.: An improved LDA topic modeling method based on partition for medium and long texts. Annals Data Sci. (2019)
Cao, B., Liu, X., Liu, J., et al.: Domain-aware mashup service clustering based on LDA topic model from multiple data sources. Inf. Softw. Technol. 90, 40–54 (2017)
Xiao, Y., Zhong, R.: A hybrid recommendation algorithm based on weighted stochastic block model (2019)
Zhang, C., Yang, M.: An improved collaborative filtering algorithm based on Bhattacharyya coefficient and LDA topic model. In: International CCF Conference on Artificial Intelligence. Springer, Singapore (2018)
Chen, L.C.: An effective LDA-based time topic model to improve blog search performance. Inf. Process. Manage. 53(6), 1299–1319 (2017)
Geng, X., Zhang, Y., Jiao, Y., et al.: A novel hybrid clustering algorithm for topic detection on chinese microblogging. IEEE Trans. Comput. Soc. Syst. 6(2), 289–300 (2019)
Juan, W., Wei, X., Statistics, S.O.: Collaborative filtering recommender system based on matrix factorization and its application. Statistics & Decision (2019)
Yuan, Z., Ke, M., Weicong, K., et al.: Collaborative filtering-based electricity plan recommender system. IEEE Trans. Ind. Inform. 1–1 (2018)
Panigrahi, S., Lenka, R.K., Stitipragyan, A.: A hybrid distributed collaborative filtering recommender engine using apache spark. Proc. Comput. Sci. 83, 1000–1006 (2016)
Song, W., Shao, P., Liu, P.: Hybrid recommendation algorithm based on weighted bipartite graph and logistic regression. In: Artificial Intelligence. Springer, Singapore (2019)
Fengrui, Y., Yunjun, Z., Chang, Z.: Hybrid recommendation algorithm based on probability matrix factorization. J. Comput. Appl. (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Peng, A., Liu, H. (2021). Hybrid Program Recommendation Algorithm Based on Spark MLlib in Big Data Environment. In: Liu, Q., Liu, X., Li, L., Zhou, H., Zhao, HH. (eds) Proceedings of the 9th International Conference on Computer Engineering and Networks . Advances in Intelligent Systems and Computing, vol 1143. Springer, Singapore. https://doi.org/10.1007/978-981-15-3753-0_48
Download citation
DOI: https://doi.org/10.1007/978-981-15-3753-0_48
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-3752-3
Online ISBN: 978-981-15-3753-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)