国内大数据赋能的智慧推荐资源管理技术研究进展

doi:10.13998/j.cnki.issn1002-1248.24-0044

摘要/Abstract

摘要： [目的/意义]为资源化数据并支持智慧推荐服务多样化、提升用户交互体验,系统分析国内大数据赋能的用户兴趣建模、项目信息管理技术研究成果,阐述智慧推荐中用户及项目信息规范管理的技术体系。[方法/过程]文章用文献研究法归纳了507篇文献内容,从用户兴趣建模、项目信息管理两方面对国内大数据赋能的智慧推荐资源管理技术进行总结分析：前者包括模型表示、初始化、进化,后者包括数据采集、挖掘、存储、更新、解释。[结果/结论]大数据赋能的用户兴趣建模技术核心研究依托大数据技术在线、离线赋能用户兴趣数据分类采集及预处理、模型进化;大数据赋能的用户、项目信息管理技术核心研究用大数据技术赋能数据分类采集,并用大数据计算基础设施赋能数据离线挖掘、数据存储、数据更新、数据在线解释。现有研究需强化用户、数据双驱型的智慧推荐方案设计、实践研究,且需强化数据安全研究。

关键词: 大数据, 智慧推荐, 数据采集, 数据挖掘, 数据可视化

Abstract: [Purpose/Significance] Data empowerment for collaborative transformation of the entire industrial chain has become a national strategy. In the context of multi-source and heterogeneous big data, we face higher demands on the diversity and real-time of recommendation services, as well as the standardization and comprehensiveness of user and item information management, in order to turn data into resources, support the diversification of smart recommendation services, and enhance the user interaction experience. To this end, we should systematically sort out and analyze the research results of user interest modeling and item information management technologies in China's big data empowerment, elaborate the technical system for standardized management of user and item information in intelligent recommendation. [Method/Process] The article summarizes the content of 507 documents using literature review methods, and summarizes and analyzes the domestic big data empowerment resource management technology for smart recommendation from two aspects: user interest modeling and item information management. User interest modeling technology includes model representation, model initialization and model evolution, the item information management technology shows the goals, implementation methods, implementation techniques, and methods of item information management, including data collection, data mining, data storage, data updating, and data interpretation. [Results/Conclusions] First, the user interest modeling technology of big data empowerment mainly studies the following issues. 1) The method of model representation is to accurately formalize the user's needs. 2) The model is initialized to collect user information in an implicit or explicit way and select tools and technologies according to data types to pre-process user data according to a fixed process, and select measurement methods to quantify user interest. 3) Model evolution aims to apply methods and technologies to evolve user interest models offline and online. Second, the item information management technology of big data empowerment mainly studies the following issues. 1) The online technology and offline equipment are used to synchronously collect item data across the whole domain, integrate domain knowledge, and use algorithm libraries and data analysis methods to mine item data. 2) The distributed storage is used such as databases and file systems to store item data. 3) Data collection, mining, and storage technologies are the main techniques, item data are updated through offline and online methods, and visualization technology is used to intuitively interpret ietm data. Finally, existing research needs to strengthen the design and practical research of user-driven and data-driven smart recommendation solutions, as well as strengthen research on data security.

Key words: big data, smart recommendation, data collection, data mining, data visualization

中图分类号: TP391;G202

孙雨生, 范颖, 祝博. 国内大数据赋能的智慧推荐资源管理技术研究进展[J]. 农业图书情报学报, 2023, 35(12): 4-17.

SUN Yusheng, FAN Ying, ZHU Bo. Research Advances in Resource Management Technology of Smart Recommendation Enabled by Big Data in China[J]. Journal of Library and Information Science in Agriculture, 2023, 35(12): 4-17.

参考文献

[1] 中华人民共和国中央人民政府. 中华人民共和国国民经济和社会发展第十四个五年规划和2035年远景目标纲要[EB/OL]. (2021-03-13)[2023-08-21]. https://www.gov.cn/xinwen/2021-03/13/content_5592681.htm.
The Central People's Government of the People's Republic of China. The 14th five-year plan for economic and social development and vision2035 of the People's Republic of China [EB/OL]. (2021-03-13)[2023-08-21].https://www.gov.cn/ xinwen/2021-03/13/content _5592681.htm.
[2] 秦鹏, 贾洪杰, 霍兴瀛, 等. 融合大数据挖掘的用户个性化POI推荐方法[J]. 计算机仿真, 2022, 39(6): 355-358, 385.
QIN P, JIA H J, HUO X Y, et al.User personalized POI recommendation method integrating big data mining[J]. Computer simulation, 2022, 39(6): 355-358, 385.
[3] 李翠平, 蓝梦微, 邹本友, 等. 大数据与推荐系统[J]. 大数据, 2015, 1(3): 23-35.
LI C P, LAN M W, ZOU B Y, et al.Big data and recommendation system[J]. Big data research, 2015, 1(3): 23-35.
[4] 胡蓉. 大数据环境下服务推荐系统及其关键方法研究[D]. 南京: 南京大学, 2014.
HU R.Research on service recommender system and its key approaches over big data[D].Nanjing: Nanjing University, 2014.
[5] 董小妹. 大数据环境下基于本体的协同过滤推荐算法改进研究[D]. 南京: 南京工业大学, 2013.
DONG X M.Research on the improvement of collaborative filtering recommendation algorithm under big data environment[D].Nanjing: Nanjing University of Technology, 2013.
[6] 刘海鸥, 姚苏梅, 黄文娜, 等. 基于用户画像的图书馆大数据知识服务情境化推荐[J]. 图书馆学研究, 2018(24): 57-63, 32.
LIU H O, YAO S M, HUANG W N, et al.Library big data knowledge service situational recommendation based on user profile[J]. Research on library science, 2018(24): 57-63, 32.
[7] 郭博林. 基于大数据分析的音乐个性化推荐系统应用研究[D]. 成都: 电子科技大学, 2018.
GUO B L.Research on the application of music personalized recommendation system based on big data analysis[D]. Chengdu: University of Electronic Science and Technology of China, 2018.
[8] 丁继红, 刘华中. 大数据环境下基于多维关联分析的学习资源精准推荐[J]. 电化教育研究, 2018, 39(2): 53-59, 66.
DING J H, LIU H Z.Accurate recommendation of learning resources based on multi-dimensional correlation analysis in age of big data[J]. E-education research, 2018, 39(2): 53-59, 66.
[9] 屠海龙. 基于大数据的协同过滤推荐算法研究[D]. 杭州: 浙江工业大学, 2018.
TU H L.Research on collaborative filtering recommendation algorithm based on big data[D]. Hangzhou: Zhejiang University of Technology, 2018.
[10] 邓玉林. 基于hadoop大数据框架的个性化推荐系统研究与实现[D]. 成都: 电子科技大学, 2016.
DENG Y L.Research and implementation of personalized recommendation system based on hadoop big data frame[D]. Chengdu: University of Electronic Science and Technology of China, 2016.
[11] 薛琳兰. 基于大数据技术的电商推荐系统的设计与实现[D]. 青岛: 青岛大学, 2020.
XUE L L.Design and implementation of E-commerce recommenda-tion system based on big data technology[D]. Qingdao: Qingdao University, 2020.
[12] 傅思维. 大数据环境下农产品电商平台智能推荐技术的研究与应用[D]. 长春: 吉林农业大学, 2019.
FU S W.Research and application on E-commerce platform intelligent recommendation technology of agricultural products based on big data[D]. Changchun: Jilin Agricultural University, 2019.
[13] 金志福. 基于大数据的教育资源个性推荐系统设计与实现[D]. 北京: 中国科学院大学(工程管理与信息技术学院), 2015.
JIN Z F.Design and implementation of big data-based educational resources personalized recommendation system[D]. Beijing: University of Chinese Academy of Sciences, 2015.
[14] 周浩. 基于大数据分析的微博好友推荐算法研究与应用[D]. 北京: 北京工业大学, 2017.
ZHOU H.Research and application of micro-blog friend recommendation algorithm based on big data analysis[D]. Beijing: Beijing University of Technology, 2017.
[15] 孟祥武, 纪威宇, 张玉洁. 大数据环境下的推荐系统[J]. 北京邮电大学学报, 2015, 38(2): 1-15.
MENG X W, JI W Y, ZHANG Y J.A survey of recommendation systems in big data[J]. Journal of Beijing university of posts and
16 telecommunications, 2015, 38(2): 1-15.
[16] 沈杰. 大数据环境下基于协同过滤的推荐系统研究与实现[D]. 杭州: 浙江工业大学, 2016.
SHEN J.Recommended system research and implementation based on collaborative filtering in the big data environment[D]. Hangzhou: Zhejiang University of Technology, 2016.
[17] 李琛轩. 面向推荐的大数据计算与存储平台设计与实现[D]. 哈尔滨: 哈尔滨工业大学, 2016.
LI C X.Design and implementation of computing and storage platform for big data recommendation[D]. Harbin: Harbin Institute of Technology, 2016.
[18] 王啸. 基于移动互联网大数据的个性化推荐系统研究[D]. 西安: 西安理工大学, 2014.
WANG X.Research on personalized recommendation system based on mobile Internet big data[D]. Xi'an: Xi'an University of Technology, 2014.
[19] 张进良, 叶求财. 大数据视阈下学习资源智能推荐模型构建[J]. 湖南科技大学学报(社会科学版), 2019, 22(4): 178-184.
ZHANG J L, YE Q C.Construction of intelligent recommendation model of learning resources from the perspective of big data[J]. Journal of Hunan university of science & technology (social science edition), 2019, 22(4): 178-184.
[20] 房璐璐. 基于大数据分析的推荐系统研究——基于Hadoop的电影推荐系统的设计与实现[D]. 北京: 北京邮电大学, 2015.
FANG L L.Design and implementation of movie recommendation system based on hadoop[D]. Beijing: Beijing University of Posts and Telecommunications, 2015.
[21] 孙雨生, 朱金宏, 李亚奇. 国内基于大数据的信息推荐研究进展: 核心内容[J]. 现代情报, 2020, 40(8): 156-165.
SUN Y S, ZHU J H, LI Y Q.Research development of information recommendation based on big data in China: Core content[J]. Journal of modern information, 2020, 40(8): 156-165.
[22] 岑凯伦, 于红岩, 杨腾霄. 大数据下基于Spark的电商实时推荐系统的设计与实现[J]. 现代计算机(专业版), 2016(24): 61-69.
CEN K L, YU H Y, YANG T X.Design and implement of E-commerce real-time recommender system with spark based on big data[J]. Modern computer, 2016(24): 61-69.
[23] 杨清智. 基于大数据技术的手机应用推荐系统的设计与实现[D]. 哈尔滨: 哈尔滨工业大学, 2017.
YANG Q Z.Design and implementation of mobile phone application recommendation system based on big data technology[D]. Harbin: Harbin Institute of Technology, 2017.
[24] 刘敏. 基于Hadoop的大数据平台设计与实现及在推荐系统中的应用[D]. 北京: 北京邮电大学, 2016.
LIU M.Design and implementation of big data platform based on hadoop and its application in recommendation system[D]. Beijing: Beijing University of Posts and Telecommunications, 2016.
[25] 田贤忠, 沈杰. 大数据环境下基于概率矩阵分解的个性化推荐[J]. 计算机科学, 2017, 44(S1): 438-441, 469.
TIAN X Z, SHEN J.Personalized recommendation based on proba-bility matrix decomposition in big data environment[J]. Computer science, 2017, 44(S1): 438-441, 469.
[26] 盛铨. 大数据环境下智慧图书馆智能推荐系统设计及其在高职教育中的应用研究[J]. 新世纪图书馆, 2019(8): 51-56.
SHENG Q.Design of intelligent recommendation system for smart library in big data environment and its application in higher vocational education[J]. New century library, 2019(8): 51-56.
[27] 陈泽. 个性化推荐算法研究及“大数据” 下的系统开发[D]. 重庆: 重庆邮电大学, 2013.
CHEN Z.Research on personalized recommendation algorithm and system development under "big data"[D]. Chongqing: Chongqing University of Posts and Telecommunications, 2013.
[28] 刘欢, 戴牡红, 龙飞. 基于评分可信度的大数据线性回归推荐算法[J]. 计算机应用研究, 2021, 38(2): 382-385.
LIU H, DAI M H, LONG F.Big data linear regression recommendation algorithm based on scoring credibility[J]. Application research of computers, 2021, 38(2): 382-385.
[29] 杨国龙. 企业间大数据推荐引流系统研究与设计[D]. 长沙: 湖南大学, 2016.
YANG G L.Research and design of inter-enterprise big data recommendation and drainage system[D]. Changsha: Hunan University, 2016.
[30] 刘海鸥, 孙晶晶, 苏妍嫄, 等. 面向图书馆大数据知识服务的多情境兴趣推荐方法[J]. 现代情报, 2018, 38(6): 62-67, 156.
LIU H O, SUN J J, SU Y Y, et al.A multi contextual interest recommender method for library big data knowledge service[J]. Journal of modern information, 2018, 38(6): 62-67, 156.
[31] 严克文. 大数据环境下电子商务个性化推荐算法应用研究[D]. 合肥: 合肥工业大学, 2016.
YAN K W.Research on the application of personalized recommendation algorithm in E-commerce under big data environment[D]. Hefei: Hefei University of Technology, 2016.
[32] 刘洋. 基于大数据的个性化智能交通推荐系统[D]. 南京: 东南大学, 2021.
LIU Y.Personalised intelligent transportation recommendation system based on big data[D]. Nanjing: Southeast University, 2021.
[33] 崔金栋, 高志豪. 基于大数据和微本体的微博信息推荐研究[J]. 情报资料工作, 2019, 40(5): 103-112.
CUI J D, GAO Z H.A probe into weibo information recommendation based on big data and micro ontology[J]. Information and documentation services, 2019, 40(5): 103-112.
[34] 翟丽丽, 王笑笑, 邢海龙. 基于改进VIKOR的大数据联盟数据资源群推荐方法研究[J]. 情报科学, 2021, 39(1): 120-127.
ZHAI L L, WANG X X, XING H L.Group recommendation method of big data alliance data resource based on improved VIKOR[J]. Information science, 2021, 39(1): 120-127.
[35] 杨宏胜. 基于大数据的用户个性化推荐系统设计与实现[D]. 南京: 南京邮电大学, 2020.
YANG H S.The design and implementation of user personalized recommendation system based on big data[D]. Nanjing: Nanjing
36 University of Posts and Telecommunications, 2020.
[36] 张玉忠, 方艾, 金铎, 等. 大数据在音乐推荐质量提升中的实践及应用[J]. 电信科学, 2014, 30(10): 43-47.
ZHANG Y Z, FANG A, JIN D, et al.Big data improving recommendation quality in music applications[J]. Telecommunications science, 2014, 30(10): 43-47.
[37] 刘海鸥, 黄文娜, 苏妍嫄, 等. 大数据深度融合的移动图书馆情境化推荐[J]. 情报科学, 2019, 37(1): 68-73.
LIU H O, HUANG W N, SU Y Y, et al.Mobile library situational recommendation based on big data deep fusion[J]. Information science, 2019, 37(1): 68-73.
[38] 李家华. 基于大数据的人工智能跨境电商导购平台信息个性化推荐算法[J]. 科学技术与工程, 2019, 19(14): 280-285.
LI J H.Personalized recommendation algorithm based on big data for artificial intelligence cross border E-commerce shopping platform[J]. Science technology and engineering, 2019, 19(14): 280-285.
[39] 黎超. 基于大数据的电商个性化推荐系统分析[J]. 商业经济研究, 2019(2): 69-72.
LI C.Analysis of personalized recommendation system for e-commerce based on big data[J]. Journal of commercial economics, 2019(2): 69-72.
[40] 杜春河. 大数据环境下隐语义模型推荐算法的改进与实现[D]. 广州: 华南理工大学, 2017.
DU C H.The improvement and implementation of latent factor model recommendation algorithm in big data environment[D]. Guangzhou: South China University of Technology, 2017.
[41] 金伟晟. 面向大数据的可信服务推荐方法研究[D]. 南京: 南京邮电大学, 2016.
JIN W S.Research on trustworthy service recommendation in big data environments[D]. Nanjing: Nanjing University of Posts and
42 Telecommunications, 2016.
[42] 王海艳, 金伟晟. 大数据场景下基于可信社团的服务推荐[J]. 华中科技大学学报(自然科学版), 2016, 44(3): 22-27.
WANG H Y, JIN W S.Service recommendation based on trustworthy community under big data environments[J]. Journal of Huazhong university of science and technology (natural science edition), 2016, 44(3): 22-27.
[43] 李嘉梦. 基于大数据的自由行行程推荐系统设计与实现[D]. 上海: 上海交通大学, 2017.
LI J M.Design and implementation of big data-based free travel recommendation system[D]. Shanghai: Shanghai Jiao Tong University, 2017.
[44] 赵杨, 杨彬, 董姝仪, 等. 多源大数据驱动的移动图书馆个性化推荐系统设计与实现[J]. 图书馆学研究, 2021(11): 20-31.
ZHAO Y, YANG B, DONG S Y, et al.Design and realization of a personalized recommendation system of mobile library driven by multi-source big data[J]. Research on library science, 2021(11): 20-31.
[45] 马晓亭. 基于情景大数据的图书馆个性化服务推荐系统研究[J]. 现代情报, 2016, 36(4): 90-94.
MA X T.Study of personalized service recommendation system for library based on contextual big data[J]. Journal of modern information, 2016, 36(4): 90-94.
[46] 陈军, 谢卫红, 陈扬森. 国内外大数据推荐算法领域前沿动态研究[J]. 中国科技论坛, 2018(1): 173-181.
CHEN J, XIE W H, CHEN Y S.Frontier dynamics of big data recommendation algorithm at home and abroad[J]. Forum on science and technology in China, 2018(1): 173-181.
[47] 姚凯, 涂平, 陈宇新, 等. 基于多源大数据的个性化推荐系统效果研究[J]. 管理科学, 2018, 31(5): 3-15.
YAO K, TU P, CHEN Y X, et al.Research on the effectiveness of personalized recommender system based on multi-source big data[J]. Journal of management science, 2018, 31(5): 3-15.
[48] 李娜, 蒋晓敏. 基于大数据挖掘技术的IPTV智能推荐系统的设计与实现[J]. 广播电视网络, 2022, 29(6): 73-76.
LI N, JIANG X M.Design and implementation of IPTV intelligent recommendation system based on big data mining technology[J]. Radio & television network, 2022, 29(6): 73-76.
[49] 刘茗. 大数据时代电子商务精准营销实现个性化推荐研究[J]. 现代营销(下旬刊), 2022(9): 164-166.
LIU M.Research on personalized recommendation of E-commerce precision marketing in big data era[J]. Marketing management review, 2022(9): 164-166.
[50] 胡一. 基于大数据的电子商务个性化信息推荐服务模式研究[D]. 长春: 吉林大学, 2015.
HU Y.The research of E-commerce personalized information rec-ommendation service mode based on big-data[D]. Changchun: Jilin University, 2015.
[51] 张婷婷. 基于大数据的Web个性化推荐系统设计[J]. 现代电子技术, 2018, 41(16): 155-158.
ZHANG T T.Design of Web personalized recommendation system based on big data[J]. Modern electronics technique, 2018, 41(16): 155-158.
[52] 陈鑫, 王斌, 曾范清. 商品搭配大数据推荐方法研究综述[J]. 计算机工程与科学, 2020, 42(1): 36-45.
CHEN X, WANG B, ZENG F Q.Reviewing big data recommenda-tion methods of commodity collocation[J]. Computer engineering & science, 2020, 42(1): 36-45.
[53] 崔金栋, 杜文强, 关杨. 基于大数据与LDA融合的微博信息推荐方法研究[J]. 情报科学, 2018, 36(9): 27-31, 76.
CUI J D, DU W Q, GUAN Y.Research on microblog information recommendation method based on big data and LDA fusion[J].
54 Information science, 2018, 36(9): 27-31, 76.
[54] 段文彬. 大数据联盟数据资源推荐系统研究[D]. 哈尔滨: 哈尔滨理工大学, 2018.
DUAN W B.Research on data resource recommendation system of big data alliance[D]. Harbin: Harbin University of Science and
55 Technology, 2018.
[55] 谢瑶瑶. 大数据模拟环境下的分布式协同过滤推荐系统的研究[D]. 武汉: 武汉理工大学, 2014.
XIE Y Y.Research on distributed collaborative filtering recommendation system in big data simulation environment[D]. Wuhan: Wuhan University of Technology, 2014.
[56] 杨光. Web大数据多层级相关推荐算法研究[D]. 武汉: 武汉理工大学, 2016.
YANG G.Research of multi-level based related recommendation algorithm for web big data[D]. Wuhan: Wuhan University of Technology, 2016.