中文    English

Journal of Library and Information Science in Agriculture ›› 2020, Vol. 32 ›› Issue (8): 4-11.doi: 10.13998/j.cnki.issn1002-1248.2020.08.20-0467

;

• Special review •     Next Articles

Review on Quantitative Methods of Science and Technology Evaluation

CHEN Yunwei   

  1. Scientometrics & Evaluation Research Center (SERC), Chengdu Library and Information Center of Chinese Academy of Sciences, Chengdu 610041
  • Received:2020-06-05 Online:2020-08-05 Published:2020-09-02

PDF (PC)

192

Abstract: [Purpose/Significance] This paper discusses the objective value of various quantitative indicators and methods used for research evaluation, and provides reference for researchers and technology managers to understand such methods objectively. [Method/Process] The basic principles, characteristics, scope of application and typical application cases of four common indicators and methods are introduced, including: traditional statistical quantity indicators, social network analysis indicators, altmetrics indicators, and multi-index integration methods. [Results/Conclusions] Four suggestions are put forward on rational use of quantitative indicators to carry out evaluation work: correct view of various quantitative indicators, reasonable selection of evaluation methods, careful use of evaluation results, and the acceptance for the evaluation results.

Key words: science and technology evaluation, quantitative indicator, paper, social network analysis, altmetrics

CLC Number: 

  • G350
[1] GUNASHEKAR S, WOODING S, GUTHRIE S.How do NIHR peer review panels use bibliometric information to support their decisions?[J]. Scientometrics, 2017, 112(3): 1813-1835.
[2] SIVERTSEN G.Experiences with a bibliometric model for perfor-mance based funding of research institutions[C]. 10th international science and technology indicators conference, University of Vienna, 2008: 126-128.
[3] 田倩飞, 张志强, 陈云伟. 瑞典研究理事会的组织特点及评审评估机制研究[J]. 世界科技研究与发展, 2018, 40(6): 634-638.
[4] MOED H F.Appropriate use of metrics in research assessment of autonomous academic institutions[J]. Scholarly assessment reports, 2020, 2(1): 1.
[5] 刘强, 陈云伟. 科学家评价方法述评[J]. 情报杂志, 2019, 38(3): 80-86.
[6] WALTMAN L.A review of the literature on citation impact indicators[J]. Journal of informetrics, 2016, 10(2): 365-391.
[7] HIRSCH J E.An index to quantify an individual's scientific research output[J]. Proceedings of the National academy of sciences of the USA, 2005, 102(46): 16569-16572.
[8] EGGHE L.Theory and practice of the g-index[J]. Scientometrics, 2006, 69(1): 131-152.
[9] WALTMAN L, VAN ECK N J, VAN LEEUWEN T N, et al. Towards a new crown indicator: Some theoretical considerations[J]. Journal of informetrics, 2011, 5(1): 37-47.
[10] BALL P.Index aims for fair ranking of scientists[J]. Nature, 2005, 436(7053): 900-900.
[11] ALONSO S, CABRERIZO F J, HERRERA-VIEDMA E, et al.Hg-index: A new index to characterize the scientific output of researchers based on the h- and g-indices[J]. Scientometrics, 2010, 82(2): 391-400.
[12] PRATHAP G.The 100 most prolific economists using the p-index[J]. Scientometrics, 2010, 84(1): 167-172.
[13] ZHANG L, ROUSSEAU R, SIVERTSEN G.Science deserves to be judged by its contents, not by its wrapping: Revisiting Seglen’s work on journal impact and research evaluation[J]. Plos One, 2017, 12(3): e0174205.
[14] Natureindex. Using the index.[EB/OL].[2020-05-13]. https://www.natureindex.com/using-the-index.
[15] SIVERTSEN G, ROUSSEAU R, ZHANG L.Measuring scientific production with modified fractional counting[J]. Journal of informetrics, 2019, 13(2): 679-694.
[16] Web of Science group. Highly cited researchers.[EB/OL].[2020-05-13].https://recognition.webofsciencegroup.com/awards/highly-cited/2019/methodology/.
[17] 章成志, 胡志刚, 徐硕, 等. 全文本计量分析理论与技术的新进展与新探索—2019全文本文献计量分析学术沙龙综述[J]. 信息资源管理学报, 2020, 10(1): 111-117.
[18] TIJSSEN R J, VISSER M S, VAN LEEUWEN T N. Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?[J]. Scientometrics, 2002, 54(3): 381-397.
[19] ZITT M, SMALL H.Modifying the journal impact factor by fractional citation weighting: The audience factor[J]. Journal of the American society for information science and technology, 2008, 59(11): 1856-1860.
[20] OECD. OECD Data.[EB/OL].[2020-05-28].https://data.oecd.org/.
[21] OTTE E, ROUSSEAU R.Social network analysis: A powerful strategy, also for the information sciences[J]. Journal of information science, 2002, 28(6): 441-453.
[22] COLIZZA V, FLAMMINI A, SERRANO M A, et al.Detecting rich-club ordering in complex networks[J]. Nat Phys, 2006, 2(2): 110-115.
[23] CHEN Y W, BORNER K, FANG S.Evolving collaboration networks in scientometrics in 1978-2010: A micro-macro analysis[J]. Scientometrics, 2013, 95(3): 1051-1070.
[24] WHITFIELD J.Collaboration: Group theory[J]. Nature, 2008, 455: 720-723.
[25] WUCHTY S, JONES BF, UZZI B.The increasing dominance of teams in production of knowledge[J]. Science, 2007, 316(5827): 1036-1039.
[26] LIN L L, XU Z M, DING Y, et al.Finding topic-level experts in scholarly networks[J]. Scientometrics, 2013, 97(3): 797-819.
[27] ABBASI A, ALTMANN J, HOSSAIN L.Identifying the effects of co-authorship networks on the performance of scholars: A correlation and regression analysis of performance measures and social network analysis measures[J]. Journal of informetrics, 2011, 5(4):594-607.
[28] LIU X, BOLLEN J, NELSON M L, et al.Co-authorship networks in the digital library research community[J]. Information processing and management, 2005, 41(6): 1462-1480.
[29] YAN E J, DING Y.Discovering author impact: A PageRank perspective[J]. Information processing and management, 2011, 47(1):125-134.
[30] NYKL M, JEZEK K, FIALA D, et al.PageRank variants in the evaluation of citation networks[J]. Journal of informetrics, 2014, 8(3): 683-692.
[31] PERSSON RAX.Bibliometric author evaluation through linear regression on the coauthor network[J]. Journal of informetrics, 2017, 11(1): 299-306.
[32] 陈云伟, 邓勇, 陈方, 等. 复合合作强度指数构建及应用研究[J]. 图书情报工作, 2015, 59(13): 96-103.
[33] 陈云伟, 邓勇, 陈方, 等. 引擎技术预见模型的构建及实证研究[J]. 图书情报工作, 2017, 61(13): 77-86.
[34] PRIEM J, TARABORELLI D, GROTH P, et al.Altmetrics: A manifesto. [2010-10-26]. Altmetrics: A manifesto. [2010-10-26]. http://altmetrics.org/manifesto.
[35] BORNMANN L.Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics[J]. Scientometrics, 2015, 103(3): 1123-1144.
[36] PIWOWAR H.Value all research products[J]. Nature, 2013, 493(7431): 159-159.
[37] ERDT M, NAGARAJAN A, SIN S C J, et al. Altmetrics: An analysis of the state-of-the-art in measuring research impact on social media[J]. Scientometrics, 2016, 109: 1117-1166.
[38] COSTAS R, ZAHEDI Z, WOUTERS P.Do "altmetrics" correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective[J]. Journal of the association for information science and technology, 2015, 66(10): 2003-2019.
[39] 张志强, 田倩飞, 陈云伟. 科技强国主要科技指标体系比较研究[J]. 中国科学院院刊, 2018, 33(10): 1052-1063.
[40] WIPO. Global Innovation Index (GII) 2019.[EB/OL].[2020-05-28]. https://www.wipo.int/global_innovation_index/en/2019/.
[41] QS. QS World University Rankings.[EB/OL].[2020-05-29]. https://www.topuniversities.com/university-rankings/world-university-rankings/2020.
[42] CWTS. CWTS Leiden Ranking Indicators.[EB/OL].[2020-05-29]. https://www.leidenranking.com/information/indicators.
[43] 刘强, 陈云伟, 张志强. 用于科技评价的挪威模型方法与应用综述[J]. 数据分析与知识发现, 2019, 3(5): 41-50.
[44] SCHNEIDER J W.An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway[J]. European political science, 2009, 8(3): 364-378.
[45] 陈云伟, 吴林寰, 陈方, 等. 青年科学家竞争力评价模型及实证研究[J]. 武汉大学学报(信息科学版), 2017, 42(专辑): 219-222.
[46] 陈云伟, 邓勇, 陈方, 等. 科学家创新力评价模型及实证研究[J]. 情报杂志, 2016, 35(3): 142-147.
[47] 陈云伟, 邓勇, 陈方, 等. 科研单元创新力评价模型及实证研究[J]. 情报杂志, 2015, 34(5): 59-65.
[48] 王志刚. 要写有价值高水平论文不唯论文并非不要论文.[EB/OL].[2020-03-17]. https://www.chinanews.com/gn/2020/03-17/9128444.shtml.
[49] The San Francisco declaration on research assessment(DO-RA)[EB/OL]. [2017-11-05].https://en.wikipedia.org/wiki/San_Francisco_Declaration_on_Research_Assessment.
[50] HICKS D, WOUTERS P, WALTMAN L, et al.The Leiden manifesto for research metrics[J]. Nature, 2015, 520: 429-431.
[51] ADAMS J, MCVEIGH M, PENDLEBURY D, et al.全面画像而非简单指标[J]. 科学观察, 2019, 14(4): 58-65.
[52] WAINER J, VIEIRA P.Correlations between bibliometrics and peer evaluation for all disciplines: The evaluation of Brazilian scientists[J]. Scientometrics, 2013, 96(2): 395-410.
[53] HERZ N, DAN O, CENSOR N, et al.Opinion: Authors overestimate their contribution to scientific work, demonstrating a strong bias[J]. Proceedings of the National academy of sciences of the USA, 2020, 117(12): 6282-6285.
[1] WAN Yijia, KU liping. Policies, Resources and Services of Global Alliance of Open Access Scholarly Communication Platforms [J]. Journal of Library and Information Science in Agriculture, 2022, 34(6): 24-35.
[2] SUN Shaodan, DENG Jun, ZHANG Zishu, ZHONG Chuyi, SHENG Panpan. Topic Knowledge Organization of Modern Newspaper Resources by Incorporating the Knowledge Element Concept: Taking the "Shengjing Times" as an Example [J]. Journal of Library and Information Science in Agriculture, 2022, 34(4): 50-62.
[3] LI Xiaohui, WANG Jue, LANG Linfang. Academic Impact of Retracted Dissertations and Theses [J]. Journal of Library and Information Science in Agriculture, 2021, 33(7): 46-53.
[4] ZHAO Rongying, CHANG Ruru, WANG Xiaoyu, ZHANG Zhaoyang. Progress of Library and Information Science During the 13th Five-year Plan：Analysis on Funded Papers [J]. Journal of Library and Information Science in Agriculture, 2021, 33(6): 18-29.
[5] HOU Jianhua, GENG Bingbing, ZHANG Yang. Academic Origin Structure and Mobility Network of Technological Talent in Chinese Universities [J]. Journal of Library and Information Science in Agriculture, 2021, 33(6): 66-80.
[6] WANG Feifei, HAN Wenfei, SU Ziyao, YI Xinyue. Exploring the Academic Exchange among Countries along the "The Belt and Road": Bibliometrics Perspective of Highly Cited Papers [J]. Journal of Library and Information Science in Agriculture, 2021, 33(6): 94-106.
[7] XU Yi, LI Jing, XU Haiyun, LI Shuying. Identification of Technology Transfer Potential Based on Patent Dynamic Characteristics [J]. Journal of Library and Information Science in Agriculture, 2021, 33(6): 107-115.
[8] LI Pei, GU Donglei, ZOU Tao, JIANG Ling, LIU Jianfeng. Exploration of Potential Reading Interest Groups in Social Sciences of Undergraduates Based on Social Network Analysis: A Case Study of China Pharmaceutical University [J]. Journal of Library and Information Science in Agriculture, 2021, 33(3): 78-89.
[9] LAN Xue, HAN Yi. Data Usability of Book Rating Scores on Social Reading Platform from the Perspective of Altmetrics Evaluation: A Case Study of Douban Reading [J]. Journal of Library and Information Science in Agriculture, 2021, 33(11): 74-82.
[10] YU Liping. An Analysis of "Four Only" Elements' Relationships, Comparison of Their Dangers and Countermeasures in Humanities and Social Sciences [J]. Journal of Library and Information Science in Agriculture, 2021, 33(1): 53-61.
[11] GUO Lina, ZUO Wenge, ZHOU Qun. A bibliometric analysis of ESI highly cited papers from China and USA [J]. Journal of Library and Information Science in Agriculture, 2020, 32(5): 45-54.
[12] GUAN Cuizhong, FAN Aihong, ZHAO Chenggang. Social Network Analysis of Research Collaborations between Departments—— Taking Tsinghua University as an Example [J]. Journal of Library and Information Science in Agriculture, 2020, 32(4): 42-50.
[13] CHANG Ruru, ZHAO Rongying, JIA Zengshuai, ZHANG Zhaoyang. The Characteristics and Impact of International Artificial Intelligence (AI) Patent Cooperation [J]. Journal of Library and Information Science in Agriculture, 2020, 32(2): 58-70.
[14] DING Xujie. Evaluation of the Humanities Resource Construction of University Libraries from the Perspective of the Guarantee Rate of Citation: Taking the Department of Philosophy of Shanghai Jiao Tong University as an Example [J]. Journal of Library and Information Science in Agriculture, 2020, 32(12): 77-86.
[15] CHAI Jiaqi, CHEN Shiji. Review on Paper Novelty Measurement [J]. Journal of Library and Information Science in Agriculture, 2020, 32(10): 56-61.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Editorial Board of Journal of Library and Information Science in Agriculture
Tel：(010)82109667　Fax：(010)82109667　E-mail：xuekan＠caas.cn
Powered by Beijing Magtech Co., Ltd.
知识共享许可协议