科技评价计量方法述评

doi:10.13998/j.cnki.issn1002-1248.2020.08.20-0467

摘要/Abstract

摘要： [目的/意义]讨论科技评价各类计量指标和方法的客观价值,为从事科学研究和科技管理等人员客观认识各类定量评价指标提供参考。[方法/过程]介绍4类常见计量指标和方法的基本原理、特点、适用范围和典型应用案例,包括：传统统计数量的指标、社会网络分析指标、替代计量学指标、多指标整合的方法等。[结果/结论]就合理利用计量方法开展科技评价工作提出了4方面建议：正确看待各类定量指标、合理选用评价方法、谨慎使用评价结果以及关注评价对象的认可度等。

关键词: 科技评价, 计量指标, 论文, 社会网络分析, 替代计量学

Abstract: [Purpose/Significance] This paper discusses the objective value of various quantitative indicators and methods used for research evaluation, and provides reference for researchers and technology managers to understand such methods objectively. [Method/Process] The basic principles, characteristics, scope of application and typical application cases of four common indicators and methods are introduced, including: traditional statistical quantity indicators, social network analysis indicators, altmetrics indicators, and multi-index integration methods. [Results/Conclusions] Four suggestions are put forward on rational use of quantitative indicators to carry out evaluation work: correct view of various quantitative indicators, reasonable selection of evaluation methods, careful use of evaluation results, and the acceptance for the evaluation results.

Key words: science and technology evaluation, quantitative indicator, paper, social network analysis, altmetrics

中图分类号:

G350

陈云伟. 科技评价计量方法述评[J]. 农业图书情报学报, 2020, 32(8): 4-11.

CHEN Yunwei. Review on Quantitative Methods of Science and Technology Evaluation[J]. Journal of Library and Information Science in Agriculture, 2020, 32(8): 4-11.

参考文献

[1] GUNASHEKAR S, WOODING S, GUTHRIE S.How do NIHR peer review panels use bibliometric information to support their decisions?[J]. Scientometrics, 2017, 112(3): 1813-1835.
[2] SIVERTSEN G.Experiences with a bibliometric model for perfor-mance based funding of research institutions[C]. 10th international science and technology indicators conference, University of Vienna, 2008: 126-128.
[3] 田倩飞, 张志强, 陈云伟. 瑞典研究理事会的组织特点及评审评估机制研究[J]. 世界科技研究与发展, 2018, 40(6): 634-638.
[4] MOED H F.Appropriate use of metrics in research assessment of autonomous academic institutions[J]. Scholarly assessment reports, 2020, 2(1): 1.
[5] 刘强, 陈云伟. 科学家评价方法述评[J]. 情报杂志, 2019, 38(3): 80-86.
[6] WALTMAN L.A review of the literature on citation impact indicators[J]. Journal of informetrics, 2016, 10(2): 365-391.
[7] HIRSCH J E.An index to quantify an individual's scientific research output[J]. Proceedings of the National academy of sciences of the USA, 2005, 102(46): 16569-16572.
[8] EGGHE L.Theory and practice of the g-index[J]. Scientometrics, 2006, 69(1): 131-152.
[9] WALTMAN L, VAN ECK N J, VAN LEEUWEN T N, et al. Towards a new crown indicator: Some theoretical considerations[J]. Journal of informetrics, 2011, 5(1): 37-47.
[10] BALL P.Index aims for fair ranking of scientists[J]. Nature, 2005, 436(7053): 900-900.
[11] ALONSO S, CABRERIZO F J, HERRERA-VIEDMA E, et al.Hg-index: A new index to characterize the scientific output of researchers based on the h- and g-indices[J]. Scientometrics, 2010, 82(2): 391-400.
[12] PRATHAP G.The 100 most prolific economists using the p-index[J]. Scientometrics, 2010, 84(1): 167-172.
[13] ZHANG L, ROUSSEAU R, SIVERTSEN G.Science deserves to be judged by its contents, not by its wrapping: Revisiting Seglen’s work on journal impact and research evaluation[J]. Plos One, 2017, 12(3): e0174205.
[14] Natureindex. Using the index.[EB/OL].[2020-05-13]. https://www.natureindex.com/using-the-index.
[15] SIVERTSEN G, ROUSSEAU R, ZHANG L.Measuring scientific production with modified fractional counting[J]. Journal of informetrics, 2019, 13(2): 679-694.
[16] Web of Science group. Highly cited researchers.[EB/OL].[2020-05-13].https://recognition.webofsciencegroup.com/awards/highly-cited/2019/methodology/.
[17] 章成志, 胡志刚, 徐硕, 等. 全文本计量分析理论与技术的新进展与新探索—2019全文本文献计量分析学术沙龙综述[J]. 信息资源管理学报, 2020, 10(1): 111-117.
[18] TIJSSEN R J, VISSER M S, VAN LEEUWEN T N. Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?[J]. Scientometrics, 2002, 54(3): 381-397.
[19] ZITT M, SMALL H.Modifying the journal impact factor by fractional citation weighting: The audience factor[J]. Journal of the American society for information science and technology, 2008, 59(11): 1856-1860.
[20] OECD. OECD Data.[EB/OL].[2020-05-28].https://data.oecd.org/.
[21] OTTE E, ROUSSEAU R.Social network analysis: A powerful strategy, also for the information sciences[J]. Journal of information science, 2002, 28(6): 441-453.
[22] COLIZZA V, FLAMMINI A, SERRANO M A, et al.Detecting rich-club ordering in complex networks[J]. Nat Phys, 2006, 2(2): 110-115.
[23] CHEN Y W, BORNER K, FANG S.Evolving collaboration networks in scientometrics in 1978-2010: A micro-macro analysis[J]. Scientometrics, 2013, 95(3): 1051-1070.
[24] WHITFIELD J.Collaboration: Group theory[J]. Nature, 2008, 455: 720-723.
[25] WUCHTY S, JONES BF, UZZI B.The increasing dominance of teams in production of knowledge[J]. Science, 2007, 316(5827): 1036-1039.
[26] LIN L L, XU Z M, DING Y, et al.Finding topic-level experts in scholarly networks[J]. Scientometrics, 2013, 97(3): 797-819.
[27] ABBASI A, ALTMANN J, HOSSAIN L.Identifying the effects of co-authorship networks on the performance of scholars: A correlation and regression analysis of performance measures and social network analysis measures[J]. Journal of informetrics, 2011, 5(4):594-607.
[28] LIU X, BOLLEN J, NELSON M L, et al.Co-authorship networks in the digital library research community[J]. Information processing and management, 2005, 41(6): 1462-1480.
[29] YAN E J, DING Y.Discovering author impact: A PageRank perspective[J]. Information processing and management, 2011, 47(1):125-134.
[30] NYKL M, JEZEK K, FIALA D, et al.PageRank variants in the evaluation of citation networks[J]. Journal of informetrics, 2014, 8(3): 683-692.
[31] PERSSON RAX.Bibliometric author evaluation through linear regression on the coauthor network[J]. Journal of informetrics, 2017, 11(1): 299-306.
[32] 陈云伟, 邓勇, 陈方, 等. 复合合作强度指数构建及应用研究[J]. 图书情报工作, 2015, 59(13): 96-103.
[33] 陈云伟, 邓勇, 陈方, 等. 引擎技术预见模型的构建及实证研究[J]. 图书情报工作, 2017, 61(13): 77-86.
[34] PRIEM J, TARABORELLI D, GROTH P, et al.Altmetrics: A manifesto. [2010-10-26]. Altmetrics: A manifesto. [2010-10-26]. http://altmetrics.org/manifesto.
[35] BORNMANN L.Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics[J]. Scientometrics, 2015, 103(3): 1123-1144.
[36] PIWOWAR H.Value all research products[J]. Nature, 2013, 493(7431): 159-159.
[37] ERDT M, NAGARAJAN A, SIN S C J, et al. Altmetrics: An analysis of the state-of-the-art in measuring research impact on social media[J]. Scientometrics, 2016, 109: 1117-1166.
[38] COSTAS R, ZAHEDI Z, WOUTERS P.Do "altmetrics" correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective[J]. Journal of the association for information science and technology, 2015, 66(10): 2003-2019.
[39] 张志强, 田倩飞, 陈云伟. 科技强国主要科技指标体系比较研究[J]. 中国科学院院刊, 2018, 33(10): 1052-1063.
[40] WIPO. Global Innovation Index (GII) 2019.[EB/OL].[2020-05-28]. https://www.wipo.int/global_innovation_index/en/2019/.
[41] QS. QS World University Rankings.[EB/OL].[2020-05-29]. https://www.topuniversities.com/university-rankings/world-university-rankings/2020.
[42] CWTS. CWTS Leiden Ranking Indicators.[EB/OL].[2020-05-29]. https://www.leidenranking.com/information/indicators.
[43] 刘强, 陈云伟, 张志强. 用于科技评价的挪威模型方法与应用综述[J]. 数据分析与知识发现, 2019, 3(5): 41-50.
[44] SCHNEIDER J W.An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway[J]. European political science, 2009, 8(3): 364-378.
[45] 陈云伟, 吴林寰, 陈方, 等. 青年科学家竞争力评价模型及实证研究[J]. 武汉大学学报(信息科学版), 2017, 42(专辑): 219-222.
[46] 陈云伟, 邓勇, 陈方, 等. 科学家创新力评价模型及实证研究[J]. 情报杂志, 2016, 35(3): 142-147.
[47] 陈云伟, 邓勇, 陈方, 等. 科研单元创新力评价模型及实证研究[J]. 情报杂志, 2015, 34(5): 59-65.
[48] 王志刚. 要写有价值高水平论文不唯论文并非不要论文.[EB/OL].[2020-03-17]. https://www.chinanews.com/gn/2020/03-17/9128444.shtml.
[49] The San Francisco declaration on research assessment(DO-RA)[EB/OL]. [2017-11-05].https://en.wikipedia.org/wiki/San_Francisco_Declaration_on_Research_Assessment.
[50] HICKS D, WOUTERS P, WALTMAN L, et al.The Leiden manifesto for research metrics[J]. Nature, 2015, 520: 429-431.
[51] ADAMS J, MCVEIGH M, PENDLEBURY D, et al.全面画像而非简单指标[J]. 科学观察, 2019, 14(4): 58-65.
[52] WAINER J, VIEIRA P.Correlations between bibliometrics and peer evaluation for all disciplines: The evaluation of Brazilian scientists[J]. Scientometrics, 2013, 96(2): 395-410.
[53] HERZ N, DAN O, CENSOR N, et al.Opinion: Authors overestimate their contribution to scientific work, demonstrating a strong bias[J]. Proceedings of the National academy of sciences of the USA, 2020, 117(12): 6282-6285.