Abstract:

Purpose/Significance The formulation of evidence-based science and technology policy critically relies on the timely and accurate provision of relevant, high-quality evidence. However, current evidence recommendation practices often suffer from significant limitations in both accuracy and efficiency, hindering the scientific rigor and intelligent application of evidence within the policy-making process. These shortcomings hinder policymakers' ability to leverage the most pertinent research and data, potentially leading to suboptimal decisions. Addressing this critical gap, this research proposes a novel knowledge graph-based evidence recommendation method. The primary objective is to substantially enhance the scientific foundation and intelligent capabilities of evidence utilization during policy formulation. This method aims to empower policymakers by providing more reliable, contextually relevant, and efficiently retrieved data support. Ultimately this will foster more robust, transparent, and demonstrably effective science and technology policies grounded in comprehensive research insights. Method/Process To achieve these objectives, this study systematically constructs a domain-specific knowledge graph meticulously centered on the intricate citation relationships between policy documents and academic research papers. This graph serves as the foundational semantic network representing entities (policies, articles, topics, authors, institute etc.) and their multifaceted interconnections. Most importantly, we introduce and adapt the Knowledge Graph Attention Network (KGAT) algorithm n an innovative way. Leveraging KGAT's sophisticated graph attention mechanisms, our model effectively captures and learns complex, high-order semantic relationships between policy requirements (represented as queries or specific nodes) and potential evidence sources (research paper nodes). This deep relational understanding enables nuanced evidence relevance scoring and personalized recommendation. To rigorously validate the proposed method's practical efficacy and performance, we conducted an extensive empirical study within the specific domain of agricultural science and technology policy. Furthermore, to demonstrate real-world applicability and provide a tangible tool for policymakers, we designed and implemented a fully functional Evidence Intelligent Recommendation System (EIRS). This system seamlessly integrates the core KG-based recommendation engine and incorporates advanced intelligent analysis capabilities. Significantly, EIRS supports an end-to-end workflow initiated by natural language policy questions posed by users, enabling intuitive interaction and precise, demand-driven evidence retrieval and recommendation. [Results/ Conclusions Experimental results, conducted on real-world datasets within the agricultural science and technology policy domain, demonstrate the superior performance of the proposed KGAT-based recommendation method. It consistently outperforms several state-of-the-art baseline algorithms across multiple key evaluation metrics, including precision, recall, normalized discounted cumulative gain (NDCG), and mean reciprocal rank (MRR). This quantitatively confirms its significantly stronger recommendation capability. In addition to quantitative metrics, the model inherently offers enhanced explainability due to the transparent nature of the knowledge graph structure and the attention weights learned by KGAT, allowing for insights into why specific evidence is recommended, based on its semantic connections to the policy query. Concurrently, the implemented EIRS has proven to be highly effective in practice. It efficiently identifies and recommends evidence resources exhibiting a strong match with complex policy requirements expressed in natural language. The system's successful deployment underscores its potential to tangibly augment the scientific underpinning of science and technology policy development. By effectively bridging the gap between vast research knowledge and specific policy needs through intelligent, accurate, and explainable recommendations, this research provides a novel, practical pathway towards realizing truly intelligent and rigorously evidence-based policy formulation processes. The methodology and system prototype offer a valuable and adaptable framework for various policy domains beyond the presented case study.

Key words: evidence-based policy, knowledge graph, evidence recommendation