SoS: BIO: Evaluating the Impact of Biomedical Tools and Methods - PROJECT SUMMARY (See instructions): Advances in biomedical sciences are often driven by new tools, ranging from novel experimental techniques, to innovative models, and new software. V Jhile bibliographic data such as citation counts track the impact of research discoveries, these metrics fail to capture the full extent to which specific tools are used in science and beyond. Indeed, the impact of these tools on both the scientific community and society at large is largely invisible to traditional metrics, along with the contributions of the researchers who develop them. We need, therefore, novel databases and metrics to trace the true impact of tools, bringing them within the scope of Science of Science. Three fundamental challenges impede this goal: a data gap, i.e., a lack of a ground truth dataset on tools and their usage, a metrics gap, i.e., the absence of meaningful impact indicators, and a mechanistic gap, i.e., a lack of theoretical foundation to understand their adoption. Our proposal aims to tackle these challenges, by developing a data-driven approach to automatically identify the tools used in biomedicine, that we collectively call Bio Tools, and developing indicators to trace the multiple dimensions of impact they have, offering a rigorous empirical foundation for the quantitative study of Bio Tools. We start with developing a ground-truth corpus by extracting laboratory techniques, software applications, and modeling methods from existing ontologies, and identifying their mentions in the full-text of millions of documents, from publications and grants, to patents and clinical trials (Aim 1). The recovered information, along with metadata, will be assembled into the knowledge base BioToolKB, which will allow us to develop a multidimensional impact measures, and quantify the extent to which traditional metrics track or underestimate Bio Tool impact (Aim 2). Finally, we rely on these indicators and BioToolKB to understand the diffusion mechanisms of Bio Tools across biomedical fields and sectors, identifying the conditions in which some Bio Tools are widely adopted and others overlooked (Aim 3). The success of this proposal is ensured by an interdisciplinary team blending domain expertise in biomedical sciences and the Science of Science with technical expertise in bibliometrics and network science, and industry collaborators, supporting the creation of a robust technical foundation.
