The Arabic Citation Index (ARCI) was launched in 2020. This article provides an overview of the scientific literature contained in this new database and explores its possible usage in research... Show moreThe Arabic Citation Index (ARCI) was launched in 2020. This article provides an overview of the scientific literature contained in this new database and explores its possible usage in research evaluation. As of May 2022, ARCI had indexed 138,283 scientific publications published between 2015 and 2020. ARCI’s coverage is characterised by using the metadata available in scientific publications. First, I investigate the distributions of the indexed literature at various levels (research domains, countries, languages, open access). Articles make up nearly all the documents indexed with a share of 99% of ARCI. The Arts & Humanities and Social Sciences fields have the highest concentration of publications. Most indexed journals are published in Egypt, Algeria, Iraq, Jordan, and Saudi Arabia. About 8% of publications in ARCI are published in languages other than Arabic. Second, I use an unsupervised machine learning model, LDA (Latent Dirichlet Allocation), and the text mining algorithm of VOSviewer to uncover the main topics in ARCI. These methods provide a better understanding of ARCI’s thematic structure. Next, I discuss how ARCI can complement global standards in the context of a more inclusive research evaluation. Finally, I suggest a few research opportunities after discussing the findings of this study. Show less
As sustainability becomes an increasing priority throughout global society, academic and research institutions are assessed on their contribution to relevant research publications. This study... Show moreAs sustainability becomes an increasing priority throughout global society, academic and research institutions are assessed on their contribution to relevant research publications. This study compares four methods of identifying research publications related to United Nations Sustainable Development Goal 13—Climate Action (SDG 13). The four methods (Elsevier, STRINGS, SIRIS, and Dimensions) have each developed search strings with the help of subject matter experts, which are then enhanced through distinct methods to produce a final set of publications. Our analysis showed that the methods produced comparable quantities of publications but with little overlap between them. We visualized some difference in topic focus between the methods and drew links with the search strategies used. Differences between publications retrieved are likely to come from subjective interpretation of the goals, keyword selection, operationalizing search strategies, AI enhancements, and selection of bibliographic database. Each of the elements warrants deeper investigation to understand their role in identifying SDG-related research. Before choosing any method to assess the research contribution to SDGs, end users of SDG data should carefully consider their interpretation of the goal and determine which of the available methods produces the closest data set. Meanwhile, data providers might customize their methods for varying interpretations of the SDGs. Show less
Research managers benchmarking universities against international peers face the problem ofaffiliation disambiguation. Different databases have taken separate approaches to this problemand... Show moreResearch managers benchmarking universities against international peers face the problem ofaffiliation disambiguation. Different databases have taken separate approaches to this problemand discrepancies exist between them. Bibliometric data sources typically conduct adisambiguation process that unifies variant institutional names and those of its subunits so thatresearchers can then search all records from that institution using a single unified name. Thisstudy examined affiliation discrepancies between Scopus, Web of Science, Dimensions, andMicrosoft Academic for 18 Arab universities over a five-year period. We confirmed that digitalobject identifiers (DOIs) are suitable for extracting comparable scholarly material acrossdatabases and quantified the affiliation discrepancies between them. A substantial share ofrecords assigned to the selected universities in any one database were not assigned to the sameuniversity in another. The share of discrepancy was higher in the larger databases (Dimensionsand Microsoft Academic). The smaller, more selective databases (Scopus and especially Webof Science) tended to agree to a greater degree with affiliations in the other databases. Manualexamination of affiliation discrepancies showed that they were caused by a mixture of missingaffiliations, unification differences, and assignation of records to the wrong institution. Show less
This study investigates the scientific mobility and international collaboration networks in the Middle East and North Africa (MENA) region between 2008 and 2017. By using affiliation metadata... Show moreThis study investigates the scientific mobility and international collaboration networks in the Middle East and North Africa (MENA) region between 2008 and 2017. By using affiliation metadata available in scientific publications, we analyze international scientific mobility flows and collaboration linkages. Three complementary approaches allow us to obtain a detailed characterization of scientific mobility. First, we uncover the main destinations and origins of mobile scholars for each country. Results reveal geographical, cultural and historical proximities. Cooperation programs also contribute to explain some of the observed flows. Second, we use the academic age. The average academic age of migrant scholars in MENA was about 12.4 years. The academic age group 6-10 years is the most common for both emigrant and immigrant scholars. Immigrants are relatively younger than emigrants, except for Iran, Palestine, Lebanon, and Turkey. Scholars who migrated to Gulf Cooperation Council countries, Jordan, and Morocco were on average younger than emigrants by 1.5 years from the same countries. Third, we analyze gender differences. We observe a clear gender gap: Male scholars represent the largest group of migrants in MENA. We conclude by discussing the policy relevance of the scientific mobility and collaboration aspects. Show less
El Ouahi, J.; Robinson-García, N.; Costas Comesana, R. 2021
We present a large-scale comparison of five multidisciplinary bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. The comparison considers scientific... Show moreWe present a large-scale comparison of five multidisciplinary bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. The comparison considers scientific documents from the period 2008-2017 covered by these data sources. Scopus is compared in a pairwise manner with each of the other data sources. We first analyze differences between the data sources in the coverage of documents, focusing for instance on differences over time, differences per document type, and differences per discipline. We then study differences in the completeness and accuracy of citation links. Based on our analysis, we discuss the strengths and weaknesses of the different data sources. We emphasize the importance of combining a comprehensive coverage of the scientific literature with a flexible set of filters for making selections of the literature. Show less
Ahlgren, P.; Chen, Y.; Colliander, C.; Eck, N.J.P. van 2020
The effects of enhancing direct citations, with respect to publication–publication relatedness measurement, by indirect citation relations (bibliographic coupling, cocitation, and extended direct... Show moreThe effects of enhancing direct citations, with respect to publication–publication relatedness measurement, by indirect citation relations (bibliographic coupling, cocitation, and extended direct citations) and text relations on clustering solution accuracy are analyzed. For comparison, we include each approach that is involved in the enhancement of direct citations. In total, we investigate the relative performance of seven approaches. To evaluate the approaches we use a methodology proposed by earlier research. However, the evaluation criterion used is based on MeSH, one of the most sophisticated publication-level classification schemes available. We also introduce an approach, based on interpolated accuracy values, by which overall relative clustering solution accuracy can be studied. The results show that the cocitation approach has the worst performance, and that the direct citations approach is outperformed by the other five investigated approaches. The extended direct citations approach has the best performance, followed by an approach in which direct citations are enhanced by the BM25 textual relatedness measure. An approach that combines direct citations with bibliographic coupling and cocitation performs slightly better than the bibliographic coupling approach, which in turn has a better performance than the BM25 approach. Show less
Waltman, L.; Boyack, K.W.; Colavizza, G.; Eck, N.J. van 2020
There are many different relatedness measures, based for instance on citation relations or textual similarity, that can be used to cluster scientific publications. We propose a principled... Show moreThere are many different relatedness measures, based for instance on citation relations or textual similarity, that can be used to cluster scientific publications. We propose a principled methodology for evaluating the accuracy of clustering solutions obtained using these relatedness measures. We formally show that the proposed methodology has an important consistency property. The empirical analyses that we present are based on publications in the fields of cell biology, condensed matter physics, and economics. Using the BM25 text-based relatedness measure as the evaluation criterion, we find that bibliographic coupling relations yield more accurate clustering solutions than direct citation relations and cocitation relations. The so-called extended direct citation approach performs similarly to or slightly better than bibliographic coupling in terms of the accuracy of the resulting clustering solutions. The other way around, using a citation-based relatedness measure as evaluation criterion, BM25 turns out to yield more accurate clustering solutions than other text-based relatedness measures. Show less
The present study is an evaluation of three frequently used institution name disambiguation systems. The Web of Science normalized institution names and Organization Enhanced system and the Scopus... Show moreThe present study is an evaluation of three frequently used institution name disambiguation systems. The Web of Science normalized institution names and Organization Enhanced system and the Scopus Affiliation ID system are tested against a complete, independent institution disambiguation system for a sample of German public sector research organizations. The independent system is used as the gold standard in the evaluations that we perform. We study the coverage of the disambiguation systems and, in particular, the differences in a number of commonly used bibliometric indicators. The key finding is that for the sample institutions, the studied systems provide bibliometric indicator values that have only a limited accuracy. Our conclusion is that for any use with policy implications, additional data cleaning for disambiguating affiliation data is recommended. Show less
This article discusses a project under development called “Inventing Indicators of Interdisciplinarity,” as an example of work in methodology development that combines quantitative methods with... Show moreThis article discusses a project under development called “Inventing Indicators of Interdisciplinarity,” as an example of work in methodology development that combines quantitative methods with interpretative approaches in social and cultural research. Key to our project is the idea that Science and Technology Indicators not only have representative value, enabling empirical insight into fields of research and innovation but simultaneously have organizing capacity, as their deployment enables the curation of communities of interpretation. We begin with a discussion of concepts and methods for the analysis of interdisciplinarity in Science and Technology Studies (STS) and scientometrics, stressing that both fields recognize that interdisciplinarity is contested. To make possible a constructive exploration of interdisciplinarity as a contested—and transformative—phenomenon, we sketch out a methodological framework for the development and deployment of “engaging indicators.” We characterize this methodology of indicating as participatory, abductive, interactive, and informed by design, and emphasize that the method is inherently combinatory, as it brings together approaches from scientometrics, STS, and humanities research. In a final section, we test the potential of our approach in a pilot study of interdisciplinarity in AI, and offer reflections on digital mapping as a pathway towards indicating interdisciplinarity. Show less
