Artificial Intelligence (AI) is a rapidly developing field of research that attracts significant funding from both the state and industry players. Such interest is driven by a wide range of AI... Show moreArtificial Intelligence (AI) is a rapidly developing field of research that attracts significant funding from both the state and industry players. Such interest is driven by a wide range of AI technology applications in many fields. Since many AI research topics relate to computer science, where a significant share of research results are published in conference proceedings, the same applies to AI. The world leaders in artificial intelligence research are China and the United States. The authors conducted a comparative analysis of the bibliometric indicators of AI conference papers from these two countries based on Scopus data. The analysis aimed to identify conferences that receive above-average citation rates and suggest publication strategies for authors from these countries to participate in conferences that are likely to provide better dissemination of their research results. The results showed that, although Chinese researchers publish more AI papers than those from the United States, US conference papers are cited more frequently. The authors also conducted a correlation analysis of the MNCS index, which revealed no high correlation between MNCS USA vs. MNCS China, MNCS China/MNCS USA vs. MSAR, and MNCS China/MNCS USA vs. CORE ranking indicators. Show less
This dissertation offers an in-depth evaluation of the Brazilian National System of Graduate Education (SNPG), overseen by the Brazilian Agency for Support and Evaluation of Graduate Education ... Show moreThis dissertation offers an in-depth evaluation of the Brazilian National System of Graduate Education (SNPG), overseen by the Brazilian Agency for Support and Evaluation of Graduate Education (CAPES). Anchored at the crossroads of research evaluation, scientometrics, and science policy, the study employs a multi-faceted methodology to dissect the complex dynamics between evaluation mechanisms, academic productivity, societal influence, and policy repercussions in Brazil.Organised into four parts, the study opens by tracing the historical evolution of graduate education and its evaluative mechanisms in Brazil. The following section introduces an international perspective to the country's research system and its evaluation, including a comparative analysis with the evaluation model practiced in The Netherlands. The third part adopts a mixed-methods analysis to examine fundamental evaluative instruments in the Brazilian context, notably those pertaining to scholarly publication metrics. The final part confronts the challenges precipitated by the SNPG's rapid expansion and advocates for self-assessment approaches as a means to integrate a more multi-dimensional evaluation framework.In its conclusion, the research cautions that Brazil's performance-oriented evaluation model, while fostering scientific development, may inadvertently stifle innovation and diversity. The author argues in favor of revising existing paradigms to adopt a more context-sensitive and holistic evaluative approach. Show less
Sandler, D.G.; Gladyrev, D.A.; Kochetkov, D.; Zorina, A.D. 2022
Relevance. One of the main goals of state university support programs in Russia is to increase the number of scientific publications. In 2021, Project 5-100 was replaced by the program PRIORITY... Show moreRelevance. One of the main goals of state university support programs in Russia is to increase the number of scientific publications. In 2021, Project 5-100 was replaced by the program PRIORITY 2030 (Strategic Academic Leadership Program). The new program increased the significance of the factors affecting the number of publications in universities and the issue of the optimal allocation of funding among research groups.Research objective. This study examines the factors that affect the productivity of research groups at the university. Unlike the majority of other studies on this topic, this study analyzes scientific productivity at the level of research groups.Data and methods. The study was possible due to the availability of data for 79 research groups at the Ural Federal University for the period from 2014 to 2020. The total number of articles and the number of articles in journals with an impact factor of more than two were used as indicators of research groups’ performance. To determine the factors influencing these indicators, we used econometric models for panel data. We used two separate samples: for social sciences and humanities and for other sciences.Results. We identified the following factors affecting the performance of research groups: the number of participants, the age of the research group, the supervisor’s scientific age, and the amount of funding (the possibility of obtaining more funds or being denied funds). The most interesting result is the following: the supervisor's scientific age and increased funding have a negative impact on the group’s performance. The article provides possible explanations for these results.Conclusion. Since the purpose of creating and funding research groups is primarily to increase their productivity, the results may be in favor of younger supervisors. University managers may also be interested in the ambiguous impact of increased funding: we suppose that research groups are more motivated not by the actual funding but by the prospective amount they may get. Show less
As the COVID-19 pandemic unfolds, researchers from all disciplines are coming together and contributing their expertise. CORD-19, a dataset of COVID-19 and coronavirus publications, has been made... Show moreAs the COVID-19 pandemic unfolds, researchers from all disciplines are coming together and contributing their expertise. CORD-19, a dataset of COVID-19 and coronavirus publications, has been made available alongside calls to help mine the information it contains and to create tools to search it more effectively. We analyse the delineation of the publications included in CORD-19 from a scientometric perspective. Based on a comparison to the Web of Science database, we find that CORD-19 provides an almost complete coverage of research on COVID-19 and coronaviruses. CORD-19 contains not only research that deals directly with COVID-19 and coronaviruses, but also research on viruses in general. Publications from CORD-19 focus mostly on a few well-defined research areas, in particular: coronaviruses (primarily SARS-CoV, MERS-CoV and SARS-CoV-2); public health and viral epidemics; molecular biology of viruses; influenza and other families of viruses; immunology and antivirals; clinical medicine. CORD-19 publications that appeared in 2020, especially editorials and letters, are disproportionately popular on social media. While we fully endorse the CORD-19 initiative, it is important to be aware that CORD-19 extends beyond research on COVID-19 and coronaviruses. Show less
The cognitive and social structures, and publication practices, of the humanities have been studied bibliometrically for the past 50 years. This article explores the conceptual frameworks, methods,... Show moreThe cognitive and social structures, and publication practices, of the humanities have been studied bibliometrically for the past 50 years. This article explores the conceptual frameworks, methods, and data sources used in bibliometrics to study the nature of the humanities, and its differences and similarities in comparison with other scientific domains. We give a historical overview of bibliometric scholarship between 1965 and 2018 that studies the humanities empirically and distinguishes between two periods in which the configuration of the bibliometric system differs remarkably. The first period, 1965 to the 1980s, is characterized by bibliometric methods embedded in a sociological theoretical framework, the development and use of the Price Index, and small samples of journal publications from which references are used as data sources. The second period, the 1980s to the present day, is characterized by a new intellectual hinterland-that of science policy and research evaluation-in which bibliometric methods become embedded. Here metadata of publications becomes the primary data source with which publication profiles of humanistic scholarly communities are analyzed. We unpack the differences between these two periods and critically discuss the analytical avenues that different approaches offer. Show less
To what extent is scientific research related to societal needs? To answer this crucial question systematically we need to contrast indicators of research priorities with indicators of societal... Show moreTo what extent is scientific research related to societal needs? To answer this crucial question systematically we need to contrast indicators of research priorities with indicators of societal needs. We focus on rice research and technology between 1983 and 2012. We combine quantitative methods that allow investigation of the relation between ‘revealed’ research priorities and ‘revealed’ societal demands, measured respectively by research output (publications) and national accounts of rice use and farmers’ and consumers’ rice-related needs. We employ new bibliometric data, methods and indicators to identify countries’ main rice research topics (priorities) from publications. For a panel of countries, we estimate the relation between revealed research priorities and revealed demands. We find that, across countries and time, societal demands explain a country's research trajectory to a limited extent. Some research priorities are nicely aligned to societal demands, confirming that science is partly related to societal needs. However, we find a relevant number of misalignments between the focus of rice research and revealed demands, crucially related to human consumption and nutrition. We discuss some implications for research policy. Show less
Mahieu, R.; Eck, N.J.P. van; Van Putten, D.; Van den Hoven, J. 2018
Our lives are increasingly intertwined with the digital realm, and with new technology, new ethical problems emerge. The academic field that addresses these problems—which we tentatively call ... Show moreOur lives are increasingly intertwined with the digital realm, and with new technology, new ethical problems emerge. The academic field that addresses these problems—which we tentatively call ‘digital ethics’—can be an important intellectual resource for policy making and regulation. This is why it is important to understand how the new ethical challenges of a digital society are being met by academic research. We have undertaken a scientometric analysis to arrive at a better understanding of the nature, scope and dynamics of the field of digital ethics. Our approach in this paper shows how the field of digital ethics is distributed over various academic disciplines. By first having experts select a collection of keywords central to digital ethics, we have generated a dataset of articles discussing these issues. This approach allows us to generate a scientometric visualisation of the field of digital ethics, without being constrained by any preconceived definitions of academic disciplines. We have first of all found that the number of publications pertaining to digital ethics is exponentially increasing. We furthermore established that whereas one may expect digital ethics to be a species of ethics, we in fact found that the various questions pertaining to digital ethics are predominantly being discussed in computer science, law and biomedical science. It is in these fields, more than in the independent field of ethics, that ethical discourse is being developed around concrete and often technical issues. Moreover, it appears that some important ethical values are very prominent in one field (e.g., autonomy in medical science), while being almost absent in others. We conclude that to get a thorough understanding of, and grip on, all the hard ethical questions of a digital society, ethicists, policy makers and legal scholars will need to familiarize themselves with the concrete and practical work that is being done across a range of different scientific fields to deal with these questions. Show less
In developing countries, researchers with strong international links potentially act as a double-edge sword. On the one hand, local researchers with international links could strengthen the... Show moreIn developing countries, researchers with strong international links potentially act as a double-edge sword. On the one hand, local researchers with international links could strengthen the research base of an institution or country while, on the other hand, they could leave the research base vulnerable should they migrate. The study identified internationally linked authors in Uganda, East Africa, by applying individual-level bibliometrics to a dataset of 3,948 Ugandan authors from the Web of Science, for the period 2011–2015. The focus was on four overlapping groups of internationally linked authors: (1) Ugandan authors with an international co-author, (2) Uganda authors with a joint international affiliation, (3) Ugandan authors affiliated with an international organisation that has a local address, and (4) Ugandan authors affiliated with an international research partnership. The study showed that without the identified forms of international linkages, the Ugandan scholarly workforce would reduce to 14% of its current size. Moreover, 74% of Ugandan authors without any international links had co-authored articles with Ugandan authors who are linked internationally. Although the extent of both international and national mobility associated with internationally linked authors seem low, benchmarking against comparative figures for other countries in sub-Saharan Africa is required. Show less
In this paper, we lay out a problem statement for the challenges that altmetrics providers face in making the data they aggregate as transparent as the community desires. We use Altmetric as a case... Show moreIn this paper, we lay out a problem statement for the challenges that altmetrics providers face in making the data they aggregate as transparent as the community desires. We use Altmetric as a case study for exploring altmetrics’ complexity and the related issues that providers face in communicating that complexity to the diverse stakeholders they serve. We then pose questions for consideration by the larger scientometrics community, with an aim towards achieving a consensus solution (or set of solutions). Show less
Tingcan, M.; Ruinan, L.; Guiyan, O.; Mingliang, Y. 2018
Research team identification is an essential task in many scientific activities. This paper gives a research team identification method based on relationship analysis. The method first incorporates... Show moreResearch team identification is an essential task in many scientific activities. This paper gives a research team identification method based on relationship analysis. The method first incorporates both cooperation length (years) and outputs (papers) into two types of relationships, i.e., strong relationship and weak relationship. Then, research team is defined based on the two kinds of relationships to guarantee the tightness of the cooperation within the team. At last, an algorithm is proposed to find research teams from large set of researchers and case study is carried out to verify the effectiveness of the algorithm on finding actual teams. Show less
This study provides a comparison of the post-evaluation effects of national research evaluation frameworks in the UK and Italy at university (macro) and researcher (micro) level. We compare how... Show moreThis study provides a comparison of the post-evaluation effects of national research evaluation frameworks in the UK and Italy at university (macro) and researcher (micro) level. We compare how universities and researchers have responded to their evaluation frameworks, within the constraints of each university system and culture. The study draws on the policies informing the operationalisation of the UK’s Research Excellence Framework (REF) and Italy’s Valutazione della Qualità della Ricerca (VQR). We compare the national approaches to evaluation by analysing the purposes, the real use of outputs, the criticism, and effects of the respective evaluation exercises. This paper discusses these effects as part of a wider approach to influencing university and researchers’ outcomes towards desirable goals. With data collected via semi-structured interviews we explore how evaluation is used to drive research actors in desirable directions. With the review of literature surrounding the effects of evaluations on the national (mega), university (macro) and researcher (micro) levels and their integration, we explore the intertwined implications for the decisional system, the organizational structures, and the individual performance. Under this perspective we look at evaluation as a technique used to influence behaviour at the macro and micro research level. Comparing post-evaluation effects of different frameworks in different context allows, on one hand, identifying some specific country/culture related aspects. On the other, it allows identifying new understandings of how auditing research productivity by altering behaviours on macro and micro level, ingenerates intentional and unintentional changes at all levels. Show less
While PhD dissertations are a core part of establishing a career in academia the publication type PhD thesis is known from prior research to be very rarely cited. In this paper I study whether the... Show moreWhile PhD dissertations are a core part of establishing a career in academia the publication type PhD thesis is known from prior research to be very rarely cited. In this paper I study whether the citation data sparsity can be ameliorated by supplementing citation index data, in this case from Scopus, with a further citation data source covering different literatures, namely Google Books. A substantial amount of additional citations was obtained (60%). The differential increase across disciplines is discussed. However, dissertations still remain relatively rarely cited. Show less
We know a great deal about how to identify the topics that researchers are working on. One can use citations and/or text to identify about one hundred thousand document clusters from either the... Show moreWe know a great deal about how to identify the topics that researchers are working on. One can use citations and/or text to identify about one hundred thousand document clusters from either the Scopus database or the WoS database. For purposes of discussion, we refer to these document clusters as topics. In our models there are about a thousand very large topics and tens of thousands of small topics. But why doesn’t topic size follow an expected linear Zipfian distribution? Is it possible that there is a different organizing principle for large vs. small topics? In this study, we explore the possibility that the organizing principle for large topics is the continued use of very expensive tools (such as specialized equipment, specialized databases and specialized software). For our initial exploration, we use grant size (NSF or NIH grants in excess of $5 million annually) as proxy for preferential investment in specialized tools. By using links between 52,097 grants and tens of thousands of topics, we will test whether large topics get more (than expected) funding from large grants and, by inference, expensive tools are an organizing principle for large topics. Show less
Universities have undergone profound changes in the last decades. A shift towards more accountability and to “new public management” practices in the administration of universities took place and... Show moreUniversities have undergone profound changes in the last decades. A shift towards more accountability and to “new public management” practices in the administration of universities took place and led to an increase of the share of project funds in some countries and to the introduction of performance-based funding systems (PRFSs) in others. In all countries, research evaluation’s importance increases. However, while research evaluation is centralized in some countries, evaluation is organized at the institutional level only in others. Thus, the importance of research evaluation and how it is organised varies across countries. In this paper, we present a typology of national research evaluation systems in Europe that sheds light on the complex issue of national differences in the organisation of research evaluation. We use the data of a two-round Delphi survey among specialists in research evaluation as a basis of our analysis and apply Multiple Correspondence Analysis to create a typology of European research evaluation systems that includes a broad range of countries, including countries, for which not much information is available to the English-speaking research community and takes into account that during the last years also the SSH are more concerned with evaluations but the commonly applied evaluation instruments do not fit their research practices. We identify five types of research evaluation: “non-metric, non-SSH” (with Iceland as the best representor), “non-metric, SSH-specific” (Switzerland, “funding, non-metric” (Norway), “funding, metric” (Denmark) and “metric, English” (Estonia). The main result of this analysis is that the national organization of research evaluation system is a complex issue and the research evaluation landscape in Europe is diverse. Yet some components can be identified that define main types of research evaluation. A secondary result is that different types of research evaluation are linked to different conditions in countries. We therefore recommend, that designers of evaluation systems make a conscious link between the goals to achieve, the incentives to promote and the design of the evaluation system, rather than to strive to the unification of evaluation systems. Show less
This paper aims to increase our understanding of the phenomenon of corporate publishing. To do so, we first systematically review extant literature to identify which incentives lead firms to... Show moreThis paper aims to increase our understanding of the phenomenon of corporate publishing. To do so, we first systematically review extant literature to identify which incentives lead firms to contribute to publications. We then examine the publication activity of 2,500 firms (and of about 570,000 subsidiaries of these firms) across 20 sectors for the 2011-2015 period. The analysis focuses on characterising firms’ publications activity in terms of proportion of publishing firms by sector; collaboration (co-authorship) with academic institutions; and impact (citations). Show less
Establishing macroscopic effects of macro-level phenomena in science or science policy requires identifying macro-level change and causally ascribing it to the phenomena in question. The... Show moreEstablishing macroscopic effects of macro-level phenomena in science or science policy requires identifying macro-level change and causally ascribing it to the phenomena in question. The standardised survey that obtains data about samples of a larger population are one of the few instruments that can observe macro-level change. However, the utility of standardised surveys in science studies is limited because it is still impossible to include the variation of research practices and epistemic conditions of action under which they occur in survey-based studies. Progress in survey-based research partly depends on developing and empirically operationalising for standardised surveys a theoretically grounded comparative framework for research practices and epistemic conditions of action under which they occur. The aim of this paper is to offer a first step by reviewing epistemic conditions of action identified in qualitative studies, suggesting how they may affect causal relationships studied by surveys, and discussing opportunities to ask standardised questions about them. Show less
Collecting the right data to represent research impact is only half of the battle for policymakers and researchers. Only when it has gained broad acceptance among stakeholders, and is placed into... Show moreCollecting the right data to represent research impact is only half of the battle for policymakers and researchers. Only when it has gained broad acceptance among stakeholders, and is placed into the right part of a university’s organisational structure can institutional data become indicators, drivers of institutional change and measures of performance. This paper is details a four stage process for constructing and implementing new indicators for entrepreneurial activity in universities. The first is the consolidation of the concept, or vision of the university. The second is the formation of indicators following academic research, here exemplified by the Global Entrepreneurial University Metrics (GEUM) initiative. The third is building public awareness and acceptance of indicators, in this case by way of a university ranking. The final stage is building the institutional structures for implementing indicators and collecting and presenting results. Show less
Guindalini, C.; Lopes, Fonseca F.; Pereira Da Silva, M.V.; Machado Da Silva, A.; Paula Fonseca, B. de 2018
Zika virus (ZIKAV) infection has recently become a public health concern, as well as an enduring scientific, and technological (S&T) challenge for researchers worldwide. Especially in Latin... Show moreZika virus (ZIKAV) infection has recently become a public health concern, as well as an enduring scientific, and technological (S&T) challenge for researchers worldwide. Especially in Latin America and Caribbean (LAC) countries, greatly affected by the outbreak, understanding the S&T response is essential to guide future research and to direct financial investments. In the present study, we focused on ZIKAV epidemics using combined publication (from international and regional databases: Web of Science, Scopus, PubMed, SciELO and LILACS), patent (Orbit Intelligence) and social network analysis (co-authorship) to contribute to the discussion of the role of S&T knowledge produced and disseminated regionally to address global challenges. International databases searches retrieved 5,831 documents, evidencing a dramatic increase in scientific publications targeting ZIKAV from 2015 onwards. Overall, only 32% of the 282 unique documents indexed in SciELO and LILACS, were retrieved in the international databases. United States, France, and Brazil were the top three most central countries of the global network for ZIKAV research. In addition, United States was shown to be the most frequent collaborator of LAC countries. Patent search retrieved 311 patent families. So far, only four are from LAC organizations, all based in Brazil. Our analysis offered a comprehensive representation of ZIKAV’s research and development and showed that scientific efforts from LAC countries, in particular from Brazil, had a significant role in the generation of global knowledge. The study highlights the need for a continued interdisciplinary approach in order to improve our ability to prepare for, control and prevent future outbreaks. Show less
In this paper, we discuss how to use topical, scientific reference maps to overview and navigate in dynamic research landscapes and how to use such maps as input to strategic thinking. An automated... Show moreIn this paper, we discuss how to use topical, scientific reference maps to overview and navigate in dynamic research landscapes and how to use such maps as input to strategic thinking. An automated procedure is developed that creates a reference map from a set of selected scholarly papers covering a specified subject area. The method is based on co-occurrences of keywords and distributes keywords on a reference grid where related keywords appear close to another and vice versa. Once the reference map is created other entities such as individual scientific papers, researchers, and research groups may be projected to the map based on the keywords they use. An article may be represented as a point calculated as the mean x and y of all the keywords it contains. Similarly, individual researchers are plotted as the mean of all articles they have written; a research group as the mean of all researchers it includes and so forth. The centre of a distribution is supplemented with point cloud- or density maps in order to map the actual form and size of the distribution in the reference map. The maps can also show how a scientific field has evolved over the years, and potentially the current direction of research. A research institute, a research group or an individual researcher can map their abilities and competencies to the reference map and compare that to potential collaborators and competitors to visualise strengths and weaknesses. These maps may be used as an input or inspiration to strategic discussions where strategic positions or moves may be discussed, evaluated, and presented as strategic narratives: (1) stay and fortify; (2) invade and conquer; (3) merge and fuse subjects; (4) re-apply methodology from other areas; (5) explore and discover new land outside the reference map; and (6) invent new research areas. Show less
