As bibliographic reference managers like Mendeley made their data openly available, it became possible to track where in the world research was being saved from. This data offered the opportunity... Show moreAs bibliographic reference managers like Mendeley made their data openly available, it became possible to track where in the world research was being saved from. This data offered the opportunity to better understand how research circulates at a global scale with measures that go beyond citations. This paper explores this circulation by studying fluctuations in rankings between countries when they are based on mean normalized citation scores (MNCS) or on mean normalized Mendeley readership scores (MNRS). Results show that both indicators are moderately correlated at the country level, but that countries from the Global South (namely African and South American countries) perform better when ranked by Mendeley readership than by citations. In addition, publications from South America and Africa tend to have a lower citation impact compared to those from Europe and North America, even when compared with publications that have the same number of readers. These results suggest that the indicator chosen (i.e., citations or Mendeley readers) creates different (dis)advantages among scholarly actors (e.g. countries, research organizations, journals, etc.). It also hints at the need to establish evaluation frameworks that consider that different metrics play different roles across institutional and geographical boundaries. We conclude by proposing further ways of exploring these metrics. Show less
Álvarez-Bornstein, B.; Bordons, M.; Costas, R.; Calero-Medina, C. 2018
This paper analyse the funding structure of seven countries that differ in their level of economic development, geo-political links, and R&D intensity (Brazil, Germany, The Netherlands, South... Show moreThis paper analyse the funding structure of seven countries that differ in their level of economic development, geo-political links, and R&D intensity (Brazil, Germany, The Netherlands, South Africa, South Korea, Spain and Sweden), in two biomedical disciplines (Cardiac & Cardiovascular Systems and Virology). The main objectives are (1)to provide a general overview of the research funding structure through the analysis of FAs recorded in publication, and (2) to explore what countries benefit more from international funding support, to what extent this support is associated to international collaboration and whether there are cross-country differences in the trend of countries to lead internationally funded research. Show less
Mongeon, P.; Xu, S.; Bowman, T.D.; Costas, R. 2018
The aim of this paper is to demonstrate how topical distance and social distance can provide meaningful results when analysing scholars’ tweets linking to scholarly publications. To do so, we... Show moreThe aim of this paper is to demonstrate how topical distance and social distance can provide meaningful results when analysing scholars’ tweets linking to scholarly publications. To do so, we analyse the social and topical distance between tweeted information science papers and their academic tweeters. This allows us to characterize the tweets of scientific papers, the tweeting behavior of scholars, and the relationship between tweets and citations. Show less
In this study the velocity of 12 Altmetric.com data sources in disseminating newly published research outputs is investigated. The Velocity Index is proposed to make a comparison of velocity among... Show moreIn this study the velocity of 12 Altmetric.com data sources in disseminating newly published research outputs is investigated. The Velocity Index is proposed to make a comparison of velocity among Altmetric.com data sources across document types and subject fields. Some altmetric posts accumulated very fast within the first few days after publication, such as Reddit, Twitter, News, and Facebook, while posts of Policy documents, Wikipedia, Q&A, and Peer review with low Velocity Index values accrued relatively slowly. Most data sources’ velocity degree also change by document types and subject fields. The velocity of most data sources confronted with the type of Review is lower than the overall and Article, while Editorial Material and Letter are higher. In general, most altmetric data sources show higher velocity values in the fields of Multidisciplinary Journals and Natural Sciences. Show less
We analyze the role of leadership and scientific collaborative relationships in constituting the disciplinary specialization between countries and its research performance. Authorship order... Show moreWe analyze the role of leadership and scientific collaborative relationships in constituting the disciplinary specialization between countries and its research performance. Authorship order provides critical information for the allocation of reward, while collaboration enables researchers to expand the network of co-authors, institutions, and countries involved in the research. Along with these factors, a country’s profile orientation within the global scientific market become of great importance to the development of countries. As bibliographic data embedded such important information about the changes in the position of authors in the byline of publications and the disciplines involved in the research, we analyze these changes over time—using a Web of Science dataset—to explore the extent to which collaboration relationships impact leadership and specialization on the scientific workforce. Using this data, we discern the importance of domestic and international outputs in determining the disciplinary structure in scientific relationships in terms of publications and citations. We found that different types of leadership translate in different results in terms of relative specialization and citations. Overall results show that non-leading internationally collaborative papers reach higher values than leading international and domestic papers especially remarkable in terms of citations. Although in general, all regions increase their performance when collaborating with leading partners, the largest differences in research performance by leadership are located in countries with the lowest investment in R&D. Countries with the highest research investment are more likely to serve as leaders and garner higher specialization and citations when they lead (domestic and lead authorship). Comparative analyses of the role of specialization between countries can be useful for informing policies and motivating further collaboration relationships in the definitions of research agendas. Show less
This article presents an exploratory analysis of which disciplines acknowledge more financial support (through funding acknowledgments - FA) and those whose publications attract more attention on... Show moreThis article presents an exploratory analysis of which disciplines acknowledge more financial support (through funding acknowledgments - FA) and those whose publications attract more attention on Twitter. We argue that such combined approach can provide interesting information for both funding bodies and policy makers about how funding activities and the attention in social media of scientific research relate to each other. Show less
Martín-Martín, A.; Costas, R.; Leeuwen, T. van; Delgado López-Cózar, E. 2018
The current ways in which documents are made freely accessible in the Web no longer adhere to the models established Budapest/Bethesda/Berlin (BBB) definitions of Open Access (OA). Since those... Show moreThe current ways in which documents are made freely accessible in the Web no longer adhere to the models established Budapest/Bethesda/Berlin (BBB) definitions of Open Access (OA). Since those definitions were established, OA-related terminology has expanded, trying to keep up with all the variants of OA publishing that are out there. However, the inconsistent and arbitrary terminology that is being used to refer to these variants are complicating communication about OA-related issues. This study intends to initiate a discussion on this issue, by proposing a conceptual model of OA. Our model features six different dimensions (authoritativeness, user rights, stability, immediacy, peer-review, and cost). Each dimension allows for a range of different options. We believe that by combining the options in these six dimensions, we can arrive at all the current variants of OA, while avoiding ambiguous and/or arbitrary terminology. This model can be an useful tool for funders and policy makers who need to decide exactly which aspects of OA are necessary for each specific scenario. Show less
The field of altmetrics has grown impressively since its inception in 2010 with the Altmetrics Manifesto (Priem, Taraborelli, Groth, & Neylon, 2010). We now have regular altmetric conferences... Show moreThe field of altmetrics has grown impressively since its inception in 2010 with the Altmetrics Manifesto (Priem, Taraborelli, Groth, & Neylon, 2010). We now have regular altmetric conferences where academic and commercial data analysts and providers meet. A number of non-profit and for-profit platforms provide altmetric data and summarize these data in visually appealing presentations. This growth of altmetrics is partly fueled by the problems encountered in both peer review and indicator-based assessments of scientific activities, and also by the easy availability of novel types of digital data on publication and communication behavior of researchers and scholars. In this paper, we review and reflect on the state of the art with respect to these new altmetric data and indicators in the context of the evaluation of scientific and scholarly performance. Show less
This study advances insights into the distribution of tweets captured by Altmetric.com in terms of the dates tweeting activity ensues. The following primary research question is investigated: When... Show moreThis study advances insights into the distribution of tweets captured by Altmetric.com in terms of the dates tweeting activity ensues. The following primary research question is investigated: When do tweeters tweet about scientific publications? For this work,the dataset in this study contained 25.2 million tweets, which were tweeted from January 2012 through December 2016. Those tweets referred to 3.3 million distinct scientific publications, which were published from 2012 onwards. In this work, tweet volume (TV)—tweets mentioning scientific publications in a specific time unit—and the publication volume (PV)—volume of distinct publications tweeted in a specific time unit—were calculated. The TV/PV ratio was calculated in order to measure the tweeting intensity in a given time unit. To observe differences in timely patterns, a heterogeneous selection of countries were selected, which included three Western European countries (the United Kingdom, the Netherlands, and Spain), a set of North African states (Morocco, Algeria, Tunisia, Libya and Egypt), as well as a representative of the Southern Hemisphere (South Africa). Results highlight the relevance of considering time dynamics of activity when studying social media metrics stemming from sources with a strong cultural or geographical component, as in the case of Twitter. Show less
The country of authors of 5,9 million Web of Science (WoS) publications with DOI from the years 2012 to 2015 have been compared with the country of Twitter users tweeting these WoS publications in... Show moreThe country of authors of 5,9 million Web of Science (WoS) publications with DOI from the years 2012 to 2015 have been compared with the country of Twitter users tweeting these WoS publications in order to study the main scholarly users of Twitter across 10 different countries. For this purpose, the visibility of country’s publications in the WoS and geographical distribution of Twitter users tweeting WoS publications have been analysed. The aim is to study how do they differ and what are their preference in tweeting their own vs. other country’s publication. The findings show that in general, US and UK with the highest proportion of outputs in the WoS, are among the main users of Twitter as well. Moreover, except for US, users tweet publications affiliated to other country more than those from their own country. Also, similar to WoS, it seems that altmetric providers are not free of international biases in their coverage and collection of metrics. Finally, various possible reasons on why publications from some countries attract more Twitter users than others have been discussed. Show less
Zahedi, Z.; Costas, R.; Larivière, V.; Haustein, S. 2016
This research deals with investigating consistency of data across three altmetrics providers or aggregators: Altmetric.com, Mendeley and the Open Source software Lagotto (used by PLOS, CrossRef... Show moreThis research deals with investigating consistency of data across three altmetrics providers or aggregators: Altmetric.com, Mendeley and the Open Source software Lagotto (used by PLOS, CrossRef and others). The aim of this study is to explore if metrics for a same set of publications are consistent across them and if not, what are possible reasons that explain these differences. By consistency we mean having (reasonably) the same score for the same DOI per source across different altmetrics providers/aggregators. For a proper development of the altmetric research and practice, it is critical to understand any potential similarity or difference in metrics across different altmetric aggregators. For this purpose, a random sample of 30,000 Crossref (15,000) and WoS (15,000) DOIs from 2013 has been considered. The data collection has been done at the same date/time on July 23 2015 starting at 2 PM CEST using the Mendeley REST API, Altmetric.com dump file and the Lagotto open source application. Similar sources and metrics across these 3 providers have been analyzed and compared (Facebook, Twitter, Mendeley, CiteULike and Reddit). Show less
In this study, the ‘academic status’ of users of scientific publications in Mendeley is explored in order to analyse the usage pattern of Mendeley users in terms of subject fields, citation and... Show moreIn this study, the ‘academic status’ of users of scientific publications in Mendeley is explored in order to analyse the usage pattern of Mendeley users in terms of subject fields, citation and readership impact. The main focus of this study is on studying the filtering capacity of Mendeley readership counts compared to journal citation scores in detecting highly cited WoS publications. Main finding suggests a faster reception of Mendeley readerships as compared to citations across 5 major field of science. The higher correlations of scientific users with citations indicate the similarity between reading and citation behaviour among these users. It is confirmed that Mendeley readership counts filter highly cited publications (PPtop 10%) better than journal citation scores in all subject fields and by most of user types. This result reinforces the potential role that Mendeley readerships could play for informing scientific and alternative impacts. Show less
The main focus of this paper is to investigate the impact of publications read (saved) by the different users in Mendeley in order to explore the extent to which their readership counts correlate... Show moreThe main focus of this paper is to investigate the impact of publications read (saved) by the different users in Mendeley in order to explore the extent to which their readership counts correlate with their citation indicators. The potential of filtering highly cited papers by Mendeley readerships and its different users have been also explored. For the analysis of the users, we have considered the information of the top three Mendeley ‘users’ reported by the Mendeley. Our results show that publications with Mendeley readerships tend to have higher citation and journal citation scores than publications without readerships. ‘Biomedical & health sciences’ and ‘Mathematics and computer science’ are the fields with respectively the most and the least readership activity in Mendeley. PhD students have the highest density of readerships per publication and Lecturers and Librarians have the lowest across all the different fields. Our precision-recall analysis indicates that in general, for publications with at least one reader in Mendeley, the capacity of readerships of filtering highly cited publications is better than (or at least as good as) Journal Citation Scores. We discuss the important limitation of Mendeley of only reporting the top three readers and not all of them in the potential development of indicators based on Mendeley and its users. Show less
The online reference manager tool Mendeley (http://www.mendeley.com/) is one of the most promising tools for altmetrics research (Li, Thelwall and Giustini, 2011;Wouters & Costas, 2012) and it... Show moreThe online reference manager tool Mendeley (http://www.mendeley.com/) is one of the most promising tools for altmetrics research (Li, Thelwall and Giustini, 2011;Wouters & Costas, 2012) and it has been already used in other previous studies, for example in Library and Information Science Journals (Bar-Ilan et al., 2012b), for Nature and Science journals (Li, Thelwall and Giustini, 2012); for PLoS ONE publications (Priem, Piwowar & Hemminger, 2012); and for a sample of all WOS disciplines (Zahedi, Costas & Wouters; 2013). The concept of Altmetrics was introduced by Priem et al. (2010) and it has been frequently referred to as an alternative way of measuring broader research impacts (other than citation) and ‘real time’ impact in social web via different tools. Most of studies investigated how altmetrics capture different type of impact compare to citations (some of them mentioned above); while in others the focus has been on how altmetrics can be used as predictor of citations. For example, in case of F1000, found that recommendations have a relatively lower predictive power in indicating high citedness as compared to journal citation scores (Waltman & Costas, 2013), moreover, correlation between F1000 labels and citation impact were not statistically significant in most cases (Mohammadi & Thelwall, 2013); also weak correlation among users’ tag and bookmarks as an indicator of journal usage and perception and citations observed for physical journals (Haustein, & Siebenlist, 2011). In the case of Mendeley, the correlation with citations has been observed to be higher (Bar-Ilan et al., 2012a; Bar-Ilan et al., 2012b; Priem, Piwowar & Hemminger, 2012; Li, Thelwall and Giustini, 2012; Li & Thelwall, 2012; Zahedi, Costas & Wouters; 2013), however, so far the relationship of the different types of readers with the impact of the publications has not yet been explored. For this reason, in this study, we present an exploratory analysis of the patterns of reading of the different types of users in Mendeley and we study their relationship with citations. Thus, our main objective is to know if there are different patterns in terms of impact depending on the different ‘career stages’, ‘disciplines’ and ‘countries’ of the readers in Mendeley. In the case of finding different types of impact and reading patterns among Mendeley readers, this could open the door to detect different types of impact (e.g. education impact or professional impact) and even to introduce the possibility of considering the different users as potential predicting elements of citations. Methodology & preliminary results: In this research we have studied two random samples of publications from the Web of Science: the first one containing 20,000 publications published between 2005 and 2011 from all disciplines, and the second sample include 200,000 publications published between 2011 and 2012 also from all disciplines. Both gathered during March and April 2013 via the Mendeley API and using the DOI of the publications as the linking element. For the two samples we have also calculated standard bibliometric indicators (Waltman et al., 2011). For the analysis of the users we have considered the information of the top three ‘career stage users’, ‘countries’ and ‘disciplines’ of the users. We acknowledge the limitation of counting only with the top three and we discuss this in the paper. Some preliminary results show that PhD students tend to read papers with higher impact than other users and also that they read more recent papers. Further research will be done in order to explore other potential factors (e.g. the higher presence of PhD students among the users of Mendeley) that can influence this observation. Show less
In this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics... Show moreIn this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics for 20,000 random publications from the Web of Science. We studied the presence and frequency of altmetrics in the set of publications, across fields, document types and also through the years. The main result of the study is that less than 50% of the publications have some kind of altmetrics. The source that provides most metrics is Mendeley, with metrics on readerships for around 37% of all the publications studied. Other sources only provide marginal information. Possibilities and limitations of these indicators are discussed and future research lines are outlined. We also assessed the accuracy of the data retrieved through Impact Story by focusing on the analysis of the accuracy of data from Mendeley; in a follow up study, the accuracy and validity of other data sources not included here will be assessed. Show less
