As bibliographic reference managers like Mendeley made their data openly available, it became possible to track where in the world research was being saved from. This data offered the opportunity... Show moreAs bibliographic reference managers like Mendeley made their data openly available, it became possible to track where in the world research was being saved from. This data offered the opportunity to better understand how research circulates at a global scale with measures that go beyond citations. This paper explores this circulation by studying fluctuations in rankings between countries when they are based on mean normalized citation scores (MNCS) or on mean normalized Mendeley readership scores (MNRS). Results show that both indicators are moderately correlated at the country level, but that countries from the Global South (namely African and South American countries) perform better when ranked by Mendeley readership than by citations. In addition, publications from South America and Africa tend to have a lower citation impact compared to those from Europe and North America, even when compared with publications that have the same number of readers. These results suggest that the indicator chosen (i.e., citations or Mendeley readers) creates different (dis)advantages among scholarly actors (e.g. countries, research organizations, journals, etc.). It also hints at the need to establish evaluation frameworks that consider that different metrics play different roles across institutional and geographical boundaries. We conclude by proposing further ways of exploring these metrics. Show less
The field of altmetrics has grown impressively since its inception in 2010 with the Altmetrics Manifesto (Priem, Taraborelli, Groth, & Neylon, 2010). We now have regular altmetric conferences... Show moreThe field of altmetrics has grown impressively since its inception in 2010 with the Altmetrics Manifesto (Priem, Taraborelli, Groth, & Neylon, 2010). We now have regular altmetric conferences where academic and commercial data analysts and providers meet. A number of non-profit and for-profit platforms provide altmetric data and summarize these data in visually appealing presentations. This growth of altmetrics is partly fueled by the problems encountered in both peer review and indicator-based assessments of scientific activities, and also by the easy availability of novel types of digital data on publication and communication behavior of researchers and scholars. In this paper, we review and reflect on the state of the art with respect to these new altmetric data and indicators in the context of the evaluation of scientific and scholarly performance. Show less
Twitter users tweeting scholarly publications from different countries have been analysed. The aim is to explore how visible are different countries on Twitter (based on their self-assigned geo... Show moreTwitter users tweeting scholarly publications from different countries have been analysed. The aim is to explore how visible are different countries on Twitter (based on their self-assigned geo-locations obtained from altmetric.com) in comparison to their output size in the Web of Science. Some indicators such as Twitter presence and activity (such as number of user’s accounts, number of tweets, and number of publications tweeted) have been analysed for each country. Finally, the relationship between Twitter activity indicators and some demographic indicators (such as country’s population, education, internet users, ICT use and access) will be explored and potential factors affecting country’s activity on Twitter will be discussed. Exploring how technological access and development (technology orientation) of a country affects its scholarly twitter usage (scientific orientation) will help in interpreting country’s activity and in understanding the reasons why some countries are underrepresented on Twitter. Identification of such factors could help in gaining some insights on important barriers and limitations that may have an effect on usage of scholarly social media platforms by users from different countries and on considering ‘altmetric divide’ for any altmetrics applications at the country level. Show less
The country of authors of 5,9 million Web of Science (WoS) publications with DOI from the years 2012 to 2015 have been compared with the country of Twitter users tweeting these WoS publications in... Show moreThe country of authors of 5,9 million Web of Science (WoS) publications with DOI from the years 2012 to 2015 have been compared with the country of Twitter users tweeting these WoS publications in order to study the main scholarly users of Twitter across 10 different countries. For this purpose, the visibility of country’s publications in the WoS and geographical distribution of Twitter users tweeting WoS publications have been analysed. The aim is to study how do they differ and what are their preference in tweeting their own vs. other country’s publication. The findings show that in general, US and UK with the highest proportion of outputs in the WoS, are among the main users of Twitter as well. Moreover, except for US, users tweet publications affiliated to other country more than those from their own country. Also, similar to WoS, it seems that altmetric providers are not free of international biases in their coverage and collection of metrics. Finally, various possible reasons on why publications from some countries attract more Twitter users than others have been discussed. Show less
Zahedi, Z.; Haustein, S.; Larivière, V.; Costas Comesana, R. 2016
This research deals with investigating consistency of data across three altmetrics providers or aggregators: Altmetric.com, Mendeley and the Open Source software Lagotto (used by PLOS, CrossRef... Show moreThis research deals with investigating consistency of data across three altmetrics providers or aggregators: Altmetric.com, Mendeley and the Open Source software Lagotto (used by PLOS, CrossRef and others). The aim of this study is to explore if metrics for a same set of publications are consistent across them and if not, what are possible reasons that explain these differences. By consistency we mean having (reasonably) the same score for the same DOI per source across different altmetrics providers/aggregators. For a proper development of the altmetric research and practice, it is critical to understand any potential similarity or difference in metrics across different altmetric aggregators. For this purpose, a random sample of 30,000 Crossref (15,000) and WoS (15,000) DOIs from 2013 has been considered. The data collection has been done at the same date/time on July 23 2015 starting at 2 PM CEST using the Mendeley REST API, Altmetric.com dump file and the Lagotto open source application. Similar sources and metrics across these 3 providers have been analyzed and compared (Facebook, Twitter, Mendeley, CiteULike and Reddit). Show less
Fraumann, G.; Zahedi, Z.; Costas Comesana, R. 2015
This paper presents the results of a study in which we have analysed the topics of interest of Mendeley users (i.e. Students, PhDs, Post Docs, Researchers, Professors, Librarians, Lecturers &... Show moreThis paper presents the results of a study in which we have analysed the topics of interest of Mendeley users (i.e. Students, PhDs, Post Docs, Researchers, Professors, Librarians, Lecturers & other Professionals) using text mining and visualization techniques. Beside analyzing topics of interest of Mendeley users, we have also identified fields of science for which readership information can be an interesting source of information complementary to citation information. For this purpose, we have used WoS citation data and Mendeley readership data for a set of 980,698 WoS publications (articles and reviews) with a DOI from 20111.The VOSviewer software tool (Van Eck & Waltman, 2010) was used to create so-called overlay visualizations. These visualizations show additional information on top of a base map. Two types of base maps were used. A base map containing the 250 WoS subject categories was used to analyze differences in readership activity across research fields and to analyze differences in interest between types of users. Base maps containing terms extracted from titles and abstracts using the text mining functionality of VOSviewer (Van Eck & Waltman, 2011) were used to analyze differences in readership activity within research fields. Show less
In this study, the ‘academic status’ of users of scientific publications in Mendeley is explored in order to analyse the usage pattern of Mendeley users in terms of subject fields, citation and... Show moreIn this study, the ‘academic status’ of users of scientific publications in Mendeley is explored in order to analyse the usage pattern of Mendeley users in terms of subject fields, citation and readership impact. The main focus of this study is on studying the filtering capacity of Mendeley readership counts compared to journal citation scores in detecting highly cited WoS publications. Main finding suggests a faster reception of Mendeley readerships as compared to citations across 5 major field of science. The higher correlations of scientific users with citations indicate the similarity between reading and citation behaviour among these users. It is confirmed that Mendeley readership counts filter highly cited publications (PPtop 10%) better than journal citation scores in all subject fields and by most of user types. This result reinforces the potential role that Mendeley readerships could play for informing scientific and alternative impacts. Show less
The main focus of this paper is to investigate the impact of publications read (saved) by the different users in Mendeley in order to explore the extent to which their readership counts correlate... Show moreThe main focus of this paper is to investigate the impact of publications read (saved) by the different users in Mendeley in order to explore the extent to which their readership counts correlate with their citation indicators. The potential of filtering highly cited papers by Mendeley readerships and its different users have been also explored. For the analysis of the users, we have considered the information of the top three Mendeley ‘users’ reported by the Mendeley. Our results show that publications with Mendeley readerships tend to have higher citation and journal citation scores than publications without readerships. ‘Biomedical & health sciences’ and ‘Mathematics and computer science’ are the fields with respectively the most and the least readership activity in Mendeley. PhD students have the highest density of readerships per publication and Lecturers and Librarians have the lowest across all the different fields. Our precision-recall analysis indicates that in general, for publications with at least one reader in Mendeley, the capacity of readerships of filtering highly cited publications is better than (or at least as good as) Journal Citation Scores. We discuss the important limitation of Mendeley of only reporting the top three readers and not all of them in the potential development of indicators based on Mendeley and its users. Show less
The online reference manager tool Mendeley (http://www.mendeley.com/) is one of the most promising tools for altmetrics research (Li, Thelwall and Giustini, 2011;Wouters & Costas, 2012) and it... Show moreThe online reference manager tool Mendeley (http://www.mendeley.com/) is one of the most promising tools for altmetrics research (Li, Thelwall and Giustini, 2011;Wouters & Costas, 2012) and it has been already used in other previous studies, for example in Library and Information Science Journals (Bar-Ilan et al., 2012b), for Nature and Science journals (Li, Thelwall and Giustini, 2012); for PLoS ONE publications (Priem, Piwowar & Hemminger, 2012); and for a sample of all WOS disciplines (Zahedi, Costas & Wouters; 2013). The concept of Altmetrics was introduced by Priem et al. (2010) and it has been frequently referred to as an alternative way of measuring broader research impacts (other than citation) and ‘real time’ impact in social web via different tools. Most of studies investigated how altmetrics capture different type of impact compare to citations (some of them mentioned above); while in others the focus has been on how altmetrics can be used as predictor of citations. For example, in case of F1000, found that recommendations have a relatively lower predictive power in indicating high citedness as compared to journal citation scores (Waltman & Costas, 2013), moreover, correlation between F1000 labels and citation impact were not statistically significant in most cases (Mohammadi & Thelwall, 2013); also weak correlation among users’ tag and bookmarks as an indicator of journal usage and perception and citations observed for physical journals (Haustein, & Siebenlist, 2011). In the case of Mendeley, the correlation with citations has been observed to be higher (Bar-Ilan et al., 2012a; Bar-Ilan et al., 2012b; Priem, Piwowar & Hemminger, 2012; Li, Thelwall and Giustini, 2012; Li & Thelwall, 2012; Zahedi, Costas & Wouters; 2013), however, so far the relationship of the different types of readers with the impact of the publications has not yet been explored. For this reason, in this study, we present an exploratory analysis of the patterns of reading of the different types of users in Mendeley and we study their relationship with citations. Thus, our main objective is to know if there are different patterns in terms of impact depending on the different ‘career stages’, ‘disciplines’ and ‘countries’ of the readers in Mendeley. In the case of finding different types of impact and reading patterns among Mendeley readers, this could open the door to detect different types of impact (e.g. education impact or professional impact) and even to introduce the possibility of considering the different users as potential predicting elements of citations. Methodology & preliminary results: In this research we have studied two random samples of publications from the Web of Science: the first one containing 20,000 publications published between 2005 and 2011 from all disciplines, and the second sample include 200,000 publications published between 2011 and 2012 also from all disciplines. Both gathered during March and April 2013 via the Mendeley API and using the DOI of the publications as the linking element. For the two samples we have also calculated standard bibliometric indicators (Waltman et al., 2011). For the analysis of the users we have considered the information of the top three ‘career stage users’, ‘countries’ and ‘disciplines’ of the users. We acknowledge the limitation of counting only with the top three and we discuss this in the paper. Some preliminary results show that PhD students tend to read papers with higher impact than other users and also that they read more recent papers. Further research will be done in order to explore other potential factors (e.g. the higher presence of PhD students among the users of Mendeley) that can influence this observation. Show less
In this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics... Show moreIn this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics for 20,000 random publications from the Web of Science. We studied the presence and frequency of altmetrics in the set of publications, across fields, document types and also through the years. The main result of the study is that less than 50% of the publications have some kind of altmetrics. The source that provides most metrics is Mendeley, with metrics on readerships for around 37% of all the publications studied. Other sources only provide marginal information. Possibilities and limitations of these indicators are discussed and future research lines are outlined. We also assessed the accuracy of the data retrieved through Impact Story by focusing on the analysis of the accuracy of data from Mendeley; in a follow up study, the accuracy and validity of other data sources not included here will be assessed. Show less
