This research deals with investigating consistency of data across three altmetrics providers or aggregators: Altmetric.com, Mendeley and the Open Source software Lagotto (used by PLOS, CrossRef... Show moreThis research deals with investigating consistency of data across three altmetrics providers or aggregators: Altmetric.com, Mendeley and the Open Source software Lagotto (used by PLOS, CrossRef and others). The aim of this study is to explore if metrics for a same set of publications are consistent across them and if not, what are possible reasons that explain these differences. By consistency we mean having (reasonably) the same score for the same DOI per source across different altmetrics providers/aggregators. For a proper development of the altmetric research and practice, it is critical to understand any potential similarity or difference in metrics across different altmetric aggregators. For this purpose, a random sample of 30,000 Crossref (15,000) and WoS (15,000) DOIs from 2013 has been considered. The data collection has been done at the same date/time on July 23 2015 starting at 2 PM CEST using the Mendeley REST API, Altmetric.com dump file and the Lagotto open source application. Similar sources and metrics across these 3 providers have been analyzed and compared (Facebook, Twitter, Mendeley, CiteULike and Reddit). Show less
Fraumann, G.; Zahedi, Z.; Costas Comesana, R. 2015
This paper presents the results of a study in which we have analysed the topics of interest of Mendeley users (i.e. Students, PhDs, Post Docs, Researchers, Professors, Librarians, Lecturers &... Show moreThis paper presents the results of a study in which we have analysed the topics of interest of Mendeley users (i.e. Students, PhDs, Post Docs, Researchers, Professors, Librarians, Lecturers & other Professionals) using text mining and visualization techniques. Beside analyzing topics of interest of Mendeley users, we have also identified fields of science for which readership information can be an interesting source of information complementary to citation information. For this purpose, we have used WoS citation data and Mendeley readership data for a set of 980,698 WoS publications (articles and reviews) with a DOI from 20111.The VOSviewer software tool (Van Eck & Waltman, 2010) was used to create so-called overlay visualizations. These visualizations show additional information on top of a base map. Two types of base maps were used. A base map containing the 250 WoS subject categories was used to analyze differences in readership activity across research fields and to analyze differences in interest between types of users. Base maps containing terms extracted from titles and abstracts using the text mining functionality of VOSviewer (Van Eck & Waltman, 2011) were used to analyze differences in readership activity within research fields. Show less
Purpose – The purpose of this paper is to analyze the disciplinary orientation of scientific publications that were mentioned on different social media platforms, focussing on their differences and... Show morePurpose – The purpose of this paper is to analyze the disciplinary orientation of scientific publications that were mentioned on different social media platforms, focussing on their differences and similarities with citation counts.Design/methodology/approach – Social media metrics and readership counts, associated with 500,216 publications and their citation data from the Web of Science database, were collected from Altmetric.com and Mendeley. Results are presented through descriptive statistical analyses together with science maps generated with VOSviewer.Findings – The results confirm Mendeley as the most prevalent social media source with similar characteristics to citations in their distribution across fields and their density in average values per publication. The humanities, natural sciences, and engineering disciplines have a much lower presence of social media metrics. Twitter has a stronger focus on general medicine and social sciences. Other sources (blog, Facebook, Google+, and news media mentions) are more prominent in regards to multidisciplinary journals. Originality/value – This paper reinforces the relevance of Mendeley as a social media source for analytical purposes from a disciplinary perspective, being particularly relevant for the social sciences (together with Twitter). Key implications for the use of social media metrics on the evaluation of research performance (e.g. the concentration of some social media metrics, such as blogs, news items, etc., around multidisciplinary journals) are identified. Show less
In this study, the ‘academic status’ of users of scientific publications in Mendeley is explored in order to analyse the usage pattern of Mendeley users in terms of subject fields, citation and... Show moreIn this study, the ‘academic status’ of users of scientific publications in Mendeley is explored in order to analyse the usage pattern of Mendeley users in terms of subject fields, citation and readership impact. The main focus of this study is on studying the filtering capacity of Mendeley readership counts compared to journal citation scores in detecting highly cited WoS publications. Main finding suggests a faster reception of Mendeley readerships as compared to citations across 5 major field of science. The higher correlations of scientific users with citations indicate the similarity between reading and citation behaviour among these users. It is confirmed that Mendeley readership counts filter highly cited publications (PPtop 10%) better than journal citation scores in all subject fields and by most of user types. This result reinforces the potential role that Mendeley readerships could play for informing scientific and alternative impacts. Show less
An extensive analysis of the presence of different altmetric indicators provided by Altmetric.com across scientific fields is presented, particularly focusing on their relationship with citations.... Show moreAn extensive analysis of the presence of different altmetric indicators provided by Altmetric.com across scientific fields is presented, particularly focusing on their relationship with citations. Our results confirm that the presence and density of social media altmetric counts are still very low and not very frequent among scientific publications, with 15%-24% of the publications presenting some altmetric activity and concentrating in the most recent publications, although their presence is increasing over time. Publications from the social sciences, humanities and the medical and life sciences show the highest presence of altmetrics, indicating their potential value and interest for these fields. The analysis of the relationships between altmetrics and citations confirms previous claims of positive correlations but relatively weak, thus supporting the idea that altmetrics do not reflect the same concept of impact as citations. Also, altmetric counts do not always present a better filtering of highly cited publications than journal citation scores. Altmetrics scores (particularly mentions in blogs) are able to identify highly cited publications with higher levels of precision than journal citation scores (JCS), but they have a lower level of recall. The value of altmetrics as a complementary tool of citation analysis is highlighted, although more research is suggested to disentangle the potential meaning and value of altmetric indicators for research evaluation. Show less
The main focus of this paper is to investigate the impact of publications read (saved) by the different users in Mendeley in order to explore the extent to which their readership counts correlate... Show moreThe main focus of this paper is to investigate the impact of publications read (saved) by the different users in Mendeley in order to explore the extent to which their readership counts correlate with their citation indicators. The potential of filtering highly cited papers by Mendeley readerships and its different users have been also explored. For the analysis of the users, we have considered the information of the top three Mendeley ‘users’ reported by the Mendeley. Our results show that publications with Mendeley readerships tend to have higher citation and journal citation scores than publications without readerships. ‘Biomedical & health sciences’ and ‘Mathematics and computer science’ are the fields with respectively the most and the least readership activity in Mendeley. PhD students have the highest density of readerships per publication and Lecturers and Librarians have the lowest across all the different fields. Our precision-recall analysis indicates that in general, for publications with at least one reader in Mendeley, the capacity of readerships of filtering highly cited publications is better than (or at least as good as) Journal Citation Scores. We discuss the important limitation of Mendeley of only reporting the top three readers and not all of them in the potential development of indicators based on Mendeley and its users. Show less
The online reference manager tool Mendeley (http://www.mendeley.com/) is one of the most promising tools for altmetrics research (Li, Thelwall and Giustini, 2011;Wouters & Costas, 2012) and it... Show moreThe online reference manager tool Mendeley (http://www.mendeley.com/) is one of the most promising tools for altmetrics research (Li, Thelwall and Giustini, 2011;Wouters & Costas, 2012) and it has been already used in other previous studies, for example in Library and Information Science Journals (Bar-Ilan et al., 2012b), for Nature and Science journals (Li, Thelwall and Giustini, 2012); for PLoS ONE publications (Priem, Piwowar & Hemminger, 2012); and for a sample of all WOS disciplines (Zahedi, Costas & Wouters; 2013). The concept of Altmetrics was introduced by Priem et al. (2010) and it has been frequently referred to as an alternative way of measuring broader research impacts (other than citation) and ‘real time’ impact in social web via different tools. Most of studies investigated how altmetrics capture different type of impact compare to citations (some of them mentioned above); while in others the focus has been on how altmetrics can be used as predictor of citations. For example, in case of F1000, found that recommendations have a relatively lower predictive power in indicating high citedness as compared to journal citation scores (Waltman & Costas, 2013), moreover, correlation between F1000 labels and citation impact were not statistically significant in most cases (Mohammadi & Thelwall, 2013); also weak correlation among users’ tag and bookmarks as an indicator of journal usage and perception and citations observed for physical journals (Haustein, & Siebenlist, 2011). In the case of Mendeley, the correlation with citations has been observed to be higher (Bar-Ilan et al., 2012a; Bar-Ilan et al., 2012b; Priem, Piwowar & Hemminger, 2012; Li, Thelwall and Giustini, 2012; Li & Thelwall, 2012; Zahedi, Costas & Wouters; 2013), however, so far the relationship of the different types of readers with the impact of the publications has not yet been explored. For this reason, in this study, we present an exploratory analysis of the patterns of reading of the different types of users in Mendeley and we study their relationship with citations. Thus, our main objective is to know if there are different patterns in terms of impact depending on the different ‘career stages’, ‘disciplines’ and ‘countries’ of the readers in Mendeley. In the case of finding different types of impact and reading patterns among Mendeley readers, this could open the door to detect different types of impact (e.g. education impact or professional impact) and even to introduce the possibility of considering the different users as potential predicting elements of citations. Methodology & preliminary results: In this research we have studied two random samples of publications from the Web of Science: the first one containing 20,000 publications published between 2005 and 2011 from all disciplines, and the second sample include 200,000 publications published between 2011 and 2012 also from all disciplines. Both gathered during March and April 2013 via the Mendeley API and using the DOI of the publications as the linking element. For the two samples we have also calculated standard bibliometric indicators (Waltman et al., 2011). For the analysis of the users we have considered the information of the top three ‘career stage users’, ‘countries’ and ‘disciplines’ of the users. We acknowledge the limitation of counting only with the top three and we discuss this in the paper. Some preliminary results show that PhD students tend to read papers with higher impact than other users and also that they read more recent papers. Further research will be done in order to explore other potential factors (e.g. the higher presence of PhD students among the users of Mendeley) that can influence this observation. Show less
In this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics... Show moreIn this paper an analysis of the presence and possibilities of altmetrics for bibliometric and performance analysis is carried out. Using the web based tool Impact Story, we have collected metrics for 20,000 random publications from the Web of Science. We studied the presence and frequency of altmetrics in the set of publications, across fields, document types and also through the years. The main result of the study is that less than 50% of the publications have some kind of altmetrics. The source that provides most metrics is Mendeley, with metrics on readerships for around 37% of all the publications studied. Other sources only provide marginal information. Possibilities and limitations of these indicators are discussed and future research lines are outlined. We also assessed the accuracy of the data retrieved through Impact Story by focusing on the analysis of the accuracy of data from Mendeley; in a follow up study, the accuracy and validity of other data sources not included here will be assessed. Show less
Costas, R.; Meijer, I.; Zahedi, Z.; Wouters, P.F. 2013
