Location filtered citation counting: How much of a difference does it make in research evaluation?

Abstract

Previous studies stressed the importance to filter out nonessential in-text citations before assigning weight in order to improve the accuracy and effectiveness of frequency-weighted citation analysis, and proposed and tested filtering nonessential citations by their in-text frequency. The present study explores an alternative filtering method with a likely better accuracy rate, i.e., removing introductory and background sections, to see what a difference it makes in ranking authors by citations. We examined the correlations and rank changes of individual authors between rankings by traditional citation counting and those by in-text frequency-weighted citation counting before and after the filtration. We found that removing introductory and background sections alone doesn’t make much of a difference in author rankings, but it makes a huge difference when combined with frequency-weighted counting. This combination appears to make essential citations stand out, as shown by it ranking biomedicine-focused authors higher and bibliometrics-focused ones lower, except those who represent methods/tools/indicators/guidelines that were directly useful for studies of biomedical fields that apply bibliometrics. Interestingly, the present study also finds that the filtering and the weighing appear to have different effects as indicated by medium correlations between rankings by each separately. This difference warrants future detailed studies to identify the separate factors involved.