The function of conducting a direct replication may be different than typically assumed: understanding contextuality and asymmetry

Abstract

Direct replication, that is, analysis of new data with the same study protocol, has a purpose in checking the reliability of earlier research findings. However, using a case study we show that direct replication can have an additional function. We find that identical numerical replication results can be the result of differential effects of biases across different data sources and we draw attention to the risk of asymmetric evaluation in such cases. We then show how exact replications have the important function to bring into focus the contextualities inherent in the generation of research findings. Understanding the contextuality allows for a better and richer understanding of what the results imply.