Background Clinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to... Show moreBackground Clinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to claim that a prediction model works well in the intended clinical context? Main body We argue to the contrary because (1) patient populations vary, (2) measurement procedures vary, and (3) populations and measurements change over time. Hence, we have to expect heterogeneity in model performance between locations and settings, and across time. It follows that prediction models are never truly validated. This does not imply that validation is not important. Rather, the current focus on developing new models should shift to a focus on more extensive, well-conducted, and well-reported validation studies of promising models. Conclusion Principled validation strategies are needed to understand and quantify heterogeneity, monitor performance over time, and update prediction models when appropriate. Such strategies will help to ensure that prediction models stay up-to-date and safe to support clinical decision-making. Show less
Wang, Q.K.; Runhaar, J.; Kloppenburg, M.; Boers, M.; Bijlsma, J.W.J.; Bierma-Zeinstra, S.M.A.; CREDO Expert Grp 2022
Objective: To internally and externally validate our diagnostic criteria of early stage knee osteoarthritis (OA) in the CHECK and OAI cohorts. Design: We applied two previously developed diagnostic... Show moreObjective: To internally and externally validate our diagnostic criteria of early stage knee osteoarthritis (OA) in the CHECK and OAI cohorts. Design: We applied two previously developed diagnostic models to all knees in CHECK and OAI cohorts to calculate probabilities of early stage knee OA at baseline. Knees were categorized into three groups based on probability: 'no OA' (probability <= 30%), 'uncertain' (probability between 30% and 70%) and 'early stage OA' (probability >= 70%). To validate the diagnosis, we obtained OA related outcome measures at 10-year follow-up in the CHECK cohort, and at 8-9-year follow-up in the OAI cohort. We compared outcome measures between 'no OA' and 'early stage OA' knees, and between 'no OA' and 'uncertain' knees using generalized estimating equations. Results: In CHECK (n = 1042 knees) both models showed 'early stage OA' knees presented with significant and clinically relevant higher WOMAC scores, higher Kellgren & Lawrence (KL) grade, and higher rates of joint space narrowing (JSN) progression after 10 years, compared to 'no OA' knees. In OAI (n = 2937 knees) both models showed 'early stage OA' knees presented with significant and clinically relevant higher WOMAC scores, higher KL grade, and higher rates of KL and JSN progression after 8-9 years, compared to 'no OA' knees. Smaller, but still significant differences between 'uncertain' and 'no OA' knees were observed in both cohorts. Conclusions: These results support internal and external validity of the two sets of diagnostic criteria for early stage knee OA. (C) 2022 The Author(s). Published by Elsevier Inc. Show less
