Bias and uncertainty of the International Normalized Ratio determined with a whole blood point-of-care prothrombin time test device by comparison to a new International Standard for thromboplastin

.


Introduction
Many patients treated with vitamin K-antagonists (VKA) employ point-of-care (POC) devices for self-testing and self-management.In addition these systems are used for management by professionals in hospitals, physician offices, and anticoagulation clinics [1].Most POC systems for VKA monitoring are based on a modification of the Prothrombin Time (PT) and calculation of the International Normalized Ratio (INR) in a whole blood sample.
The most popular point-of-care PT-INR monitor in the Netherlands is the CoaguChek system (manufactured by Roche Diagnostics, Mannheim, Germany).All CoaguChek test strips imported in the Netherlands are validated by the Coagulation Reference Laboratory (CRL), Leiden University Medical Center, Leiden, in co-operation with several anticoagulant clinics [2].In this procedure, results are used to calculate the INR bias for each lot of imported test strips in relation to the appropriate International Standard for thromboplastin established by the World Health Organization (WHO).Before we reported on the assessment of bias in 54 consecutive test strip lots using the previous International Standard for thromboplastin coded rTF/09 as reference [3].In that study, we used only the average INR of patient's blood samples to calculate the mean bias for each new lot of test strips.
In the present study, we assessed the bias of 47 consecutive test strip lots at three different INR levels, i.e. 2.0, 3.0, and 4.0, using the current International Standard for thromboplastin coded rTF/16.These strip lots were introduced in the Netherlands in the years 2017, 2018, and 2019 and were validated for use according to the above mentioned methods [2,3].In the present paper, we estimated the uncertainty of the bias using the scatter of measurements about each orthogonal regression line used for validation [4,5].
According to Trautsch et al. [6], CoaguChek XS PT Test strips were calibrated by Roche to the most recent WHO International Standard.Roche received postmarketing results of too high INR test results with specific lot numbers.On 12th September, 2018, Roche promulgated a pressing medical device correction statement concerning CoaguChek XS PT Test Strip lot numbers 27216700 through 33449899 [6].On 2nd November, 2018, Roche promulgated a FDA class I recall of all concerned lot numbers [7].The company recommended to use different testing methods until correctly calibrated lots could be dispatched.In the present study, we included several test strip lots with lot numbers that were in the range 27216700 through 33449899 mentioned in Roche's medical device correction statement [6].

Materials and methods
The study has been approved by the Leiden University Medical Center Ethics Committee.Informed consent was obtained from all individuals included in this study.

Design
Validation of new lots of test strips was performed in two steps.In the first step, a reference lot (RL) of test strips was compared to the International Standard for thromboplastin using venous blood samples.In the second step, each new lot of test strips was compared to the reference lot using capillary blood samples.The first step was performed in the Coagulation Reference Laboratory (CRL) and the second step by a group of 3 thrombosis centers in the Netherlands.

Validation of reference lots
CoaguChek XS PT instruments were provided by Roche Diagnostics BV, Almere, the Netherlands.Each year, a fresh lot of CoaguChek XS PT test strips (imported by Roche Diagnostics Nederland BV, Almere, the Netherlands) was selected by the CRL to be utilized as a provisional reference lot (RL).Three provisional reference lots were utilized in the period 2017-2019.Each RL was compared to the World Health Organization (WHO) International Standard for thromboplastin, recombinant, human, coded rTF/16, using freshly collected venous blood samples from 20 healthy adult subjects and approximately 60 patients treated chronically with VKA [2].Healthy adults and patients were selected as described for the primary calibration procedure of PT systems [8].The International Standard (coded rTF/16) and instructions for use were provided by the National Institute for Biological Standards and Control (Hertfordshire, UK).A plastic syringe was used to collect venous blood.The blood was applied to a test strip of the provisional Reference Lot in the CoaguChek instrument.The INR was read from the instrument's screen and referred to as INR RL, V .Another venous blood specimen was obtained from the same puncture using a Monovette tube containing 0.106 mol/L sodium citrate (Ref 04.1922.001,Sarstedt, Nümbrecht, Germany).The citrated blood sample was centrifuged in the Monovette tube at 2800 ×g for 15 min.Each plasma was analyzed in the PT test using the International Standard with the manual tilt tube technique.The PT measured with the International Standard was converted to INR employing the mean normal PT of 20 healthy subjects and the established ISI of rTF/16 [8].

Comparison of consecutive test strip lot to reference lot
Samples of each fresh lot of CoaguChek XS PT test strips intended for use in the Netherlands were provided to CRL by Roche Diagnostics BV [2].The samples of each new lot were dispatched together with the prevailing reference lot to three anticoagulant clinics in the Netherlands.Each clinic was requested to measure INRs with the new lot (referred to as INR NL, C ) and the reference lot (referred to as INR RL, C ) using capillary blood samples from eight patients on long-term treatment with VKA.The results obtained by the three anticoagulant clinics were returned to the CRL for evaluation.The new lots were identified by sequential numbers according to the date of receipt by the CRL (see supplementary material).

Assessment of INR bias and uncertainty of bias
Bias is defined as the difference between the expectation of the test result and an accepted reference value.Bias can be assessed in a single patient's sample, but in the present study we determined bias as an average value obtained from multiple samples.The INR bias, i.
where PT RTF is the patient's prothrombin time, MNPT RTF is the mean normal PT, and ISI is the established International Sensitivity Index for rTF/16.Orthogonal regression analysis [5,8] was applied on log-transformed INR data pairs obtained from VKA treated patients, yielding the following relationship: where φ is log e (INR RTF ) and ψ is log e (INR RL, V ).
The bias B 1 for a given value ψ 0 is defined as where The INR bias between a new lot of test strips and the reference lot was assessed by comparing INRs of capillary blood samples.The INR read from the instrument's screen using the reference lot is represented by INR RL, C and the INR obtained with the new lot by INR NL, C .Orthogonal regression analysis was applied on log-transformed INR data pairs provided by the three thrombosis centers and the following relationship was obtained for each new lot of strips: where ζ is log e (INR RL, C ) and θ is log e (INR NL, C ).The bias B 2 for a given value θ 0 is defined as where For ψ 0 = ζ 0 , the bias B 3 between the new lot and rTF/16 can be calculated as.
where s 1 is the standard deviation around the orthogonal regression line, In a similar way the standard deviation of ζ 0 , i.e. s(ζ 0 ), was calculated as where s 2 is the standard deviation around the orthogonal regression line, N 2 the number of measurement pairs, θ m the average of the N 2 measurements of θ, and s d the standard deviation of d.
Since the bias B 1 is calculated according to Eq. ( 3), the standard deviation of B 1 is equal to the standard deviation of φ 0 , i.e.: Since the bias B 2 is calculated according to Eq. ( 5), the standard    The assessments of B 1 and B 2 are independent of each other.According to Eq. ( 6), the standard deviation of B 3 can be calculated from s (B 1 ) and s(B 2 ): For a given value of θ 0 the 95% confidence interval of φ 0 is: φ 0 ± 2 s (B 3 ).This interval corresponds to the following INR confidence interval: The relative INR bias (in %) of a reference lot was calculated with the formula: The relative INR bias (in %) of a new lot was calculated with the formula: 3. Results

Validation of reference lots
The relationships between log e (INR RL, V ) and log e (INR RTF ) according to Eq. ( 2) are shown in Table 1.The relative bias at high therapeutic levels (e.g.INR RTF = 4) for reference lot number 29494415, which has been used in 2018, was greater than the corresponding relative bias for the previous and subsequent reference lots used in 2017 and 2019, respectively.It should be noted that lot number 29494415 was one of the lots affected by the manufacturer's recall, whereas the other two reference lots were before and after the recall and were unaffected.INR difference plots for the three successive reference lots are shown in Figs. 1, 2 and 3, respectively.Linear regression lines of the individual INR differences plotted against the individual mean INR indicate the trend of the bias with increasing mean INR.For two reference lots the trends were statistically significant (P < 0.05), for the third reference lot it was not.

Assessment of bias
The number of capillary blood samples (N 2 ) for the comparison of new PT test strip lots to the prevailing RL ranged from 15 to 24 (see Supplementary file).
The relative bias for each new lot of test strips was assessed at different levels of INR RTF , i.e. 2.0, 3.0, and 4.0.The relative bias for all successive new lot numbers and the 95% confidence intervals for the bias is shown in Fig. 4. At INR RTF = 2.0, there was no trend of the relative bias with the sequence number of successive lots (Fig. 4a).At INR RTF greater than approximately 2.5, we observed an increase of the relative bias starting with sequence number 17, and a decrease following sequence number 27 (Fig. 4b and c).There were 5 lots with a bias 10-15% at INR = 4.0.In all strip lots the bias was <10% in the INR range 2.0-3.0.

Uncertainty of bias
The standard deviation of log e (INR) bias for new PT test strip lots, i.e. s(B 3 ) (see Eq. ( 9)), is shown in Table 2.The standard deviation of the bias at INR = 4.0 is greater than the standard deviations at INR = 2.0 and INR = 3.0, respectively.

Discussion
The purpose of our study was to estimate the bias of successive CoaguChek XS PT test strip lots at different INR levels.The estimated bias is the average difference in INR between the test strip calibrated by the manufacturer and the International Standard (rTF/16) procedure performed by the CRL.We used orthogonal regression lines of logtransformed INRs to estimate the bias.Most of the measured INRs used for the regression analysis were in the therapeutic range, i.e. between INR 2.0 and 3.5, as can be seen in Figs. 1, 2 and 3. Therefore, the reliability of estimating the INR bias is maximal within the therapeutic range.A limitation of our study was the relatively low number of INRs between 3.5 and 4.5, as a result of the excellent therapeutic control of patients by the thrombosis centers in the Netherlands.In our previous evaluation of our test strip validation, the bias was calculated as the mean value of all actual patients whose mean INR is always a value between 2.0 and 3.0 [2,3].Since the bias depends on the INR level, the mean bias at INR levels between 2.0 and 3.0 was always less than 10%.In this study we have assumed that there is no bias in the INR determined with the International Standard rTF/16.Although rTF/16 has been calibrated against predecessor International Standards, it represents the top of the traceability chain at the present time [9].A limitation of our study is that the comparison of the Reference Lots to the International Standard rTF/16 was performed by a single laboratory.In this respect it may be relevant to compare the results of our laboratory to those of others in previous studies.The operator who performed the reference lot validations against rTF/16 in the present study, participated also in a workshop on the manual tilt tube technique and was identified as operator no. 3 [10].It is reassuring that this operator's INR determination was very close to the mean INR of all operators in the workshop [10].
The standard deviation of the INR bias was estimated from the spread of individual patient's measurement results around the orthogonal regression line.Part of the scatter is due to analytical imprecision of both PT measurements with the International Standard rTF/16 and Coa-guChek XS PT measurements.The imprecision of the PT measured with rTF/16 and the harmonized manual tilt tube technique depends on the skill of the operator [10].The imprecision of the CoaguChek XS measurements in venous and capillary blood is approximately 2.5% and 4.0%, respectively [11].Another part of the scatter around the line is due to systematic deviations of each individual patient's relationship from the average relationship [12].In other words, the scatter is determined by both imprecision (random error) and patient specific deviations.One way to reduce the uncertainty of the mean bias is to increase the number of measurements (i.e., N 1 and/or N 2 ) according to Eqs. ( 7) and ( 8) (see Materials and methods section).N 2 may be increased if more laboratories participate in the lot-to-reference lot comparison.
We observed a temporary increase of the INR bias with a number of successive test strip lots released in the year 2018 (Fig. 4).These test strip lots, with sequence numbers 17 through 27, corresponded to the affected lot numbers in Roche Diagnostics' medical device correction statement [6].The temporary INR bias increase could hardly be detected at INR = 2.0, but was clearly observed at higher INRs (Fig. 4).The 95% confidence intervals of the INR bias of approximately 8 successive strip lots assessed in 2018 did not overlap with zero bias, suggesting that the bias was due to a systematic effect.An INR difference of 10% is considered as a critical difference in international guidelines [8,13].Several lots had an INR bias greater than 10% at INR = 4.0, i.e. greater than 0.4 INR (Fig. 4).In ISO document 17,593 it is stated that the bias between an anticoagulation monitoring system and the reference measurement procedure in the therapeutic interval (INR 2.0 to 4.5) shall be equal to or less than ±0.3 INR [14].It cannot be excluded that INR alteration due to test strip lot change influenced patient management, but based on the experience of one anticoagulation clinic, the Coa-guChek XS PT Test Strip calibration revision and class I recall did not result in a significant clinical impact [6].We think that a distinction should be made with respect to the sign (i.e.positive or negative) of the INR bias.A bias is called positive if the working PT system measures a higher INR than the Reference System rTF/16.A bias is called negative if the working PT system measures a lower INR than the Reference System rTF/16.A positive bias at the upper limit of the therapeutic range may result in VKA dose reduction or no dose change without increased bleeding risk.A negative bias at the upper limit of the therapeutic range may be associated with an increased bleeding risk if the clinician or the patient is not aware of the existence of the negative bias.It is reassuring that In the present study we did not find strip lots with a negative mean bias greater than 7% at INR 4.0 (Fig. 4).
The origin of the INR bias is not completely clear.Several potential explanations may be considered.One possible source of bias may be a change in the production process or calibration of the test strips.
Alternatively the bias might be due to confounding factors in the patients' samples used for the assessment.For example, it has been suggested that INR bias between various PT systems may be due to the presence of antiphospholipid syndrome (APS) or lupus anticoagulant [15][16][17].Other studies concluded that monitoring of VKA therapy with laboratory INR measurements seems to be suitable for the majority of APS patients [18,19].It is unlikely that the mean positive INR bias observed in this study was due to confounding factors in the patients' samples.
It has been shown that there may be variation in the PT determination with the Manual Tilt Tube technique (MTT) which is used with the International Standard for thromboplastin calibration [20].Bias and uncertainty in the INR determined with a non-harmonized MTT and the International Standard may be propagated to commercial PT systems.Harmonization of the MTT is an essential requirement for establishing a reference measurement system of the plasma prothrombin time [10].In the future, an internationally recognized Reference Measurement Procedure (RMP) for the MTT has to be established as part of a Reference Measurement System (RMS).It is to be expected that, when the RMS has been validated and adopted, occasional temporary INR bias increase in successive test strip lot calibration can be avoided to a large extent.
Information about deviant reagent lots can be useful for the manufacturers to detect possible problems with the lots, e.g.storage conditions and calibration issues.Based on the results from the lot evaluations the manufacturer may consider withdrawal of the lots from the market [21].

Conclusions
Validation of commercial PT-INR POC devices and their successive test strip lots is an important requirement.Assessment of INR bias can be performed by a reference laboratory using an agreed Reference Measurement System and a series of patients' blood samples.The uncertainty of the INR bias depends on several factors, e.g. the analytical imprecision, the number of blood samples and the spread of the INRs of the samples.The results of the present study demonstrate the robustness of the CoaguChek XS system for INR determinations in the therapeutic range of 2.0-3.0INR.

CRediT authorship contribution statement
A. van den Besselaar: study design, data analysis and manuscript writing.C. van Rijn: study design and data collection.C. Abdoel: data collection.C. Cobbaert: study design and manuscript writing.
The authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Table 2
Standard deviation of CoaguChek XS bias s(B 3 ).s(B 3 ) was calculated according to Eq. ( 9) for three values of INR RTF .The mean and the range for all lot numbers in each year is shown.
e. the average INR difference between a new test strip and the International Standard rTF/16, was assessed in two successive steps.In the first step the INR bias between a reference lot (RL) test strip and rTF/16 was assessed by comparing INRs of approximately 60 patients on VKA therapy in the central reference laboratory.In the second step the INR bias between a new test strip (NL) and the reference lot was assessed by comparing INRs of approximately 24 patients recruited by three collaborating thrombosis centers.In the first step, each patient's INR was read from the screen of the CoaguChek instrument operated with the reference lot (INR RL, V ), and the same patient's INR with the International Standard (rTF/16) was calculated according to the formula: 1 the number of measurement pairs, ψ m the average of the N 1 A.M.H.P. van den Besselaar et al. measurements of ψ, and s b the standard deviation of b.
Intercept a and slope b refer to orthogonal regression equation: log e (INR RTF ) = a + b log e (INR RL, V ).INR bias was calculated with Eq. (11).

Fig. 1 .
Fig. 1.INR difference plot for CoaguChek XS Reference Lot number 17619014.The mean INR of CoaguChek XS Reference Lot and the International Standard rTF/16 for each patient is plotted on the horizonal axis.A linear regression line for the INR difference on mean INR is shown in the plot.The Pearson correlation coefficient is − 0.285 (P < 0.05).

Fig. 2 .
Fig. 2. INR difference plot for CoaguChek XS Reference Lot number 29494415.The mean INR of CoaguChek XS Reference Lot and the International Standard rTF/16 for each patient is plotted on the horizonal axis.A linear regression line for the INR difference on mean INR is shown in the plot.The Pearson correlation coefficient is 0.535 (P < 0.001).

Fig. 3 .
Fig. 3. INR difference plot for CoaguChek XS Reference Lot number 36889311.The mean INR of CoaguChek XS Reference Lot and the International Standard rTF/16 for each patient is plotted on the horizonal axis.A linear regression line for the INR difference on mean INR is shown in the plot.The Pearson correlation coefficient is − 0.051 (P > 0.05).

Fig. 4 .
Fig. 4. INR bias of consecutive CoaguChek XS PT test strip lots relative to International Standard rTF/16 at three different values of INR RTF .The sequence number of each new PT test strip lot is plotted on the horizontal axis.The relative bias for each test strip lot is represented by a black symbol.Relative INR bias was estimated using Eq.(12).The 95% confidence interval of the relative bias is represented by vertical bars and was calculated from Eq. (10).Horizontal interrupted lines mark an INR bias of ±10%.Lower panel (a): INR bias at INR RTF = 2.0; middle panel (b): INR bias at INR RTF = 3.0; upper panel (c): INR bias at INR RTF = 4.0.

Table 1
Relationships between INR determined with CoaguChek XS reference lots (INR RL, V ) and INR determined with International Standard rTF/16 (INR RTF ).