An office-based cardiovascular prediction model developed and validated in cohort studies of a middle-income country

Objective: Prediction models for cardiovascular disease (CVD) mortality come from high-income countries, comprising laboratory measurements, not suitable for resource-limited countries. This study aims to develop and validate a non-laboratory model to predict CVD mortality in a middle-income setting. Study design and setting: We used data of population aged 40-80 years from three cohort studies: Tehran Lipid and Glucose Study (n = 5160), Isfahan Cohort Study (n = 4350), and Golestan Cohort Study (n = 45,500). Using Cox proportional hazard models, we developed prediction models for men and women, separately. Cross-validation and bootstrapping procedures were applied. The models’ discrimination and calibration were assessed by concordance statistic (C-index) and calibration plot, respectively. We calculated the models’ sensitivity, speciﬁcity and net beneﬁt fraction in a threshold probability of 5%. Results: The 10-year CVD mortality risks were 5.1% (95%CI: 4.8-5.5) in men and 3.1% (95%CI: 2.9%-3.3%) in women. The optimism-corrected performance of the model was c = 0.774 in men and c = 0.798 in women. The models showed good calibration in both sexes, with a predicted-to-observed ratio of 1.07 in men and 1.09 in women. The sensitivity was 0.76 in men and 0.66 in women. The net beneﬁt fraction was higher in men compared to women (0.46 vs. 0.35). Conclusion: A low-cost model can discriminate well between low-and high-risk individuals, and can be used for screening in low-middle income countries.


Introduction
Cardiovascular disease (CVD) is one of the leading causes of death, as almost one-third of all deaths worldwide are attributed to them.More than 75% of CVD deaths occur in low-income and middle-income countries (LMICs) [1] .As a middle-income country, Iran suffers from a high incidence of CVD and CVD mortality [2] .Therefore, timely identifying the high-risk people for appropriate interventions is necessary and will lead to the most significant benefit.Consideration of a combination of multiple risk factors is more cost-effective than having interventions on single risk factors.Clinical CVD prevention guidelines suggest various CVD risk prediction models for screening high-risk individuals [3][4][5] .
Most prediction models include laboratory measurements in their functions to enhance model performance.Using such lab-based prediction models is costly and resource-intensive, so non-laboratory risk prediction models have been introduced as low cost, easy to implement, and feasible strategies to prevent CVD in low and middleincome countries with resource-constrain settings [ 3 , 6 ].Despite the benefits of non-laboratory prediction models, the number of such risk assessment models to predict CVD is limited [7] .Non-laboratory Framingham risk score [8] and the model developed using data from the Third National Health and Nutrition Examination Survey (NHANES III) [9] , Globorisk [10] , and WHO cardiovascular disease risk charts [11] are some examples.
Although fatal and non-fatal cardiovascular events are important for clinical and public health interventions, national data on death rates are more reliable than data for disease incidence, especially in developing countries in which non-laboratory-based risk scores are more beneficial [10] .Considering our access to three large populationbased cohort studies from Western Asia, the purpose of this study is to develop a prediction model based on nonlaboratory indicators to predict CVD mortality and evaluate its generalizability using internal-external cross-validation.

Study population
Individual-level data from three Iranian populationbased cohort studies were selected from the Iran Cohort Consortium ( www.irancohorts.ir).Data from multiple cohorts enhance the statistical power and allow for quantification of the between-cohort variation on coefficients.These studies are Tehran Lipid and Glucose Study (TLGS) as the first population-based cohort study, Isfahan Cohort Study (ICS), as the first cohort study designed explicitly for CVD, and Golestan Cohort Study (GCS) as the largest cohort study in Iran.The studies' profiles have been published elsewhere [12][13][14] .All these cohorts started between 1999-2004, and are still ongoing.The baseline characteristics of the included cohorts and the pooling process have been published before [2] .Supplementary Table 1 summarized the main characteristics of the included cohort studies.From the baseline population (n = 75104), among individuals aged 40 to 80 years (n = 60902), those who were free of CVD at baseline (n = 56708) were eligible for the current study.Excluding 459 participants with biologically implausible risk factor levels (BMI > 60 or BMI < 16 or SBP > 270 mmHg or SBP < 60 mmHg), and 1239 subjects who had no information for follow-up resulted in 55010 participants (23820 men); Fig. 1 shows the study flow chart.
Considering the high incidence rates of CVD mortality in cohorts under study (296-418 in 100000 person-year) [2] , the sample size was reasonable for model development.
The institutional review board of Tehran University of Medical Sciences, Tehran, Iran approved this study.Informed consent was obtained from the individuals in all cohorts.

Exposures
To find the best office-based predictors for CVD mortality, we considered all available variables potentially assumed as CVD risk factors and can be obtained from history or clinical examinations.
Candidate variables were selected through literature review, expert opinion, availability in the cohorts, and simplicity for measurement.The variables were sex, age, education status, systolic blood pressure (SBP), diastolic blood pressure (DBP), self-reported diabetes, smoking, anthropometric measures including waist circumference (WC), and body mass index (BMI).Some variables including age, educational status, history of diabetes and taking glucose-lowering medications, history of CVD, and cigarette smoking had been acquired by interviews at the initiation of the study.Interviews were done by trained experts, using checklists and questionnaires.Other required variables including anthropometric indices, systolic and diastolic blood pressure were obtained through clinical examinations using standard protocols.The details of measurements have been reported in study protocols [12][13][14] .
We defined self-reported diabetes as a past physician diagnosis and/or taking glucose-lowering medication.Smoking was considered as being a current smoker.BMI was calculated as the weight in kg divided by the square of the height in meters.Education was defined as 0: illiterate, 1: primary (1-5 years of schooling), 2: secondary (6-8 years), 3: high (9-12 years), and 4: university levels as an ordinal variable.

Outcome
As per cohorts' protocols, the events are being followed up by phone interviews.In the case of any new diseases leading to hospital admission or death, detailed information is gathered.The multi-professional specialists' committees in each cohort review the documents and confirm the diagnosis.Cardiovascular mortality was defined as death from ischemic heart disease (ICD10 codes I20-I25), sudden cardiac death (I46.1), or death from stroke (ICD10 codes I60-I69).We considered the time to event for each individual as the interval between the time of study entry and CVD death, the date of the latest follow-up, and/or date of death caused by other reasons, whichever had happened earlier.

Statistical analysis
A summary of the candidate predictors is reported as mean (S.D.) or frequency (percentage) at baseline.Supplementary Table 2 shows the range of missing values in each predictor throughout the cohort studies.Considering the low percentage of missing values ( < 2%), single imputation using the "impute" command in Stata was applied to handle missing values of the predictors based on all candidate variables for model development.
Using Cox proportional hazard models, stratified on cohorts, we developed two prediction models for men and women separately.To have an easy interpretation of regression model estimates, we centered the continuous variables [11] , with age at 60 years, BMI at 25 kg/m2, WC at 90 cm, education at the secondary level (ordinal level 2), and SBP at 120 mm Hg.In this way, based on the model, the baseline survival means the mean of survival in those with age 60 years (midpoint of 40 to 80) without any proposed risk factors (e.g.smoking and diabetes) and at the optimal level of proposed variables (e.g.BMI, WC, and SBP).We considered the ordinal variable of education as continuous covariate since the association was linear.We used the "mfp" command in STATA to combine backward elimination with the selection of multivariable fractional polynomial (MFP) functions [15] .To be conservative, the level of significance was set at 0.1 for inclusion/exclusion of variables and 0.01 for non-linearity.More details about the MFP are available in the supplementary.
The final model was developed on total study samples, with stratification by cohort studies.
The assumption of proportionality was checked by examining the global test of the Schoenfeld residuals [16] .Since cardiovascular disease hazard ratios often decrease with age [10] , we tested the interactions between age and all variables and selected the best model according to the Akaike information criterion (AIC).To account for the effect of other mortalities as competing risks, all steps were repeated using the Fine and Gray approach as well [17] .
The discriminatory power of the model was assessed by a concordance statistic (C-index).We calculated the optimism-corrected performance by bootstrap resampling validation, i.e. the selected model was evaluated both in the bootstrap sample and in the original sample; the mean of 200 differences between bootstrap samples and the original sample indicated the optimism.This optimism was subtracted from the performance of the original model [18] .
The sensitivity and specificity of the model were calculated for a probability threshold of 5%.The Kaplan-Meier estimator was used to estimate the true positive/negative and false positive/negative results [19] .The 95% CI was calculated using 1000 bootstrap resampling and estimated bootstrap standard errors.We assessed calibration in a calibration plot, i.e. how closely the observed risks corresponded to the predicted risks.
Since internal-external cross-validation assesses the generalizability of prediction models in large datasets, we applied this approach in the cohorts included.Firstly, the model was developed in TLGS and ICS, as the train set, and validated in GCS, as the test set, using the coefficients derived from the train set and the baseline survival from the test set; again this approach was used to develop the model in GCS and test the performance on the two cohorts that were left out; [ 1 , 20 ].So in the process of validation, different cohorts with different settings (Supplementary Table 1) were involved and the process of variable selection (backward stepwise method) was also applied.Of note, the eligibility criteria and the definition of outcome were the same throughout the cohorts and all the follow-up times were truncated at 10 years for calculating baseline survivals.
The models' potential usefulness in clinical practice was assessed in terms of net benefit [21] .The net benefit of a prediction model at a risk threshold is defined as (TPsw × FPs) /N where TP is a true positive result, FP as false positive, N is the number of the study population, and w is the harm-to-benefit ratio of the recommended treatment which equals to the odds of the risk threshold for that treatment [22] .We used the standardized net benefit (Net benefit fraction) which is the net benefit divided by the incidence of the outcome (CVD mortality in this study).In other words, the standardized net benefit is the true positive rate penalized by the false positive rate and means the net fraction of incidence that could be predicted and potentially prevented.( [ 23 , 24 ].Net benefit can be plotted against a range of threshold probabilities and is called a "decision curve".The decision curve shows the net benefit  Education was considered ordinal from illiterate (0) to primary school (1), secondary school (2), high school (3), and university (4) levels ‡ Self-reported diabetes was defined as a diagnosis by a physician or using glucose-lowering medications resulting from the decision making based on the predicted risk, versus decision making without the model (treating none or treating all of the population).We considered risk thresholds for treatment between 0 and 15% as the plausible range for treatment [ 25 , 26 ].All statistical analyses were conducted using Stata14 for Windows (Stata Corporation, College Station, Texas, USA).Two-sided P < 0.05 was considered statistically significant.

Results
A total of 55010 individuals were included from three cohorts.The mean ages were 53.6, 53.7, and 51.9 years in the study participants of TLGS, ICS, and GCS, respectively ( Table 1 ).Overall, the participants were similar across the cohorts regarding the baseline characteristics, except for the literacy levels with pronounced differences of illiteracy ranging from 8% in TLGS to 49% in GCS.The median [Interquartile range] of follow-up was 13.9 (10.1-14.5)years in TLGS, 11.3 (10.9-12.3) in ICS, and 9.1 (8.0-10.0) in GCS.
Table 2 displays the coefficients of the CVD mortality predictors in men and women.None of the fractional polynomial were significantly better than the linear model.‡ Age * SBP, Shows the interaction between age and systolic blood pressure.Baseline survival estimates at 10-years for men and women are 0.9631963 and 0.9852014, respectively.It means the mean of survival in those with age 60 years without any proposed risk factors (smoking and diabetes) and at the normal level of proposed variables (BMI, WC and SBP).
Diastolic blood pressure had no significant association with CVD mortality in both sexes, and BMI showed a significant negative association only in women.Age significantly modified the association between SBP and CVD mortality in both sexes, and with increasing age, this association decreased.In both sexes, education showed a negative association.
Table 3 shows an example that explains the process for estimating the CVD mortality risk and helps programmers to easily translate these models into the application.
Considering the competing risk, the results didn't change significantly, so we used the simple model for better understanding and more simplicity (Supplementary Table 3).
The validation results for the stability of the variable selection showed that the method is applicable with a mean value of discrimination 0.775 (95%CI: 0.750-0.779) in men and 0.778 (95%CI: 0.763-0.813) in women.
Fig. 2 depicts the calibration and discrimination of the models.The model showed a good calibration with a predicted-to-observed ratio of 1.07 in men and 1.09 in women.The apparent performance of the model was 0.776 (95%CI: 0.762-0.790) in men and 0.799 (95%CI: 0.784-0.813) in women; the optimism-corrected performances were nearly the same (0.775 in men and 0.798 in women).Using this model and considering the risk threshold of 5%, 8355 (35.1%) of men and 6261 (20.1%) of women were categorized as the high-risk population for CVD mortality.
Fig. 3 presents the results of cross-validations.Firstly, the model was developed using the data of TLGS and ICS (244 CVD deaths in men and 140 CVD deaths in women during 50167 and 58093 years of follow-up, respectively) and validated in GCS (908 CVD mortality in men and 788 CVD mortality in women during 167073 and 231556 years of follow-up, respectively).Secondly, the model was developed in GCS and validated in TLGS and ICS.Good agreements between observed outcomes and predictions were shown in both approaches of cross-validations.When the model was developed using TLGS and ICS data and validated in GCS, the observed-to-predicted ratio was 0.999 and 0.997 in men and women, respectively.The corresponding values were 1.005 and 1.007, when the model was developed using GCS data and cross-validated in TLGS and ICS.
Table 4 shows the sensitivity, specificity, and net benefit fraction at the risk threshold of 5%.The model showed more sensitivity (0.76 vs. 0.66) and more net benefit fraction (0.46 vs. 0.35) in men than women.
Fig. 4 shows the decision curves for each sex.Compared to strategies that either all or no patients took intervention, using the risk prediction model showed higher net benefit.The interventions could be useful when the absolute risk threshold was between 2%-15%.

Discussion
We developed and validated a non-laboratory risk prediction model for fatal CVD in a middle-income country.We showed that a simple office-based model has a good discrimination power and calibration in both genders To easy interpretation of regression model estimates, and simplify recalibration of the model in the new populations, continuous variables were centred (age centred at 60 years, Waist circumference at 90 cm, BMI at 25 kg/m ², education at 2 and systolic blood pressure at 120 mm Hg).
† Estimated 10-Y risk is calculated using this formula: Risk = 1-S0t exp ( ΣBiXi) for identifying those at high risk of fatal CVD.We also demonstrated its usefulness in a wide range of risk thresholds.Notably, the performance of the model did not change considerably in cross-validation.
Our risk prediction model has a good discrimination power of 0.775 in men and 0.798 in women, comparable with a previous risk prediction model for predicting CVD events developed [7] .Our model had higher discrimination power than the Framingham no-lab CVD risk  prediction model, which had a C-statistics of 0.749 in men and 0.785 in women in its original population [8] .However, the Swedish consultation-based risk prediction method and Gaziano non-lab-based algorithm had slightly higher discrimination powers [ 6 , 27 ].Similar results be-tween the optimized-corrected and original discrimination power indicated that overfitting is unlikely to be the leading explanation for our model's high performance.The variation in the endpoints of each prediction model may explain the differences in their discrimination power.While the endpoint was fatal CVD in this study, a variety of endpoints were used in the other risk prediction models [7] .For example, in the non-laboratory Framingham risk prediction model, the outcome was a coronary, cerebrovascular, or peripheral arterial disease as well as heart failure [8] .In contrast, the non-laboratory WHO/ISH risk prediction model has acute myocardial infarction and stroke (fatal and nonfatal) as its main outcomes [11] .Since non-lab variables have different prediction power for each endpoint, the models' performance can vary for each set of the endpoint.
Our findings are in line with previous studies showing the non-inferiority of office-based CVD risk prediction models compared to the lab-based CVD risk prediction models [ 6 , 9 , 28 ].An office-based model was developed using the data of more than 6000 participants aged between 25-74 years in the NHEFS study.The predictive discrimination of the model was comparable with that of the lab-based model.The analysis using cardiovascular death as the endpoint yielded a C-statistics of 0.820 for the laboratory model and 0.821 for the non-laboratory model in men.The same results for women were detected as C-statistics of 0.858 for lab-based and 0.860 for the no-lab-based model.The results indicated that using nolab-based models can make the risk assessment simpler in resource-limited countries in which laboratory testing is inconvenient [6] .
The office-based Framingham risk prediction model has been previously developed to predict the risk of all CVD events, including fatal and non-fatal [8] .The algorithm showed good calibration and discrimination (C statistic, 0.763 in men, and 0.793 in women).The predictors of this model were previously used to predict CVD mortality in Golestan Cohort Study.They showed no significant effects of BMI in the prediction of CVD mortality in men.The Area Under the Curve (AUC) was 0.772 (95% CI: 0.753-0.791) in women, and 0.763 (95% CI: 0.747-0.779) in men.The model showed good calibration in women while such an overestimation in men [29] .
This study has important implications.The risk assessment chart of PARS is the first CVD risk assessment tool developed in Iran using the data from ICS [30] ; however, for the first time, we used individual data of three large population-based cohort studies to develop a no-lab based prediction model.Having data from multiple cohorts with heterogeneity is a strength of our research that enhanced our statistical power and let us validate the results appropriately.Such variability should be appreciated when we want to make predictions, and the prediction model would be more discriminatory and clinically useful when the validation is done with a more heterogeneous population [18] .
We presented a no-lab CVD risk prediction model developed in a low-and middle-income country tailored to be used in the resource-constrained settings.Importantly this risk prediction model only needs easy-to-collect officebased measurements, which considerably improves the feasibility of the screening for high-risk individuals and lowers the related costs.The current national CVD prevention programs in Iran use the WHO/ISH cardiovascular risk prediction charts for the Eastern Mediterranean region, demanding glucose and lipid measurements in all screened populations [31] .Such a screening strategy exposes a considerable burden on the resources of the healthcare system in Iran.By replacing the lab-based prediction models with no-lab ones, a substantial amount of resources can be saved without a significant reduction in the screening strategy's performance.Recently, the WHO risk charts have been updated, and no-lab risk charts were added.This updating has been carried out at the regional level and not the countylevel because there was no-data for CVD events in many countries.
The choice of the fatal CVD as the endpoint for the current study enhances other researchers' ability for future research to externally validate the model in other populations, especially in low-and middle-income countries where there is no accurate system for recording the CVD events.Moreover, this risk prediction model accounts for the socioeconomic patterns in low-and middle-income countries by accounting for education status in the model.Education status can be easily measured in routine clinical practice.It can be used as a proxy variable for one's economy, access to healthcare services, and knowledge/attitudes toward health matters [ 32 , 33 ].There are no no-lab risk prediction models and few lab-based CVD risk prediction models that accounted for socioeconomic factors in their equations, which developed in the high-income countries [34] .
The current model showed a sensitivity of 0.76 and a net benefit fraction of 0.46 in men at the risk threshold of 5%.It means that using the model, as a prognostic test, results in the correct prediction of 76% of the events; however, the correct prediction and appropriate prevention in 46% of the events (CVD mortalities).Of note, this result is under the assumption of 100% effective preventive strategies [35] .The corresponding net benefit fraction in women was 0.35 that is lower than men; i.e around a net one-third of CVD mortalities could be predicted and prevented using the model appropriately.
The findings from this study should be interpreted considering its limitations.Some data on no-lab measurements, such as the family history of CVD, drug history (including statin therapy), physical activity, and nutrition, were not available or were not measured similarly in cohorts and therefore were not used for model development.However, history variables are self-reported and would not be so valid in practice.Physical activity and nutrition are not simple measurements and are too timeconsuming to be measured in the clinic.Although the developed model was cross-validated, it should be validated for other nationalities before implementation.It is supposed that the effect of risk factors are not changed during the time dramatically, however, the risk prediction model will need to be recalibrated with the updated mortality rates.

Conclusion
We developed and validated an office-based risk prediction model for fatal CVD in a middle-income country.We showed that a simple office-based model has a good discrimination power, calibration, and utility in both genders for identifying those at high risk of fatal CVD.The model could be validated in other settings, especially in developing countries with limited access to laboratory measurements.If scaled, such a model for screening high-risk individuals for CVD can potentially save a considerable amount of lives and resources.

Fig. 1 .
Fig. 1.Flow chart of the study population * Excluded because of age younger than 40 or older than 80 * * Excluded because of a positive history of cardiovascular diseases at baseline or unknown history Ɨ Excluded because of having BMI > 60 or BMI < 16 or SBP > 270 mmHg or SBP < 60 mmHg ƗƗ Excluded because of no follow-up information.

Fig. 2 .
Fig. 2. Calibration and discrimination of non-laboratory CVD mortality prediction model in all cohorts, men and women.

Fig. 3 .
Fig. 3. Performance of the cross-validated non-laboratory models in men and women.The upper graphs indicate the model's performance, developed in the GCS, and validated in TLGS and ICS.In lower graphs, the model was developed in TLGS and ICS, and validated in GCS.

Fig. 4 .
Fig. 4. Decision curve for the predicted probabilities of the non-laboratory based model in different thresholds."Treat none" indicates no prediction and no treatment and, thus, no benefit; "treat all" means treating all subjects regardless of any prediction.

Table 1 .
Information of candidate predictors at baseline by cohort and sex.

Table 2 .
The Coefficients of CVD mortality risk factors in non-laboratory based model in Iranian cohorts, by sex.
* None of the fractional polynomial were significantly better than the linear model.* * CI, Confidence interval, Ɨ SBP, Systolic blood pressure,

Table 3 .
Step-by-step process to estimate the 10-Year risk for CVD mortality using no-laboratory based model.MenExample: 65 years of age with systolic BP 140 mm Hg, waist circumference 104 cm, high school education (12 years of schooling), smoker and without history of diabetes.

Table 4 .
The performance of the non-laboratory based model at the threshold of 5% to predict cardiovascular mortality in Iranian cohorts.Net Benefit Fraction shows a sensitivity penalized for false-positive decisions.