All-cause mortality after ERCP

• Introduction
• Patients and methods
• Patients
• Analysis of data and statistics
• Validation of the English 30-day post-ERCP mortality risk model 8
• Development of a prediction model for all-cause 3 – 12-month mortality post-ERCP
• Results
• Validation of the English 30-day post-ERCP mortality risk model 8
• Development of a prediction model for all-cause 3 – 12-month mortality post-ERCP
• Discussion
• Appendix 1
• Recalibration of the English model for the Swedish data
• Assessment of the proportional hazards assumption in the developed Cox model
• References

Background and study aims: This study aimed to externally validate a recently developed English model for the prediction of 30-day mortality after endoscopic retrograde cholangiopancreatography (ERCP). Real-world mortality data beyond 30 days post-ERCP are scarce; thus, the study also aimed to develop a prediction model for mortality up to 12 months post-ERCP.

Patients and methods: All patients who underwent their first ERCP during a 3-year period (n = 16 478), as identified from the Swedish Hospital Discharge Registry, were linked to the Swedish Death Registry. Factors associated with all-cause mortality up to 12 months post-ERCP were identified by Cox proportional hazards analysis. A prediction model was developed.

Results: Post-ERCP mortality was 5 % at 30 days and increased to 11.9 % at 3 months. The English model slightly overpredicted 30-day mortality, which was corrected with recalibration. Discriminant validity of the recalibrated model was very good (c-statistic = 0.82). Independent predictors of medium-term mortality were: emergency admission (hazard ratio [HR] 1.48), cancer (HR 3.79), noncancer co-morbidity (1.33), gallstone-related diagnosis (HR 0.21), and age (HR 4.86 for ≥ 85 years vs. < 55 years). The c-statistic for 3 – 12-month mortality was 0.86 – 0.88. Specific ERCP complication codes were identified in 1.8 % of deaths within 30 days post-ERCP (0.09 % of ERCPs), and 75 % of deaths (18 % of ERCPs) within 1 year post-ERCP were due to cancer.

Conclusions: Mortality doubled from 30 days to 3 months post-ERCP, but it was attributed mainly to underlying disease, notably cancer, and infrequently to ERCP-related causes. A previously developed model predicting 30-day mortality was externally validated. A model accurately predicting mortality up to 12 months post-ERCP was developed.

Introduction

Endoscopic retrograde cholangiopancreatography (ERCP) is useful in the treatment of pancreaticobiliary conditions, but it is also related to a significant risk of complications ranging between 5 % and 12 % [1] [2] [3] [4] [5]. Thus, the selection of patients to undergo ERCP involves taking into consideration expected benefits and procedural risks, disease prognosis, and co-morbid illness.

Risk prediction models are important tools to inform decision making and management of patients with a particular condition [6]. All-cause 30-day mortality following ERCP ranges between 2 % and 5.9 % [1] [3] [4] [5] [7] [8] and, according to a recent registry study from England and Wales, increasing age, underlying cancer, co-morbid illness, and emergency admission for ERCP are major predictors of death [8]. In the same report, a bedside tool was developed for estimating 30-day mortality risk after first ERCP [8]. External validation is an essential step in the development of any risk model in order to evaluate its performance outside the development sample [6]; however, to date, this bedside prediction model [8] has not been externally validated.

Even large nationwide prospective ERCP databases do not routinely track patients after 30 days [2] [3]. Although potentially important for patient selection and the consent process, there are very few real-world data on mortality risk following ERCP beyond 30 days after the procedure.

The main aims of the current study were to externally validate the previously published 30-day post-ERCP mortality risk model [8], and to study medium-term (3 – 12 months) mortality following ERCP, including attempting to devise a prediction model with simple outcome predictors derived from administrative data for Swedish hospitals.

Patients and methods

Patients

All patient admissions in Swedish hospitals (inpatient and outpatient episodes) are registered in the Swedish Hospital Discharge Registry (HDR) [9]. Each record in the HDR corresponds to one admission episode, containing up to six discharge diagnoses, coded according to the International Classification of Diseases (v10 since 1997), and up to six surgical codes, according to the Swedish Classification of Operations and Major procedures [9]. The registry has been shown to have high levels of accuracy and completeness [10], with a dropout rate of < 1 % in 2006 [9]. Regarding ERCP data, according to a national administrative audit in 2008, the registry had an entry for 71.8 % of all ERCP procedures performed [11]. As no ERCPs are performed outside of a hospital setting in Sweden, this dropout can probably be explained by imperfect coding.

For the current study, the HDR was searched for all inpatient or outpatient episodes containing ERCP codes during a 4-year period between 1 January 2005 and 31 December 2008. All patients undergoing their first (index) ERCP, defined as the first appearance of one of these codes in a discharge record in the registry during the period 1 July 2005 to 30 June 2008, were included in the study cohort (cohort 1). Patients with their first procedures between 1 January 2005 and 30 June 2005 were excluded in order to ensure that no earlier ERCP had been performed within 6 months from the first procedure identified.

Data extracted from the registry were age at the time of index ERCP, sex, elective vs. emergency admission, and date of admission. Data in the HDR were also used to calculate the Charlson co-morbidity index, as previously described [12]. In order to summarize co-morbidity in the same way as in the original study in which the English 30-day mortality model was developed [8], a simple binary variable was devised to identify the presence of any noncancer co-morbidity included in the Charlson index. To establish the date of death, patients were linked to the Swedish Death Registry (searched until 31 December 2009), by means of the unique national identification number assigned to all Swedish residents. The Swedish Death Registry is essentially complete [13]. A part of this cohort has been previously used in a study exploring patient outcomes following ERCP performed for benign disease [14]. The protocol for the current study was approved by the local ethics committee.

Analysis of data and statistics

Data are presented as means (SD) or number (%) as appropriate, unless specified otherwise. In group comparisons, the chi-squared test was used for categorical variables and the Student’s t test was used for continuous variables. The McNemar test was used to assess significant changes in the proportions of cause-specific mortality between 30 days and 12 months after ERCP. Hospitals were defined as high ERCP volume centers if the mean annual number of first ERCPs was above the 75th percentile of all hospitals in the country during the study period.

Validation of the English 30-day post-ERCP mortality risk model [8]

Variables used in the development of the model in the original cohort from England and Wales [8] were assessed in cohort 1, and the probability of 30-day mortality of each patient according to the published model was calculated. As a measure of the distance between the predicted and observed all-cause 30-day post-ERCP mortality, the Brier score was computed [15]. The Brier score may range from 0 for a perfect model up to a maximum value beyond which the model is considered noninformative. The maximum Brier score for a noninformative model may be calculated taking into consideration the incidence of the outcome in the cohort (i. e. all-cause 30-day mortality) [15]. Discriminant validity was assessed as the c-statistic. Model calibration was assessed using the scatter plot between the predicted and observed mortality probabilities [15]. The calibration slope was calculated (i. e. the regression coefficient of the logit of the predicted mortality probability when it was entered into a linear regression model as the sole predictor of the observed mortality probabilities). The intercept of the regression model, which evaluates whether predictions are systematically too low or too high (calibration-in-the-large), was also calculated [15]. The model was recalibrated [16] to fit the Swedish data. Specifically, the model showed adequate calibration-in-the-large (as assessed by a negligible difference between the intercept of the calibration line with the observed mortality probability axis and that of the ideal 45-degree line in the scatter plot), but overestimated 30-day mortality by a certain percentage as assessed by means of the calibration slope. Recalibration was performed by means of shrinkage of all probabilities in the calculator of the prediction model (with multiplication by the calibration slope) [16]. The Brier score and c-statistic of the recalibrated model were subsequently assessed in cohort 1 and its calibration was again evaluated with the scatter plot between the predicted and observed mortality probabilities [15] [16].

Subsequently, the recalibrated model was further validated in a separate cohort of patients who underwent their first ERCP between 1 July 2008 and 30 September 2009 (cohort 2) and who were identified from the HDR in the same way as patients in cohort 1. Dead vs. alive status was also ascertained by linkage to the Swedish Death Registry, which was searched until the end of 2010.

Development of a prediction model for all-cause 3 – 12-month mortality post-ERCP

The potential relation of variables used in the development of the 30-day mortality risk model with medium-term survival was evaluated using Cox proportional hazards models. The presence of gallstone-related disease has been shown to be related to complications and 30-day post-ERCP survival [4] [14] [17] and, thus, this factor was tested as a potential predictor of medium-term mortality. Teaching hospital status and hospital ERCP volume status were also tested as potential predictors of medium-term mortality. Any factor that was related to survival at P < 0.1 in the univariable analysis was entered into a multivariable Cox proportional hazards model (with forward selection). Cox modeling was chosen instead of logistic regression analysis owing to the expected higher power of the former to detect mortality predictors and for simplicity, as the latter approach would most likely have led to a different model being developed for each time point (3, 6, and 12 months). Deaths occurring at ≤ 30 days after the first ERCP were not excluded from Cox regression analysis. Every variable in the final model was an independent predictor of mortality. The proportional hazards assumption was evaluated i) by means of covariate specific tests, in which the Schoenfeld residuals were regressed against time, ii) visually, by plotting Schoenfeld residuals against time [18], and iii) visually, with the help of log minus log plots.

For the calculation of a prognostic score, each variable in the multivariable Cox model was multiplied by its β coefficient and the products were summed. The baseline survival at 3, 6, and 12 months (survival for prognostic score = 0) was estimated so that survival probabilities at these time points may be calculated according to the equation:

S_(t) = S_0(t) ^exp(PS)

where t = time, S_0(t) = baseline survival for t, and PS = prognostic score [19].

The expected survival probability at 3, 6, and 12 months was also plotted as a function of the prognostic score. The coefficient R² _D, proposed by Royston and Sauerbrei, was calculated as a measure of explained variation [19]. Model discriminant validity was determined as the c-statistic for 3-, 6-, and 12-month survival. Model calibration was assessed by comparing predicted vs. actual survival by deciles of predicted survival.

The model was validated in cohort 2 identified from the HDR (i. e. the same cohort in which the recalibrated 30-day post-ERCP mortality model was validated).

All tests were two tailed and conducted at a 5 % significance level. All analyses were performed using the IBM SPSS statistical package (v.20; IBM Corp., Armonk, New York, USA).

Results

A total of 16478 first ERCPs performed in 66 hospitals were identified for cohort 1 ([Table 1]). Observed mortality doubled from 5 % at 30 days to 11.9 % at 3 months, rising further to 24.5 % at 12 months following first ERCP. Hepatobiliary and pancreatic cancer dominated as the main cause of death throughout the first year after ERCP, followed by other types of malignancies ([Table 2]). Gallstone-related or other benign hepatobiliary/pancreatic conditions were reported as the main cause of death in 6.1 % and 8.4 %, respectively, at 30 days, and decreased to 2.0 % and 4.0 % at 12 months postprocedure, respectively ([Table 2]). In only few cases of 30-day mortality was there a diagnosis recorded in the Death Registry that could be related to a specific post-ERCP complication (n = 15, 1.8 % of all deaths or 0.09 % of all first ERCPs at 30 days postprocedure) ([Table 3]). All but 2 of the 15 cases occurred as inhospital mortality during hospitalization for first ERCP. In all, 14/15 deaths recorded ERCP as a contributory factor and only 1/15 recorded it as the main cause of death. After 30 days post-ERCP, there were no new deaths up to 12 months that were coded in the death registry with a procedure complication code (Txx.x).

A total of 22 patients who died during the first 30 days after ERCP had acute cholangitis (with/without common bile duct stones) registered as the main cause of death ([Table 2]). Although these cases may not be considered directly related to ERCP with certainty, a link is possible. Two of these patients also had a complication code in their record in the Death Registry, and thus the sum of probable and possible cases of 30-day mortality related to ERCP in cohort 1 was 35 (4.3 % of all deaths or 0.2 % of all first ERCPs at 30 days postprocedure).

Validation of the English 30-day post-ERCP mortality risk model [8]

The presence of cancer-related diagnosis and noncancer co-morbidity were higher in the Swedish cohort 1 compared with the English derivation cohort (24.7 % vs. 14.6 % and 35.0 % vs. 22.4 %, respectively). The two cohorts were very similar in terms of patient demographics, type of admission, and the presence of gallstone-related diagnosis (data not shown for the English cohort). The observed all-cause 30-day post-ERCP mortality was also remarkably similar (5.0 % vs. 5.3 %, respectively) [8].

Application of the English 30-day post-ERCP prediction model in cohort 1 yielded satisfactory overall model characteristics, as assessed by means of the Brier score, and discriminant validity, as assessed by means of the c-statistic ( [Table1]). However, the model was not adequately calibrated in the Swedish cohort 1 ([Fig. 1a]).

Recalibration of the original model [8] was performed (see Appendix 1 for details). The resulting recalibrated model had the same Brier score and discriminant validity (c-statistic) in cohort 1 ([Table 1]), but appeared to fit the data better in the scatter plot of observed vs. predicted mortality probabilities ([Fig.1b]). The overall performance (i. e. Brier score) and discriminant validity of the recalibrated model were similar in cohort 2 ([Table 1]). A look-up table was generated for estimating all-cause 30-day mortality following first ERCP according to the recalibrated model ([Table 4]).

Development of a prediction model for all-cause 3 – 12-month mortality post-ERCP

Male sex, emergency admission at first ERCP, cancer diagnosis, older age, and noncancer co-morbid illness were predictors of survival in univariable analysis in cohort 1 (derivation cohort; [Table 5]). Noncancer co-morbidity was summarized as a simple binary variable, identifying the presence of any nonmalignant component of the Charlson co-morbidity index [8], in an attempt to ensure model simplicity. In a multivariable Cox proportional hazards model, emergency admission, cancer diagnosis, and noncancer co-morbidity were related to worse survival, while gallstone-related diagnosis was related to better survival ([Table 5]). Evaluation of the proportional hazards assumption yielded satisfactory results at least for the period of the first 18 months following ERCP, in which 86.5 % of the deaths observed during the study occurred (Appendix 1).

A prognostic score was calculated for each patient, in which each variable in the multivariable Cox model was multiplied by its β coefficient and the products were summed: prognostic score = 0.286 × emergency admission (if yes = 1, if no = 0) – 1.317 × gallstone-related diagnosis (if yes = 1, if no = 0) + 1.469 × cancer diagnosis (if yes = 1, if no = 0) + 0.321 × any nonmalignant co-morbidity (if yes = 1, if no = 0) + age factor, where age factor was: 0 if age < 55 years, 0.743 if age 55 – 64 years, 0.963 if age 65 – 74, 1.307 if age 75 – 84, and 1.711 if age > 84.

The baseline survival at 3, 6, and 12 months after first ERCP (survival for prognostic score = 0) was estimated so that survival probability at these time points for different prognostic scores could be calculated ([Table 6]). The model discriminant validity for survival at 3 – 12 months post-ERCP was very good ([Table 7]). The correlation between observed and predicted survival rates (model calibration) was also very good ([Table 7]; [Fig. 2a]). To facilitate use of the model in clinical practice, the predicted survival probability at 3, 6, and 12 months were also plotted as a function of the prognostic score ([Fig. 3]).

Application of the multivariable Cox proportional hazards model for post-ERCP survival in cohort 2 (validation cohort) showed similar model characteristics ([Table 5], [Table 7]; [Fig. 2b]).

Discussion

In this national registry linkage study, death from all causes following a first ERCP was 5.0 % at 30 days, which is in accordance with published data [3] [8]. To date, there have been no published reports focusing on mortality after 30 days; the current study showed that it doubles at 3 months and that almost a quarter of unselected patients with a first ERCP die within 12 months after the procedure. However, mortality appears to be mainly associated with underlying co-morbid conditions present in patients undergoing ERCP. This is reflected in the variables included in the previously developed English model predicting 30-day mortality post-ERCP [8], which was validated in the current independent population-based nationwide cohort of unselected patients with a first ERCP. It also holds true for mortality up to 12 months post-ERCP, as demonstrated in the prediction model developed with the same dataset. These prediction models showed very good discriminant validity, and they could hopefully help to inform the decision making and consent process when selecting patients to undergo ERCP.

The 30-day ERCP-related mortality was found to be 0.09 %. This is similar to numbers reported in previous reports from large population-based ERCP cohorts [2] [5] [8], as well as from large studies with prospectively collected data [17] [20] [21], in which it ranged between 0 % and 0.19 %. Although other investigators have reported somewhat higher ERCP-related mortality rates (up to 1.4 %) [4] [22], published data indicate that most mortality following ERCP is not directly related to the procedure. However, the current study also showed that the cause of death of 2.7 % of all patients with a first ERCP who die within 30 days postprocedure is acute cholangitis, which could, at least in part, be related to technical failure or delay in the performance of ERCP [14] [23]. Technical details and information on treatment delays were not available in the current study. Large prospective studies are warranted to investigate further the extent to which deaths occurring after ERCP may be related to the procedure. This would require registration not only of complications, but probably also of technical data, such as procedural failure in relation to preprocedural aims [14] [24].

This study showed that post-ERCP mortality was more frequent in older patients with co-morbid conditions and/or cancer, which is in line with previously published data on hospital admissions in general [25] and ERCP in particular [8]. This is not unexpected, as these patient characteristics are widely known to impact survival in general. The same is true for emergency admission, as it usually denotes severe disease such as acute cholangitis. All of these factors were identified in the previous registry study from England and Wales [8]. Interestingly, the presence of gallstone disease was a favorable patient characteristic for survival after first ERCP. The reasons for this are unclear, but they may be related to a decreased risk of post-ERCP complications according to a recent large prospective report [4]. Findings of the current study further showed that underlying cancer was the main cause of death in 18 % of patients during the first 12 months after their first ERCP procedure, accounting for almost 75 % of deaths during this period.

Hospital ERCP volume status was not a predictor of mortality up to 12 months in Cox regression analysis. This is in line with previous reports showing no relation between short-term mortality after ERCP in unselected patients and hospital volume status [7] [8] [26]. This fact, taken together with the very low mortality rates in patients with low prognostic scores (i. e. without risk factors identified in the Cox model), suggest that the use of post-ERCP mortality would probably not be informative as a quality indicator, and thus other measures and indicators would need to be utilized when assessing ERCP service quality of individual providers [27].

The strengths of the current study include its population-based nationwide design, as opposed to case enrollment in selected or volunteer institutions. The Swedish Death Registry, which is essentially complete [13], was used for ascertainment of dead/alive status, as well as date and cause of death. Despite the difference in case-mix between cohort 1 and 2, the models performed similarly in both cohorts. This strengthens the conclusion that the models described are relevant to the Swedish population [28].

The main study limitations are its retrospective nature and the lack of potentially useful clinical details and technical procedural data, including, but not limited to, complications and procedural failure. Further investigations are warranted to determine whether the prediction models used may be improved by including certain clinical and/or procedural parameters (although this is likely to increase model complexity), and to assess whether the models may be useful in cohorts from other countries with different healthcare systems. Furthermore, coding accuracy and completeness in the HDR may be of concern. However, the registry is under continuous quality control, and it is considered to contain valid and correct information [9] [10]. In addition, possible uncoded or miscoded procedures or other data in the registry are not expected to occur in a systematic manner in relation to the main outcome (i. e. mortality) or to case severity.

In conclusion, almost a quarter of unselected patients undergoing a first ERCP die within 12 months after the procedure. Mortality is mostly due to underlying disease, in particular cancer. This is reflected in both the variables of an English model predicting all-cause 30-day mortality after ERCP, which were externally validated in the current nationwide Swedish ERCP cohort, and the variables of a prediction model for mortality up to 12 months, which was developed in the current study. These prediction models may be useful in patient selection for ERCP, for example by identifying high-risk ERCP candidates who require close follow-up after the procedure, and in the informed consent process. Finally, the presented prediction models for post-ERCP mortality may be used by researchers when designing or interpreting studies in the field.

Appendix 1

Recalibration of the English model for the Swedish data

The English 30-day post-ERCP mortality model was not adequately calibrated in the Swedish cohort 1 ([Fig. 1]). The estimated calibration slope was 0.64 (i. e. the model overestimated 30-day mortality probabilities by 0.36, i. e. 36 %). The intercept of the calibration line with the observed mortality probability axis differed only by 0.0021 from that of the ideal 45-degree line (calibration-in-the-large), indicating a negligible degree of systematically higher predictions. Recalibration of the original model was performed by multiplying all probabilities in the calculator of the prediction model by the calibration slope (i. e. by 0.64). The recalibrated model appeared to fit the data better ([Fig. 1b]).

Assessment of the proportional hazards assumption in the developed Cox model

When assessing the proportional hazards assumption, regression of the Schoenfeld residuals against time yielded significant correlations for all covariates (P < 0.001), apart from emergency admission (P > 0.05). Although regression coefficients indicated weak correlations (– 0.08 to 0.18) and lines in log minus log plots did not cross each other, the plots of Schoenfeld residuals against time were also suggestive of slight, albeit significant, violation of the proportional hazards assumption (data not shown). In the latter plots, it was noted that there were relatively few events after approximately 18 months after the first ERCP, which may have had an impact on the results. Restricting analyses to events (mortality) occurring within 12 months from first ERCP, as the initial aim of the study was to assess mortality within 1 year from first ERCP, yielded nonsignificant correlations between Schoenfeld residuals and time (P > 0.05). Furthermore, log minus log plots yielded noncrossing lines for all covariates, and plots of Schoenfeld residuals against time were not suggestive of departure from the proportional hazards assumption.

Competing interests: None

• References

• 1 Williams EJ, Taylor S, Fairclough P et al. Are we meeting the standards set for endoscopy? Results of a large-scale prospective survey of endoscopic retrograde cholangio-pancreatograph practice. Gut 2007; 56: 821-829
  CrossrefPubMedSearch in Google Scholar
• 2 Kapral C, Duller C, Wewalka F et al. Case volume and outcome of endoscopic retrograde cholangiopancreatography: results of a nationwide Austrian benchmarking project. Endoscopy 2008; 40: 625-630
  Thieme ConnectPubMedSearch in Google Scholar
• 3 Enochsson L, Swahn F, Arnelo U et al. Nationwide, population-based data from 11,074 ERCP procedures from the Swedish Registry for Gallstone Surgery and ERCP. Gastrointest Endosc 2010; 72: 1175-1184.e3
  CrossrefPubMedSearch in Google Scholar
• 4 Glomsaker T, Hoff G, Kvaloy JT et al. Patterns and predictive factors of complications after endoscopic retrograde cholangiopancreatography. Br J Surg 2013; 100: 373-380
  CrossrefPubMedSearch in Google Scholar
• 5 Coelho-Prabhu N, Shah ND, Van Houten H et al. Endoscopic retrograde cholangiopancreatography: utilisation and outcomes in a 10-year population-based cohort. BMJ Open 2013; 3: e002689
  CrossrefPubMedSearch in Google Scholar
• 6 Debray TP, Vergouwe Y, Koffijberg H et al. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol 2015; 68: 279-289
  CrossrefPubMedSearch in Google Scholar
• 7 Cote GA, Imler TD, Xu H et al. Lower provider volume is associated with higher failure rates for endoscopic retrograde cholangiopancreatography. Med Care 2013; 51: 1040-1047
  CrossrefPubMedSearch in Google Scholar
• 8 Bodger K, Bowering K, Sarkar S et al. All-cause mortality after first ERCP in England: clinically guided analysis of hospital episode statistics with linkage to registry of death. Gastrointest Endosc 2011; 74: 825-833
  CrossrefPubMedSearch in Google Scholar
• 9 Jacobsson A. The National Patient Register. Stockholm: The National Board of Health and Welfare (Socialstyrelsen); 2014 Available from: http://www.socialstyrelsen.se/register/halsodataregister/patientregistret/inenglish
  PubMedSearch in Google Scholar
• 10 Nilsson AC, Spetz CL, Carsjo K et al. [Reliability of the hospital registry. The diagnostic data are better than their reputation]. Lakartidningen 1994; 91: 598, 603-595
  PubMedSearch in Google Scholar
• 11 Nyqvist K, Passmark H, Muzollo T. [Attachment to Comparisons regarding Quality and Effectiveness in Health Care 2009]. Stockholm: The National Board of Health and Welfare (Socialstyrelsen); 2009 Available from: http://webbutik.skl.se/sv/artiklar/halso-och-sjukvard/bilagor-till-oppna-jamforelser-av-halso-och-sjukvardens-kvalitet-och-effektivitet-2009-indikatorbesk.html
  PubMedSearch in Google Scholar
• 12 Quan H, Sundararajan V, Halfon P et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005; 43: 1130-1139
  CrossrefPubMedSearch in Google Scholar
• 13 Mattsson B, Rutqvist LE, Wallgren A. Undernotification of diagnosed cancer cases to the Stockholm Cancer Registry. Int J Epidemiol 1985; 14: 64-69
  CrossrefPubMedSearch in Google Scholar
• 14 Kalaitzakis E, Toth E. Hospital volume status is related to technical failure and all-cause mortality following ERCP for benign disease. Dig Dis Sci 2015; 60: 1793-1800
  CrossrefPubMedSearch in Google Scholar
• 15 Steyerberg EW, Vickers AJ, Cook NR et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21: 128-138
  CrossrefPubMedSearch in Google Scholar
• 16 Steyerberg EW, Borsboom GJ, van Houwelingen HC et al. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004; 23: 2567-2586
  CrossrefPubMedSearch in Google Scholar
• 17 Cotton PB, Garrow DA, Gallagher J et al. Risk factors for complications after ERCP: a multivariate analysis of 11,497 procedures over 12 years. Gastrointest Endosc 2009; 70: 80-88
  CrossrefPubMedSearch in Google Scholar
• 18 Hosmer DW, Leneshow S, May S. Applied survival analysis: regression modeling of time-to-event data. 2nd. . edn. New Jersey: Wiley-Interscience; 2008
  CrossrefSearch in Google Scholar
• 19 Royston P, Altman DG. External validation of a Cox prognostic model: principles and methods. BMC Med Res Methodol 2013; 13: 33
  CrossrefPubMedSearch in Google Scholar
• 20 Masci E, Toti G, Mariani A et al. Complications of diagnostic and therapeutic ERCP: a prospective multicenter study. Am J Gastroenterol 2001; 96: 417-423
  CrossrefPubMedSearch in Google Scholar
• 21 Testoni PA, Mariani A, Giussani A et al. Risk factors for post-ERCP pancreatitis in high- and low-volume centers and among expert and non-expert operators: a prospective multicenter study. Am J Gastroenterol 2010; 105: 1753-1761
  CrossrefPubMedSearch in Google Scholar
• 22 Christensen M, Matzen P, Schulze S et al. Complications of ERCP: a prospective study. Gastrointest Endosc 2004; 60: 721-731
  CrossrefPubMedSearch in Google Scholar
• 23 Khashab MA, Tariq A, Tariq U et al. Delayed and unsuccessful endoscopic retrograde cholangiopancreatography are associated with worse outcomes in patients with acute cholangitis. Clin Gastroenterol Hepatol 2012; 10: 1157-1161
  CrossrefPubMedSearch in Google Scholar
• 24 James PD, Kaplan GG, Myers RP et al. Decreasing mortality from acute biliary diseases that require endoscopic retrograde cholangiopancreatography: a nationwide cohort study. Clin Gastroenterol Hepatol 2014; 12: 1151-1159
  CrossrefPubMedSearch in Google Scholar
• 25 Campbell MJ, Jacques RM, Fotheringham J et al. Developing a summary hospital mortality index: retrospective analysis in English hospitals over five years. BMJ 2012; 344: e1001
  CrossrefPubMedSearch in Google Scholar
• 26 Varadarajulu S, Kilgore ML, Wilcox CM et al. Relationship among hospital ERCP volume, length of stay, and technical outcomes. Gastrointest Endosc 2006; 64: 338-347
  CrossrefPubMedSearch in Google Scholar
• 27 Adler DG, Lieb 2nd JG, Cohen J et al. Quality indicators for ERCP. Gastrointest Endosc 2015; 81: 54-66
  CrossrefPubMedSearch in Google Scholar
• 28 Toll DB, Janssen KJ, Vergouwe Y et al. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 2008; 61: 1085-1094
  CrossrefPubMedSearch in Google Scholar

Corresponding author

• References

• 1 Williams EJ, Taylor S, Fairclough P et al. Are we meeting the standards set for endoscopy? Results of a large-scale prospective survey of endoscopic retrograde cholangio-pancreatograph practice. Gut 2007; 56: 821-829
  CrossrefPubMedSearch in Google Scholar
• 2 Kapral C, Duller C, Wewalka F et al. Case volume and outcome of endoscopic retrograde cholangiopancreatography: results of a nationwide Austrian benchmarking project. Endoscopy 2008; 40: 625-630
  Thieme ConnectPubMedSearch in Google Scholar
• 3 Enochsson L, Swahn F, Arnelo U et al. Nationwide, population-based data from 11,074 ERCP procedures from the Swedish Registry for Gallstone Surgery and ERCP. Gastrointest Endosc 2010; 72: 1175-1184.e3
  CrossrefPubMedSearch in Google Scholar
• 4 Glomsaker T, Hoff G, Kvaloy JT et al. Patterns and predictive factors of complications after endoscopic retrograde cholangiopancreatography. Br J Surg 2013; 100: 373-380
  CrossrefPubMedSearch in Google Scholar
• 5 Coelho-Prabhu N, Shah ND, Van Houten H et al. Endoscopic retrograde cholangiopancreatography: utilisation and outcomes in a 10-year population-based cohort. BMJ Open 2013; 3: e002689
  CrossrefPubMedSearch in Google Scholar
• 6 Debray TP, Vergouwe Y, Koffijberg H et al. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol 2015; 68: 279-289
  CrossrefPubMedSearch in Google Scholar
• 7 Cote GA, Imler TD, Xu H et al. Lower provider volume is associated with higher failure rates for endoscopic retrograde cholangiopancreatography. Med Care 2013; 51: 1040-1047
  CrossrefPubMedSearch in Google Scholar
• 8 Bodger K, Bowering K, Sarkar S et al. All-cause mortality after first ERCP in England: clinically guided analysis of hospital episode statistics with linkage to registry of death. Gastrointest Endosc 2011; 74: 825-833
  CrossrefPubMedSearch in Google Scholar
• 9 Jacobsson A. The National Patient Register. Stockholm: The National Board of Health and Welfare (Socialstyrelsen); 2014 Available from: http://www.socialstyrelsen.se/register/halsodataregister/patientregistret/inenglish
  PubMedSearch in Google Scholar
• 10 Nilsson AC, Spetz CL, Carsjo K et al. [Reliability of the hospital registry. The diagnostic data are better than their reputation]. Lakartidningen 1994; 91: 598, 603-595
  PubMedSearch in Google Scholar
• 11 Nyqvist K, Passmark H, Muzollo T. [Attachment to Comparisons regarding Quality and Effectiveness in Health Care 2009]. Stockholm: The National Board of Health and Welfare (Socialstyrelsen); 2009 Available from: http://webbutik.skl.se/sv/artiklar/halso-och-sjukvard/bilagor-till-oppna-jamforelser-av-halso-och-sjukvardens-kvalitet-och-effektivitet-2009-indikatorbesk.html
  PubMedSearch in Google Scholar
• 12 Quan H, Sundararajan V, Halfon P et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005; 43: 1130-1139
  CrossrefPubMedSearch in Google Scholar
• 13 Mattsson B, Rutqvist LE, Wallgren A. Undernotification of diagnosed cancer cases to the Stockholm Cancer Registry. Int J Epidemiol 1985; 14: 64-69
  CrossrefPubMedSearch in Google Scholar
• 14 Kalaitzakis E, Toth E. Hospital volume status is related to technical failure and all-cause mortality following ERCP for benign disease. Dig Dis Sci 2015; 60: 1793-1800
  CrossrefPubMedSearch in Google Scholar
• 15 Steyerberg EW, Vickers AJ, Cook NR et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010; 21: 128-138
  CrossrefPubMedSearch in Google Scholar
• 16 Steyerberg EW, Borsboom GJ, van Houwelingen HC et al. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004; 23: 2567-2586
  CrossrefPubMedSearch in Google Scholar
• 17 Cotton PB, Garrow DA, Gallagher J et al. Risk factors for complications after ERCP: a multivariate analysis of 11,497 procedures over 12 years. Gastrointest Endosc 2009; 70: 80-88
  CrossrefPubMedSearch in Google Scholar
• 18 Hosmer DW, Leneshow S, May S. Applied survival analysis: regression modeling of time-to-event data. 2nd. . edn. New Jersey: Wiley-Interscience; 2008
  CrossrefSearch in Google Scholar
• 19 Royston P, Altman DG. External validation of a Cox prognostic model: principles and methods. BMC Med Res Methodol 2013; 13: 33
  CrossrefPubMedSearch in Google Scholar
• 20 Masci E, Toti G, Mariani A et al. Complications of diagnostic and therapeutic ERCP: a prospective multicenter study. Am J Gastroenterol 2001; 96: 417-423
  CrossrefPubMedSearch in Google Scholar
• 21 Testoni PA, Mariani A, Giussani A et al. Risk factors for post-ERCP pancreatitis in high- and low-volume centers and among expert and non-expert operators: a prospective multicenter study. Am J Gastroenterol 2010; 105: 1753-1761
  CrossrefPubMedSearch in Google Scholar
• 22 Christensen M, Matzen P, Schulze S et al. Complications of ERCP: a prospective study. Gastrointest Endosc 2004; 60: 721-731
  CrossrefPubMedSearch in Google Scholar
• 23 Khashab MA, Tariq A, Tariq U et al. Delayed and unsuccessful endoscopic retrograde cholangiopancreatography are associated with worse outcomes in patients with acute cholangitis. Clin Gastroenterol Hepatol 2012; 10: 1157-1161
  CrossrefPubMedSearch in Google Scholar
• 24 James PD, Kaplan GG, Myers RP et al. Decreasing mortality from acute biliary diseases that require endoscopic retrograde cholangiopancreatography: a nationwide cohort study. Clin Gastroenterol Hepatol 2014; 12: 1151-1159
  CrossrefPubMedSearch in Google Scholar
• 25 Campbell MJ, Jacques RM, Fotheringham J et al. Developing a summary hospital mortality index: retrospective analysis in English hospitals over five years. BMJ 2012; 344: e1001
  CrossrefPubMedSearch in Google Scholar
• 26 Varadarajulu S, Kilgore ML, Wilcox CM et al. Relationship among hospital ERCP volume, length of stay, and technical outcomes. Gastrointest Endosc 2006; 64: 338-347
  CrossrefPubMedSearch in Google Scholar
• 27 Adler DG, Lieb 2nd JG, Cohen J et al. Quality indicators for ERCP. Gastrointest Endosc 2015; 81: 54-66
  CrossrefPubMedSearch in Google Scholar
• 28 Toll DB, Janssen KJ, Vergouwe Y et al. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 2008; 61: 1085-1094
  CrossrefPubMedSearch in Google Scholar