Advanced Statistics: Multiple Logistic Regression,Cox Proportional Hazards,and Propensity Scores

Abstract

Background:

Randomized controlled trials (RCTs) are generally regarded as the gold standard for demonstrating causality because they effectively mitigate bias from both known and unknown confounders. However, conducting an RCT is not always feasible because of logistical and ethical considerations. This is especially true when evaluating surgical interventions, and non-randomized study designs must be utilized instead.

Methods:

Statistical methods that adjust for baseline differences in non-randomized studies were reviewed.

Results:

The three methods used most commonly to adjust for confounding factors are multiple logistic regression, Cox proportional hazard, and propensity scoring. Multiple logistic regression (MLR) is implemented to analyze the influence of categorical and/or continuous variables on a single dichotomous outcome. The model controls for multiple covariates while also quantifying the magnitude of each covariate's influence on the outcome. Selecting which variables to include in a model should be the most important consideration, and authors must report how and why variables were chosen. Cox proportional hazards modeling is conceptually similar to logistic regression and is used when analyzing survival data. When applied to survival curves, Cox proportional hazards can adjust for baseline group differences and provide a hazard ratio to quantify the effect that any single factor contributes to the survival curve. Propensity scores (PS) range from zero to one and are defined as the probability of receiving an intervention based on observed baseline characteristics. Propensity score matching (PSM) is especially useful when the outcome of interest is a rare event. Treated and untreated subjects with similar propensity scores are paired, forming balanced samples for further analysis.

Conclusions:

The method by which to address confounding should be selected according to the data format and sample size. Reporting of methods should provide justification for selected covariates, confirmation that data did not violate model assumptions, and measures of model performance.

The integration of clinical research and medical decision-making occurs through the use of evidence-based practice whereby physicians use the best available evidence to guide clinical decision making [1]. Ideally, the evidence should demonstrate a definitive cause-and-effect relation, or causality, between an exposure and outcome, i.e., a given intervention will lead to patient improvement. Randomized controlled trials (RCTs) are generally regarded as the gold standard for proving causality because successful randomization mitigates bias and confounding for both known and unknown factors [2]. However, conducting an RCT is not always feasible because of logistical, financial, or ethical considerations. This is especially true when evaluating surgical interventions because there are inherent obstacles to recruiting subjects, blinding patients and physicians, and standardizing treatments [3]. Thus, a different approach is usually selected out of necessity.

When carefully designed and conducted, non-randomized studies may serve as a reasonable alternative to RCTs. Non-randomized studies are useful for identifying correlations that, depending on the robustness of the study, may allow for inferences on causation to be made. For example, although there are no RCTs comparing mortality after early (<12 hours) and late (>12 hours) surgical debridement for necrotizing soft tissue infections, early debridement is widely accepted as the preferred treatment based on several large observational studies [4]. However, unlike RCTs, non-randomized studies always carry an inherent amount of confounding, because subjects are intentionally allocated to treatment groups according to baseline factors. Multiple statistical methods have been developed to adjust for these differences and improve balance between groups. However, it is important to note that unlike RCTs, only known and measured confounders can be adjusted. Any bias caused by unknown or unmeasured differences between groups will persist.

In the following sections, we highlight three of the methods used most commonly to adjust for confounding factors: multiple logistic regression (MLR), Cox proportional hazard, and propensity scoring (PS). For each method, we provide a non-technical overview, some common pitfalls, and critical reporting guidelines. Examples of these methods are provided using articles published previously in Surgical Infections.

Multiple Logistic Regression

A simple correlation between two variables is the amount of change expected in variable x when variable y is increased or decreased. The strength of the correlation is measured by Pearson and Spearman correlation coefficients, which range from −1 to 1. Refer to the detailed description in Ho and Pieracci [5]. Using the example of body mass index and surgical site infection, examples of positive, negative, and no correlations are displayed in Figure 1. Multiple logistic regression can be conceptualized as a multidimensional version of the correlation and is a statistical tool used for both explanatory and predictive modeling [6]. Using this technique, one tries to determine the strength of association between a predictive (independent) variable and an outcome (dependent) variable while holding all other predictive variables constant. In non-randomized studies, Multiple logistic regression is frequently implemented to analyze the influence of categorical and/or continuous variables (independent variables) on a single dichotomous outcome (dependent variable) [7]. Its utility resides in its ability to control for multiple covariates while quantifying the magnitude of each covariate's influence on the outcome, which is reported as an odds ratio [8]. By this same process, predictive models may be generated to estimate an individual's odds of an outcome based on the presence or absence of known prognostic factors [9]. We use the study by McGillen et al. [10] as an example to illustrate these concepts. In this study, the authors explored whether the duration of post-operative antibiotic agents was associated with infection after appendectomy for complicated appendicitis. When the authors compared patients who developed an infection with those who did not, they found that patients with an infection were more likely to have free fluid on their initial computed tomography (CT) scan, and were also more likely to have had a drain placed intra-operatively. Therefore, these two potentially confounding factors were used in the MLR, as well as duration of antibiotic therapy (independent variables), to determine their impact on the odds of post-operative infection (dependent variable).

FIG. 1.

Depictions of strong positive correlation (a), strong negative correlation (b), weak positive correlation (c), and no correlation (d) between body mass index (BMI) and surgical site infection (SSI) rates.

Selecting the appropriate covariates to include in the model is a critical consideration when preparing for the analysis. The conventional process for selecting variables begins by conducting multiple comparisons between outcome groups, such as control group and intervention group, to identify significant variables, with a more liberal cutoff for significance (commonly p < 0.2) [11]. Results of these univariate comparisons are shown in Table 1. Variables with statistically significant differences between the outcome groups should then be reviewed to ensure their relation to the outcome is both plausible and clinically meaningful [11]. In McGillen et al. [10], both free fluid on CT and intra-operative decision to leave a drain are markers of severity of appendicitis, so a relation to post-operative infections is indeed plausible. All included variables should be independent without relation to other variables. For example, white blood cell count and absolute neutrophil count would have highly related values, and therefore should not both be used; this colinearity weakens the predictive model. An additional assumption for MLR is that the relation between the predictive variables and the outcome is constant (linear correlation) across all values for the variable [12]. This assumption may not always be true, for example at the physiologic extremes of variables such as age, height, weight, etc.

Table 1.

Baseline Demographic and Clinical Characteristics According to Post-Operative Surgical Site Infection Development^a

Variable	No infection (n = 163)	Infection (n = 25)	p
Demographics
Age	41.5 ± 16.2	44.2 ± 17.9	NS (0.445)
Male (%)	55.8 (91)	64.0 (16)	NS (0.442)
Female (%)	44.2 (72)	36.0 (9)
Race (%)			NS (0.457)
White, non-Hispanic	23.3 (38)	36.0 (9)
Black, non-Hispanic	22.1 (36)	28.0 (7)
Hispanic/Latino	35.6 (58)	28.0 (7)
Asian	7.4 (12)	4.0 (1)
Other/decline	11.7 (19)	4.0 (1)
Language spoken (%)			NS (0.121)
English	46.6 (76)	68.0 (17)
Spanish	41.7 (68)	28.0 (7)
Other	11.7 (19)	4.0 (1)
Insurance type (%)			NS (0.487)
Private	33.74 (55)	44.0 (11)
Public	49.1 (80)	36.0 (9)
Charity	14.7 (24)	20.0 (5)
Self-pay	2.5 (4)	0
Comorbid conditions (%)
Diabetes	9.8 (16)	24.0 (6)	0.04
HIV	1.2 (2)	4.0 (1)	NS (0.303)
CAD	4.3 (7)	4.0 (1)	NS (0.946)
Current smoker	18.4 (30)	8.0 (2)	NS (0.197)
Clinical variables at presentation
WBC count	13.9 ± 4.8	15.6 ± 5.3	NS (0.099)
ASA class (%)			NS (0.744)
1	39.9 (65)	32.0 (8)
2	49.1 (80)	60.0 (15)
3	9.8 (16)	8.0 (2)
4	1.2 (2)	0
Imaging finding (%)
Free fluid	25.2 (41)	56.0 (14)	0.002
Abscess	3.1 (5)	12.0 (3)	0.039
Perforation	9.8 (16)	28.0 (7)	0.012
Clinical variables post-operatively (%)
Laparoscopic surgery	86.5 (141)	88.0 (22)	NS (0.837)
Perforation (as described by surgeon)	75.5 (123)	80.0 (20)	NS (0.620)
Intra-operative drain placement	9.8 (16)	36.0 (9)	0.001
LOS (h)	63.4 ± 63.3	168.0 ± 126.5	<0.001
Time on antibiotics (pre-complication, if there was one) (d)^b	40.7 ± 49	92.0 ± 75.9	<0.001
Complications (%)
Ileus	8.0 (13)	44.0 (11)	<0.001
Re-admission	6.8 (11)	60.0 (15)	<0.001
Extra-abdominal infection	1.2 (2)	0	NS (0.578)
Clostridioides difficile colitis	0.6 (1)	4.0 (1)	NS (0.124)
Post-operative intervention (%)	6.8 (11)	80.0 (20)	<0.001
IR drain	0	56.0 (14)	<0.001
Re-operation	0.6 (1)^c	4.0 (1)	NS (0.124)

Reprinted with permission [10].

NS = not significant; HIV = human immunodeficiency virus; CAD = coronary artery disease; WBC = white blood cells; ASA = American Society of Anesthesiologists; LOS = length of stay; IR = interventional radiology.

Mean ± standard deviation.

t-test is only between intravenous antibiotic agents and intravenous plus oral antibiotic agents.

Patient underwent delayed abdominal closure and did not develop surgical site infection.

Including too many variables in the model can be problematic, especially for rare events. One issue, termed overfitting the model, occurs when the number of events is small compared with the number of variables included in the regression [13]. In this situation, the model is fit to “noise” in addition to underlying variables, which limits the utility of the model with a different patient sample [13]. Therefore, it has been suggested that the number of covariates be limited by the sample size and the event rate (i.e., the incidence of the dependent variable) [14]. In the referenced article [10], there were 25 subjects with a post-operative infection, and three variables included in the model, which comes out to roughly eight events per variable (EPV). The conventionally accepted rule of thumb states that there should be a minimum 10 EPV, and no fewer than 20 EPV for risk prediction [15,16]. However, this remains a controversial topic, because simulations have shown that results using five to nine EPV are comparable with those with 10–16 EPV [16].

The validity of a given model is assessed through measures of discrimination and calibration. Discrimination is the ability of a model to correctly distinguish subjects at high risk of experiencing the event from subjects at low risk [9]. In the study by McGillen et al. [10], discrimination is the ability of the three included variables to predict who would develop an infection and who would not. For logistic regression, which has a binary categorical outcome (infection or no infection), discrimination is best determined by a concordance measure known as the C-index (or C-statistic) [9]. This measure is equivalent to the area under the receiver operating characteristics (AUROC) curve and is usually reported as such [9]. Calibration, which is also referred to as goodness-of-fit, is the model's ability to estimate the absolute risk correctly [17]. In the referenced article [10], the authors report that 24 hours of post-operative antibiotic agents was associated with an increase in the odds of infection by a factor of 10 compared with no post-operative antibiotic agents (Table 2). In a well-calibrated model, for every infection in a subject who received no antibiotic agents, we would expect 10 infections in subjects with 24 hours of antibiotic agents. Calibration is most often reported using the Hosmer-Lemeshow test, which compares the difference between predicted (what we would expect based on the odds ratios) and observed (what the sample actually shows) event rates and determines a probability that the difference is attributable to chance alone [17]. Therefore, a Hosmer-Lemeshow p value that is statistically significant (p < 0.05) suggests that the difference between the observed and expected event rates cannot be explained by chance alone and is instead the result of a poorly calibrated model [17]. It is worth noting that the Hosmer-Lemeshow test may be misleading with large samples, because a statistically significant p value may correspond to a clinically insignificant difference between expected and observed odds [17]. These values should always be reported in predictive models, however, they are less important for explanatory models [6].

Table 2.

Odds of Surgical Site Infection Development Based on Post-Operative Antibiotic Duration^a

Duration of antibiotic therapy (h)	n	Odds ratio	95% CI	p
None	53	Ref	—	—
24	38	10.39	1.16-92.8	0.036
48	58	5.36	0.61–47.13	0.13
>72	39	13.78	1.62–117.11	0.016

Reprinted with permission [10].

CI = confidence interval.

Adjusted for post-operative drain placement and free fluid on imaging.

Multiple logistic regression is among the methods most commonly utilized to control for confounding in non-randomized studies. Selecting which variables to include in a model should be the most important consideration. As such, authors must report how and why the variables in their model were selected. Performance of the model should be reported in terms of discrimination and calibration, and we suggest using the AUROC and the Hosmer-Lemeshow test for these values, respectively. Whereas model performance should always be reported for predictive models, it is unnecessary for explanatory models.

Cox Proportional Hazards

Cox proportional hazards modeling is conceptually similar to MLR and is used when analyzing survival data over time [9]. Proper application of Cox proportional hazards is contingent upon understanding the unique aspects of survival analyses, including censorship and time to event outcomes. Although multiple methods exist, Kaplan-Meier (KM) curves are the most commonly utilized and will serve as the focus of this section. The example used in this section comes from Guidry et al. [18], whose study explored how aggressive and conservative antimicrobial protocols in the intensive care unit (ICU) might impact long-term survival.

The use of KM curves is preferred because they allow subjects with incomplete data, seen commonly in non-randomized studies, to be included in the analysis as censored subjects [19]. Censored subjects are those who did not experience the event (usually death, but may also be infection, cancer recurrence, etc.), either because they are lost to follow-up or because the study ends [9]. This allows those subjects with incomplete records to contribute partially to the analysis for the time period in which they were followed, and to be excluded from the analysis after that timepoint. For each subject, three datapoints are required to perform the analysis: (1) serial time, which is the total time of observation of the subject; (2) status at specific timepoints, which is either the outcome (death) or the censoring (last encounter) of the subject; and (3) subject's treatment group (intervention or control) [20]. In the study by Guidry et al. [18], serial time would be the interval from ICU admission until the date of death, last known survival date, or the date of termination of the study, depending upon the patient. Each patient was followed for four years. It therefore follows that all patients who survived to four years become censored at the end of the study period, because they are no longer being followed. The two treatment groups in this study were based on aggressive and conservative antimicrobial protocols. Curves for each group are generated to display survival, which is the proportion of all subjects without the event over set time intervals [21]. The hazard rate, which instead represents the incidence of event occurrence, can also be calculated from the curve [22]. To determine if a significant difference in survival (or hazard) exists between groups, the log-rank test is performed [20]. In the cited study, there was no significant difference in survival between patients in the aggressive and conservative protocol groups at four years.

Cox proportional hazards may be applied to the survival curves to adjust for baseline group differences and provide a hazard ratio to quantify the effect that any single factor contributes to the survival curve [22,23]. Guidry et al. [18] performed Cox proportional hazards to control for differences in demographics and comorbidities between patients in each of the protcols. Two key assumptions must be met for the model to hold. The first is called the proportionality assumption, which stipulates that the hazards between the study groups are proportional (i.e., curves do not cross). The second assumption is that all variable effects are roughly constant throughout the study period [23]. The authors of this study clearly state that variables were checked to make sure these assumptions held true, and variables that did not were excluded. As with MLR, the rule of 10 EPV has been recommended for Cox proportional hazards [15,16] After controlling for demographics and comorbidities on Cox proportional hazards, the authors found a significant association between the aggressive antimicrobial protocol and death at four years (Table 3).

Table 3.

Cox Proportional Hazards Model Results

	One year		Four years
Variable	Hazard ratio (95% CI)	p	Hazard ratio (95% CI)	p
Aggressive group	2.26 (1.24–4.10)	0.007	1.70 (1.03–2.79)	0.05
Age: 70–79 y	4.13 (2.22–7.68)	<0.0001	4.47 (2.63–7.62)	<0.0001
APACHE II ≥25	2.78 (1.54–4.99)	0.0007	2.24 (1.34–3.76)	0.002
Chronic kidney disease^a	Not included^b		2.57 (1.05–6.34)	0.04
Hemodialysis^c	2.79 (1.38–5.64)	0.004	2.66 (1.34–5.26)	0.005
Trauma	0.28 (0.14–0.57)	0.0004	0.36 (0.20–0.64)	0.0005

Reprinted with permission.

CI = confidence interval; APACHE = Age and Acute Physiology and Chronic Health Evaluation.

Defined as baseline creatine ≥2.0 mg/dL.

No significant association was identified at 1 year and therefore was not included in the model.

Refers to any renal replacement required at the time of diagnosis.

One important consideration in KM curves and Cox proportional hazards is the presence of any competing risks [24]. Competing risks are events that prevent the subject from experiencing the outcome of interest. For example, the authors in the referenced article [18] might have chosen re-admission to the ICU with an infection as their outcome of interest. Any patient who died during the study period is no longer at risk of being re-admitted to the ICU with an infection, therefore, death would be considered a competing risk. There are several strategies available for handling data with competing risks, however, this is outside of the scope of this article.

Kaplan-Meier curves and Cox proportional hazards are best applied when the outcome of interest is the time to an event. Kaplan-Meier curves are generated to compare survival between two treatment groups; the log-rank test is subsequently performed to determine if differences in survival are statistically significant, and this result should be reported. Cox proportional hazards may be applied to adjust for baseline differences between groups. Similar to MLR, variables must be selected carefully and authors must provide justification for all variables included.

Propensity Score

Propensity score (PS) methods are a category of analyses that includes four distinct approaches to address confounding in non-randomized studies. The four analyses are based on calculating a PS for each subject, which ranges from zero to one and is defined as the probability of receiving an intervention based on observed baseline characteristics [14,25]. One way to understand the PS score is to consider “assuming that my patient has these observed baseline characteristics, what is the likelihood that he/she will receive this intervention?” A PS of 0 means that the patient would never receive the intervention, a PS of 0.5 means that the patient would receive the intervention 50% of the time, and a PS of 1 means that the patient would always receive the intervention. The example in this section comes from Gabriel et al. [26], who performed a PS matched analysis between patients who underwent rapid source control laparotomy (RSCL) and those who had primary fascial closure (PFC) to compare hospital length of stay (LOS) and mortality. All baseline variables that predict outcome independently should be included, whereas those that only affect the probability of receiving the intervention (that are not associated with the outcome) should be excluded [14,25]. The authors in the cited study selected indication for surgery (based on International Classification of Disease [ICD]-10 diagnostic code), patient comorbidities, and presence of pre-operative sepsis as their matching variables, given their influence on LOS and mortality. It therefore holds that for any given PS, baseline characteristics should have similar distributions in both treatment groups, which is demonstrated in Table 4 [27]. Because PS consolidates multiple variables into one numeric value, these methods are ideal for their ability to handle even few events per variable [28].

Table 4.

Indications for Intervention, Baseline Patient Characteristics, and Operative Risk in RSCL and PFC Cohorts

Variable		RSCL cohort	% of total	PFT cohort	% of total
ICD-10 Code and Diagnosis
A41.90	Sepsis	77	36.67	77	36.67
K25.10	Acute gastric ulcer perforation	3	1.43	3	1.43
K43.00	Incisional hernia with obstruction	5	2.38	5	2.38
K43.60	Ventral hernia with obstruction	3	1.43	3	1.43
K55.10	Chronic vascular disorder of intestine	2	0.95	2	0.95
K55.90	Vascular disorder of intestine, not specified	21	10	21	10
K56.20	Intestinal volvulus	6	2.86	6	2.86
K56.50	Intestinal adhesions with obstruction	8	3.81	8	3.81
K56.60	Intestinal obstruction, not specified	3	1.43	3	1.43
K56.69	Ileus with obstruction	9	4.29	9	4.29
K57.20	Diverticulitis of large intestine with perforation or abscess	14	6.67	14	6.67
K57.80	Diverticulitis of intestine with perforation or abscess	3	1.43	3	1.43
K63.10	Non-traumatic perforation of intestine	45	21.43	45	21.43
K65.10	Peritoneal abscess	6	2.86	6	2.86
K65.90	Peritonitis, not specified	5	2.38	5	2.38
Number of risk factors
0–3		120	57.14	120	57.14
4+		90	42.9	90	42.9
Presence of sepsis		186	88.57	187	88.57

Reprinted with permission [26].

ICD = International Classification of Diseases; PFC = primary fascial closure; RSCL = rapid source-control laparotomy.

The four methods for PS analysis are: (1) propensity score matching (PSM), (2) PS stratification, (3) PS adjustment, and (4) PS analysis by inverse probability of treatment weighting [14]. Of these methods, PSM is used most frequently because it has been shown to balance the greatest proportion of baseline differences [25]. Therefore, we focus here exclusively on PSM. For a more detailed discussion of all four PS analysis techniques, the authors refer the reader to an article by Zakrison et al. [25]. With PSM, subjects from the treatment group are matched to subjects from the control group according to PS [14]. Several matching algorithms exist to identify a match between subjects. The most common matching algorithm is greedy nearest neighbor matching, whereby subjects from the intervention group are matched to the nearest subject from the control group [14,29]. Greedy matching will always produce a match, even if the matched pair are dissimilar in their PS. A variation on this algorithm is greedy nearest neighbor matching within specified caliber widths [29]. This is similar to greedy nearest neighbor with the exception that a match will only occur if the difference between the PS does not exceed a fixed value (most commonly 0.2 of the standard deviation of the logit of the PS) [30]. An alternative approach is optimal matching, which minimizes the average difference in PS between all matched pairs [27]. Finally, matching may occur with or without replacement. In matching without replacement, once a control is matched to a treated subject, that control is no longer available to match to a different treated subject [29]. This is in contrast to matching with replacement, which allows a control to match to more than one treated subject [29]. In comparing these methods, greedy nearest neighbor caliper matching without replacement appears to result in the least amount of bias, and is therefore the recommended algorithm [29]. In the cited example from Gabriel et al. [26], PS matching was integrated into their NSQIP database query. Although not explicitly stated by the authors, this strategy most resembles the greedy nearest neighbor matching without replacement algorithm.

Matching is performed most commonly in a 1:1 ratio, however, this may be expanded so that each treated subject is matched to multiple controls [25,27]. Matching multiple controls to each subject has the advantage of increasing the precision of the estimated treatment effect (by increasing the sample size), however, this may also result in dissimilar matches that will increase bias [31]. It is therefore recommend to limit matches to two controls per treated subject [31]. To determine the adequacy of matching and ensure balance between the two groups, the covariate balance must be assessed and reported. Table 4 confirms that Gabriel et al. [26] achieved perfect balance in the matched variables between the RSCL and PFC groups. Balance is critical to appropriate matching and unbiased effect estimation [14]. It is recommended that the standardized difference of variables be used for this analysis, with an accepted cutoff of <0.1 [25]. Once matching is completed, an analysis that reflects the paired relation of the samples (rather than independent samples), i.e., paired samples t-test, McNemer test, and Cox proportional hazards, may be performed [25]. In the above example, the authors performed an MLR to identify risk factors for death. Subjects who underwent RSCL had approximately two-fold increased odds of death compared with those who underwent PFC.

Analyses based on PS can be highly variable given the numerous methods and algorithms discussed above. As such, it is essential that authors be comprehensive in reporting their methods and results. All variables used to generate the PS, as well as a rationale for inclusion, should be provided. The PS method, matching algorithm (if applicable), and any additional tests used for analysis should be explicitly stated. Finally, authors should provide confirmation that covariate balance was assessed and achieved.

Conclusion

Randomized controlled trials are the preferred study design for mitigating bias from both known and unknown factors. When RCTs are not feasible, non-randomized studies may serve as a reasonable alternative if adjustment for baseline differences is performed. The method by which to address confounding should be selected according to the data format and sample size. Reporting of these methods should provide justification for the selected covariates, confirmation that data did not violate the model assumptions, and measures of model performance.

Footnotes

Funding Information

No funding was received.

Author Disclosure Statement

None of the authors report any financial disclosures/conflicts of interest

References

Portney

, Watkins

Foundations of Clinical Research Applications to Practice, 3rd ed. Philadelphia, PA: F.A. Davis Company, 2015.

Biondi-Zoccai

, Romagnoli

, Agostoni

, et al. Are propensity scores really superior to standard multivariable analysis?. Contemp Clin Trials, 2011; 32:731–740.

McCulloch

, Taylor

, Sasako

, et al. Randomised trials in surgery: problems and possible solutions. BMJ, 2002; 324:1448–1451.

Gelbard

, Ferrada

, Yeh

, et al. Optimal timing of initial debridement for necrotizing soft tissue infection. J Trauma Acute Care Surg, 2018; 85:208–214.

Bensken

, Ho

, Pieracci

. Basic introduction to statistics in medicine, part 2: Comparing data. Surg Infect, 2021; 22:597–603.

Shmueli

To explain or to predict?. Stat Sci, 2010; 25:289–310.

Elmore

JG.

Jekel's Epidemiology, Biostatistics, Preventive Medicine, and Public Health. Philadelphia, PA: Elsevier Inc., 2020.

Aggarwal

, Ranganathan

. Common pitfalls in statistical analysis: Linear regression analysis. Perspect Clin Res, 2017; 8:100–102.

Guller

, DeLong

. Interpreting statistics in medical literature: A vade mecum for surgeons. J Am Coll Surg, 2004; 198:441–458.

10.

McGillen

, Drake

, Vallejo

, et al. Retrospective analysis of post-operative antibiotics in complicated appendicitis. Surg Infect, 2019; 20:359–366.

11.

Ranganathan

, Pramesh

, Aggarwal

. Common pitfalls in statistical analysis: Logistic regression. Perspect Clin Res, 2017; 8:148–151.

12.

Tolles

, Meurer

. Logistic regression: Relating patient characteristics to outcomes. JAMA, 2016; 316:533–534.

13.

Pavlou

, Ambler

, Seaman

, et al. Review and evaluation of penalised regression methods for risk prediction in low-dimensional data with few events. Stat Med, 2016; 35:1159–1177.

14.

Grose

, Wilson

, Barkun

, et al. Use of propensity score methodology in contemporary high-impact surgical literature. J Am Coll, 2020; 230:101–111.

15.

Vittinghoff

, McCulloch

. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol, 2007; 165:710–718.

16.

Peduzzi

, Concato

, Feinstein

, Holford

. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol, 1995; 48:1503–1510.

17.

Alba

, Agoritsas

, Walsh

, et al. Discrimination and calibration of clinical prediction models: Users' guides to the medical literature. JAMA, 2017; 318:1377–1384.

18.

Guidry

, Hranjec

, Shah

, et al. Aggressive antimicrobial initiation for suspected intensive care unit-acquired infection is associated with decreased long-term survival after critical illness. Surg Infect, 2017; 18:664–669.

19.

Kaplan

, Meier

Nonparametric estimation from incomplete observations. J Am Stat Assoc, 1958; 53:457–486.

20.

Rich

, Neely

, Paniello

, et al. A practical guide to understanding Kaplan-Meier curves. Otolaryngol Head Neck Surg, 2010; 143:331–336.

21.

Telke

, Eberly

. Statistical hypothesis testing: Associating patient characteristics with an incident condition: Kaplan-Meier curves, hazard ratios, and cox proportional hazards regression. J Wound Ostomy Continence Nurs, 2011; 38:621–626.

22.

Clark

, Bradburn

, Love

, Altman

. Survival analysis part I: Basic concepts and first analyses. Br J Cancer, 2003; 89:232–238.

23.

Bradburn

, Clark

, Love

, Altman

. Survival analysis part II: Multivariate data analysis. An introduction to concepts and methods. Br J Cancer, 2003; 89:431–436.

24.

The Fine-Gray model under interval censored competing risks data. J Multivar Anal, 2016; 143:327–344.

25.

Zakrison

, Austin

, McCredie

. A systematic review of propensity score methods in the acute care surgery literature: Avoiding the pitfalls and proposing a set of reporting guidelines. Eur J Trauma Emerg Surg, 2018; 44:385–395.

26.

Gabriel

, Grigorian

, Phillips

, et al. A propensity score analysis of Clostridium difficile infection among adult trauma patients. Surg Infect, 2018; 19:661–666.

27.

Austin

PC.

An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res, 2011; 46:399–424.

28.

Cepeda

, Boston

, Farrar

, Strom

. Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. Am J Epidemiol, 2003; 158:280–287.

29.

Austin

PC.

A comparison of 12 algorithms for matching on the propensity score. Stat Med, 2014; 33:1057–1069.

30.

Austin

PC.

Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm Stat, 2011; 10:150–161.

31.

Austin

PC.

Statistical criteria for selecting the optimal number of untreated subjects matched to each treated subject when using many-to-one matching on the propensity score. Am J Epidemiol, 2010; 172:1092–1097.