For fitted models, this command calculates (1) how much bias there must be in an estimate to invalidate/sustain an inference; (2) the impact of an omitted variable necessary to invalidate/sustain an inference for a regression coefficient. Currently works for: models created with lm() (linear models).

konfound(model_object, tested_variable, alpha = 0.05, tails = 2,
  to_return = "print", test_all = FALSE)

Arguments

model_object

output from a model (currently works for: lm)

tested_variable

Variable associated with the unstandardized beta coefficient to be tested

alpha

probability of rejecting the null hypothesis (defaults to 0.05)

tails

integer whether hypothesis testing is one-tailed (1) or two-tailed (2; defaults to 2)

to_return

whether to return a data.frame (by specifying this argument to equal "raw_output" for use in other analyses) or a plot ("plot"); default is to print ("print") the output to the console; can specify a vector of output to return

test_all

whether to carry out the sensitivity test for all of the coefficients (defaults to FALSE)

Value

prints the bias and the number of cases that would have to be replaced with cases for which there is no effect to invalidate the inference

Examples

# using lm() for linear models m1 <- lm(mpg ~ wt + hp, data = mtcars) konfound(m1, wt)
#> Percent Bias Necessary to Invalidate the Inference: #> To invalidate an inference, 66.664% of the estimate would have to be due to bias. This is based on a threshold of -1.293 for statistical significance (alpha = 0.05). #> To invalidate an inference, 21 observations would have to be replaced with cases for which the effect is 0. #> #> Impact Threshold for a Confounding Variable: #> An omitted variable would have to be correlated at 0.787 with the outcome and at 0.787 with the predictor of interest (conditioning on observed covariates) to invalidate an inference based on a threshold of -0.36 for statistical significance (alpha = 0.05). #> Correspondingly the impact of an omitted variable (as defined in Frank 2000) must be 0.787 X 0.787 = 0.619 to invalidate an inference.
#> For more detailed output, consider setting `to_return` to table
#> To consider other predictors of interest, consider setting `test_all` to TRUE.
konfound(m1, wt, test_all = TRUE)
#> Note that this output is calculated based on the correlation-based approach used in mkonfound()
#> # A tibble: 2 x 8 #> var_name t df action inference pct_bias_to_change_in… itcv r_con #> <chr> <dbl> <dbl> <chr> <chr> <dbl> <dbl> <dbl> #> 1 wt -6.13 30 to_invali… reject_null 53.1 0.609 0.78 #> 2 hp -3.56 30 to_invali… reject_null 35.8 0.3 0.548
konfound(m1, wt, to_return = "table")
#> Dependent variable is mpg
#> Warning: Unknown or uninitialised column: 'itcv'.
#> Warning: Unknown or uninitialised column: 'impact'.
#> # A tibble: 3 x 7 #> term estimate std.error statistic p.value itcv impact #> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> #> 1 (Intercept) 37.2 1.60 23.3 0 NA NA #> 2 wt -3.88 0.633 -6.13 0 0.243 NA #> 3 hp -0.032 0.009 -3.52 0.001 NA 0.511
# using glm() for non-linear models if (requireNamespace("forcats")) { d <- forcats::gss_cat d$married <- ifelse(d$marital == "Married", 1, 0) m2 <- glm(married ~ age, data = d, family = binomial(link = "logit")) konfound(m2, age) }
#> Loading required namespace: forcats
#> Note that for a non-linear model, impact threshold should not be interpreted.
#> Note that this is only an approximation. For exact results in terms of the number of cases that must be switched from treatment success to treatment failure to invalidate the inference see: https://msu.edu/~kenfrank/non%20linear%20replacement%20treatment.xlsm
#> Error in eval(model[["call"]][["data"]], env): object 'd' not found
# using lme4 for mixed effects (or multi-level) models if (requireNamespace("lme4")) { library(lme4) m3 <- fm1 <- lme4::lmer(Reaction ~ Days + (1 | Subject), sleepstudy) konfound(m3, Days) }
#> Loading required package: Matrix
#> Warning: binding factor and character vector, coercing into character vector
#> Warning: binding character and factor vector, coercing into character vector
#> Percent Bias Necessary to Invalidate the Inference: #> To invalidate an inference, 84.83% of the estimate would have to be due to bias. This is based on a threshold of 1.588 for statistical significance (alpha = 0.05). #> To invalidate an inference, 137 observations would have to be replaced with cases for which the effect is 0. #> #> Impact Threshold for a Confounding Variable: #> An omitted variable would have to be correlated at 0.817 with the outcome and at 0.817 with the predictor of interest (conditioning on observed covariates) to invalidate an inference based on a threshold of 0.155 for statistical significance (alpha = 0.05). #> Correspondingly the impact of an omitted variable (as defined in Frank 2000) must be 0.817 X 0.817 = 0.667 to invalidate an inference.
#> Note that the Kenward-Roger approximation is used to estimate degrees of freedom for the predictor(s) of interest. We are presently working to add other methods for calculating the degrees of freedom for the predictor(s) of interest. If you wish to use other methods now, consider those detailed here: https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html#why-doesnt-lme4-display-denominator-degrees-of-freedomp-values-what-other-options-do-i-have. You can then enter degrees of freedom obtained from another method along with the coefficient, number of observations, and number of covariates to the pkonfound() function to quantify the robustness of the inference.
#> NULL