Skip to contents

Summarise the vibration of effects across a multiverse

Source: R/voe.R
voe.Rd

Given per-specification estimates, voe() returns the distribution summaries used in vibration-of-effects analysis (Patel, Burford and Ioannidis, 2015): the median effect, the 1st and 99th percentiles, the relative ratio (the 99th divided by the 1st percentile of the exponentiated effect, for example the relative hazard or odds ratio), the Janus fraction (the share of estimates above zero, so a value near 0 or 1 means the sign is stable), and the proportion of specifications reaching a significance threshold.

Usage

voe(
  results,
  estimate = estimate,
  p_value = p.value,
  exponentiate = TRUE,
  alpha = 0.05
)

Arguments

results

A tibble of per-specification results, for example from fit_multiverse().

estimate

<data-masking> column of effect estimates on the model link scale, for example a log hazard ratio.

p_value

<data-masking> column of p-values.

exponentiate

Logical; exponentiate the effect before forming the relative ratio. Use TRUE for log-scale effects such as Cox or logistic models.

alpha

Significance threshold for prop_significant.

Value

A one-row tibble of vibration-of-effects summaries with class voe_summary.

References

Patel CJ, Burford B, Ioannidis JPA (2015). Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. Journal of Clinical Epidemiology, 68(9), 1046-1058.

Examples

res <- tibble::tibble(
  estimate = c(-0.1, 0.0, 0.2, 0.35, 0.5),
  p.value  = c(0.60, 0.90, 0.20, 0.04, 0.01)
)
voe(res)
#> <vibration of effects>
#>   models             : 5
#>   median effect      : 1.221
#>   relative ratio     : 1.80   (p99 / p01)
#>   Janus fraction     : 0.60   (0 or 1 = stable sign)
#>   prop. significant  : 0.40