Definition
Bootstrap sampling resamples observations from the observed sample with replacement to create many same-size pseudo-samples. A statistic computed on each pseudo-sample approximates its sampling distribution under the empirical distribution. Ordinary iid bootstrap resampling is not automatically appropriate for dependent time series.
How it is calculated
A bootstrap sample X_1 star through X_n star consists of n independent draws with replacement from the empirical distribution F-hat-n.
| Symbol | Meaning | Unit |
|---|---|---|
| X | observed variable | units of observation |
| n | original sample size | count |
| F | empirical distribution | probability distribution |
Interpretation
Treat the observed data as a small jar of plausible observations and draw a new jar by putting each draw back. Repeating that process shows how much the statistic changes under resampling. The jar cannot reveal patterns that the original sample never contained.
Assumptions and conventions
-
The empirical distribution is a useful approximation to the data-generating distribution.
-
For the ordinary iid bootstrap, original observations are iid; exchangeability alone is insufficient. Dependent data need a suitable block or model bootstrap.
Resampled observations retain the units of X; n is a count.
Limitations and common misreadings
-
Resampling rows independently can destroy time dependence.
-
A bootstrap distribution reflects the observed sample and its limitations.
References
- B. Efron. (1979). Bootstrap Methods: Another Look at the Jackknife. The Annals of Statistics. Sections 1–2. doi:10.1214/aos/1176344552. Source

