Selection Bias

Selection bias happens when the people or items studied are not a good match for the whole group they represent. This makes results misleading because some kinds of people or items are left out on purpose or by accident.

Selection bias is a systematic distortion that occurs when the sampled subset differs non-randomly from the target population, altering observed associations. It compromises external validity and can induce spurious correlations between variables when inclusion depends on exposure or outcome.

A news website runs a poll asking readers "Do you support this new policy?" and reports that 80% of respondents are in favor. But only people who already visit that particular website — who may strongly share its editorial slant — bothered to click and answer, so the result tells us nothing reliable about what the general public thinks.

A retrospective cohort study of occupational chemical exposure and lung-cancer mortality recruits workers still employed at the plant as of the study start date, inadvertently triggering the healthy worker survivor effect: workers who became ill and left employment earlier are excluded, truncating the high-exposure, high-mortality stratum. The resulting hazard ratio is attenuated toward the null, and the conditional independence assumption required for unbiased Cox regression is violated because inclusion probability is a collider function of both exposure level and early-exit health status. Applying inverse-probability weighting using time-varying employment propensity scores, calibrated against company payroll records that capture departed workers, partially restores exchangeability and shifts the estimated HR from 1.1 (95% CI: 0.9–1.4) to 1.6 (95% CI: 1.2–2.1), revealing the previously masked dose-response relationship.

If people with certain traits are more likely to be included, results will lean toward those traits. Leaving out others changes averages and hides true links between things.

Selection operates via unequal inclusion probabilities from enrollment rules or censoring, creating asymmetry in observed distributions and constrained support for variables; for example, truncated sampling of a subpopulation skews estimated associations. Structural elements like sampling frames and eligibility criteria weight observations unevenly, producing biased parameter estimates.

Try to include a wide range of people and reduce rules that leave out groups. Check if the sample looks like the whole group you care about and adjust if it does not.

Use randomized sampling frames, weighting adjustments, and sensitivity analyses to correct for unequal inclusion probabilities and test robustness. Implement inverse-probability weighting or selection models to mitigate bias from observed selection mechanisms.

Nonrandom participation; Attrition over time; Measurement-dependent inclusion

An adversarial actor can deliberately engineer selection bias by designing eligibility criteria or recruitment channels that systematically exclude populations whose outcomes would contradict a desired conclusion — for example, a pharmaceutical sponsor enrolling only healthy, young trial participants to inflate efficacy estimates and suppress adverse-event rates. In observational contexts, a motivated analyst can restrict the analytic sample post-hoc to a subgroup where the target association is strongest, then report findings as if they apply universally. Survey or polling operations can weaponize voluntary response mechanisms to over-represent sympathetic demographics, manufacturing consent for a preferred narrative.

Pre-register sampling frames, eligibility criteria, and exclusion rules before data collection to prevent post-hoc sample manipulation and enable independent audit. Apply inverse-probability weighting or selection models calibrated against external benchmarks (e.g., census data) to correct for known asymmetric inclusion probabilities. Conduct sensitivity analyses — including worst-case bounds and E-value calculations — to quantify how much unmeasured selection would be required to nullify observed associations, making covert bias harder to conceal.