The Atlas 6,943 concepts
☆ Favorites

Goodhart Law Gaming

Cognitive Biases Failure mode Empirical
Contextual Analysis
Also known as: Goodhart Law Exploitation, Goodharts Law Exploitation
Detection: high Stability: persistent Level: intermediate
When a goal becomes a rule for judging success, people change behavior to meet that rule. This change makes the rule a poor guide for the real goal.
Goodhart-style gaming occurs when an operational metric becomes the target, inducing agents to optimize for the metric rather than the underlying objective. This leads to metric capture where the proxy diverges from the true performance due to adaptive responses and incentive distortion.
A school district measures teacher quality by student test scores, so teachers begin spending all instructional time on test-prep drills. Scores rise, but students' actual understanding of the subject weakens—the metric improved while the real goal was undermined.
A hospital system adopts 30-day readmission rate as its primary quality KPI under a value-based purchasing contract. Clinical teams respond by extending initial admissions beyond medical necessity, discharging patients to skilled nursing facilities that reset the 30-day clock, and selectively avoiding high-risk patient panels whose comorbidities inflate readmission probability. The proxy metric improves on paper—satisfying the payer's compliance algorithm—while true care continuity and population health outcomes degrade. This is a textbook case of metric capture: the proxy's centrality in the incentive architecture channels agent strategies entirely toward proxy optimization, producing systematic divergence between observed KPI values and the latent objective of durable patient health.
When the score is used as the target, people change actions to boost that score. Those actions make the score stop matching the real goal.
The proxy metric sits within the observation channel and is selectively emphasized, creating asymmetrical incentives that weight actions by their effect on the proxy. This structural weighting yields adaptive behavior and constraint exploitation, producing divergence between observed proxy values and the latent objective.
Change how success is measured to include more of the real goal. Watch for new tricks and adjust measures over time.
Use composite metrics and randomized audits to reduce single-proxy domination and detect manipulation. Implement adaptive evaluation that reweights signals and penalizes narrow optimization strategies.
proxy_capture; perverse_incentives; gaming_externalities
An adversarial actor can deliberately design or lobby for a proxy metric they know is easy to game, then exploit the resulting divergence between the metric and the true objective for private gain while appearing compliant—e.g., a contractor hitting numerical delivery targets while degrading actual service quality. In competitive or regulatory contexts, sophisticated actors can reverse-engineer evaluation rubrics to surface-optimize for inspectable signals, effectively weaponizing auditors' reliance on the proxy against the auditing institution itself. This is especially potent in opaque environments where the principal cannot cheaply observe the latent objective, enabling sustained metric capture with minimal detection risk.
Rotate and randomize the composition of composite metrics over time to prevent agents from locking in on stable gaming strategies, and supplement quantitative proxies with qualitative audits that assess the latent objective directly. Build adaptive evaluation systems that penalize sudden, unexplained improvements in a narrow proxy (a statistical flag for gaming) and reward consistency across multiple independent signals. Structural separation of those who define metrics from those who benefit from them reduces principal-agent conflicts that enable proxy capture.