KPI Systems and the Illusion of Safety: Measurement, Incentives, and Epistemic Blindness

Lessons from High-Risk Industries and Paths Toward Resilient Performance Management Abstract Key Performance Indicators (KPIs) are widely hailed as essential tools for organizational governance, promising objectivity, transparency, and control—especially in high-risk domains such as oil and gas, nuclear power, aviation, healthcare, and chemical manufacturing. However, the reliance on KPIs often produces epistemic blindness: a failure to recognize risks and realities that lie outside the scope of measurement. This article critically examines the theoretical foundations of measurement, the performative nature of KPIs, and the mechanisms that produce epistemic blindness. Drawing on case studies and empirical evidence from multiple industries, it reveals how KPIs can mask underlying vulnerabilities and foster an illusion of safety. Practical recommendations are o ered for designing resilient KPI systems, fostering organizational learning, and integrating qualitative and quantitative knowledge. The synthesis underscores that genuine safety and risk management require moving beyond metric-driven assurance toward adaptive, critical, and inclusive practices.

Introduction

Key Performance Indicators (KPIs) have become central instruments in the governance of contemporary organizations, particularly in high-risk sectors where the stakes of safety, reliability, and performance are profound. Originally introduced to provide objective, comparable metrics for evaluating performance (Kaplan & Norton, 1992; Power, 1997), KPIs are now embedded in regulatory frameworks, incentive structures, and management routines. The promise is clear: transform complex realities into manageable numbers. Yet, as this article will show, the widespread adoption of KPIs has revealed significant limitations, including the risk of fostering an illusion of safety and epistemic blindness—where critical risks go unrecognized because they are not captured by measurement systems (Behn, 2003; Espeland & Stevens, 1998). The dangers are particularly acute in complex socio-technical systems, where safety is emergent and cannot be reduced to static indicators (Hollnagel, 2014). This article analyses the use of classical backward-looking KPI systems and their impact on organizational knowledge, behavior, and safety. Through concrete examples and academic sources, we examine the mechanisms by which KPIs shape incentives, attention, and decision-making, and how epistemic blindness arises. References from oil and gas, nuclear power, aviation, healthcare, chemical manufacturing, and other sectors illuminate the real-world consequences of metric-driven assurance. Finally, practical recommendations are o ered for designing more resilient KPI systems and fostering organizational learning.

Theoretical Foundations: Measurement, Metrics, and Epistemic Blindness

The measurement theory reveals that metrics are not mere reflections of reality; they are social constructions shaped by technical, political, and cultural factors (Espeland & Stevens, 1998; Porter, 1995; Desrosières, 1998; Weick, 1995). The process of quantification involves choices—what to measure, how to measure, and what to ignore. These choices are rarely neutral and reflect organizational priorities, power dynamics, and prevailing epistemologies (Power, 2004; Hollnagel, 2014). KPIs reify abstract concepts such as safety, making them appear manageable and objective. However, the complexities of high-risk environments often elude quantification. Measurement systems inherently filter reality, foregrounding some signals while relegating others to the background (Reason, 1997; Perrow, 1984). The performative nature of KPIs means that once embedded in routines, they shape how actors interpret situations and define priorities (Espeland & Stevens, 1998; Porter, 1995). The phenomenon of epistemic blindness arises when organizations become incapable of perceiving or responding to risks outside the scope of measurement. This is not merely a technical failure, but a product of social, cognitive, and political processes that determine what knowledge is constructed and validated (Vaughan, 1996; Turner, 1976; Taleb, 2007). The literature on risk and performance management warns that over-reliance on quantitative metrics can crowd out qualitative judgment, intuition, and critical inquiry (Hollnagel, 2014; Dekker, 2011). Recent research has deepened understanding of these dynamics. Hollnagel’s Safety-II paradigm emphasizes the need to capture variability and emergent risks, not just compliance with predefined metrics (Hollnagel, 2014). Woods (2006) and Dekker (2011, 2014) highlight the importance of resilience and adaptive capacity, arguing that measurement systems should support learning and anticipation rather than just retrospective assurance. The social construction of measurement is further explored by Desrosières (1998) and Espeland & Stevens (1998), who show how metrics become embedded in organizational politics and routines.

KPIs and Organizational Behavior: Shaping Attention, Incentives, and Decision-Making

KPIs function as filters of organizational attention, determining which signals are visible, actionable, and reportable (Kaplan & Norton, 1992; Power, 1997). The selectivity inherent in measurement means that phenomena such as safety or system reliability cannot be fully encapsulated by a finite set of indicators. KPIs foreground certain aspects of organizational reality while relegating others to the background—a process that is not merely technical but deeply social and political, reflecting prevailing priorities and power dynamics (Espeland & Stevens, 1998; Porter, 1995). Quantification can reify abstract concepts, making complex issues appear more manageable and encouraging gaming behaviors aimed at metric attainment rather than genuine improvement. Consequently, organizational learning and adaptation may be stifled, and blind spots proliferate, producing partial representations that risk being mistaken for complete accounts (Hood, 2006; Kerr, 1975). The performative nature of KPIs is central to their influence. Once embedded in routines, KPIs shape how actors interpret situations and define priorities, with measured dimensions gaining prominence at the expense of those left unmeasured (Holmström & Milgrom, 1991). Incentive structures tied to KPIs further reinforce these dynamics. When rewards, promotions, or penalties are linked to specific metrics, organizational actors are incentivized to optimize performance on those measures, sometimes at the expense of broader goals. This can foster gaming, manipulation, and strategic behavior—such as underreporting incidents or focusing on easily achievable targets—while critical risks remain unaddressed (Behn, 2003; Hood, 2006). The literature warns of the dangers of "teaching to the test," where the pursuit of high scores supplants genuine improvement (Kerr, 1975; Holmström & Milgrom, 1991). Organizational culture and structure also play crucial roles. Siloed information flows, fragmented responsibilities, and hierarchical reporting can impede the detection and escalation of emerging risks (Reason, 1997; Weick, 1995). Political interests and power dynamics may suppress dissenting voices or inconvenient truths, further reinforcing epistemic blindness (Vaughan, 1996; Turner, 1976). Recent empirical studies in healthcare (Mannion & Braithwaite, 2012), aviation (Stolzer, Halford, & Goglia, 2016), and nuclear power (IAEA, 2012) show that KPIs often drive superficial compliance, concealment of problems, and neglect of process safety. The focus on measurable outcomes crowds out attention to culture, learning, and adaptation—critical factors for resilience in high-risk environments (Hollnagel, 2014; Woods, 2006).

The Illusion of Safety: How KPIs Mask Underlying Risks

The promise of KPIs is to provide objective assurance of safety and performance. However, the limitations of measurement can create a false sense of security. When organizations rely on a narrow set of indicators, they may overlook emerging threats, systemic vulnerabilities, and dynamic risks that do not fit neatly within established metrics (Dekker, 2011; Hollnagel, 2014). This illusion of safety is particularly acute in complex socio-technical systems, where safety is emergent, context-dependent, and resistant to reductionist quantification (Perrow, 1984; Reason, 1997). KPIs can mask underlying risks in several ways. First, they may fail to capture weak signals, anomalies, or qualitative concerns that precede major incidents. Second, the pressure to achieve high scores can incentivize superficial compliance, concealment of problems, or neglect of unmeasured dimensions. Third, the focus on measurable outcomes can crowd out attention to process, culture, and learning—critical factors for resilience in high-risk environments (Hollnagel, 2014; Woods, 2006). The result is a partial representation of reality, where measured dimensions are mistaken for the whole, and safety is assumed rather than critically interrogated. The illusion of safety is not merely a cognitive error; it is reinforced by organizational routines, reporting structures, and external pressures. Boards, regulators, auditors, and stakeholders often demand quantitative evidence of performance, further entrenching the dominance of KPIs (Power, 1997; Porter, 1995). As a result, organizations may become blind to risks that fall outside the scope of measurement, and vulnerabilities may intensify beneath the surface of apparent control. Scholarly research in risk management and resilience engineering underscores these dynamics. Hollnagel (2014) argues that Safety-I approaches, which focus on the absence of negative outcomes, fail to anticipate emergent risks. Safety-II approaches emphasize learning from variability and near-misses, integrating qualitative and quantitative knowledge. Dekker (2011, 2014) and Woods (2006) highlight the importance of adaptive capacity and continuous learning, warning that metric-driven assurance can foster complacency and false confidence.

Epistemic Blindness: Unmeasured Risks and Organizational Ignorance

Epistemic blindness is a phenomenon where organizations become incapable of recognizing or responding to risks that are not captured by their measurement systems. This blindness arises from the social construction of knowledge: what is measured is deemed real and important, while what is unmeasured is ignored or marginalized (Espeland & Stevens, 1998; Porter, 1995; Power, 2004). In high-risk industries, this can have catastrophic consequences (Vaughan, 1996; Turner, 1976). The mechanisms of epistemic blindness are multifaceted. First, cognitive biases—such as confirmation bias and the tendency to focus on familiar metrics—can limit the scope of attention (Reason, 1990; Kahneman, 2011). Second, organizational silos and fragmentation may impede the flow of information about emerging risks (Weick, 1995; Reason, 1997). Third, power dynamics and political interests can suppress dissenting voices or inconvenient truths (Vaughan, 1996; Turner, 1976). Fourth, the technical limitations of measurement tools may prevent the detection of complex, latent, or evolving threats (Taleb, 2007; Hollnagel, 2014). Epistemic blindness is not simply a failure of individual actors; it is a systemic property of organizations. Once established, it can be self-reinforcing: the absence of evidence becomes evidence of absence, and unmeasured risks are assumed not to exist (Dekker, 2011; Taleb, 2007). This dynamic is particularly dangerous in high-risk industries, where the stakes of ignorance are high and the consequences of failure can be severe. Recent research on organizational accidents and disasters (Reason, 1997; Perrow, 1984; Vaughan, 1996) shows that epistemic blindness often precedes major incidents. Weak signals, anomalies, and dissenting voices are ignored or suppressed, and the focus on compliance with metrics creates blind spots. The literature on sensemaking (Weick, 1995) and organizational learning (Woods, 2006; Hollnagel, 2014) emphasizes the need for critical inquiry and adaptive capacity.

Case Studies: Analysis of Macondo, Three Mile Island, Challenger, Fukushima, Aviation, Healthcare, Chemical Manufacturing, and Others Macondo Disaster (Deepwater Horizon Oil Spill, 2010)

The Macondo disaster, culminating in the Deepwater Horizon oil spill, provides a vivid illustration of KPI-driven epistemic blindness. The Rig had just achieved 7 years without a Lost Time Injury and senrior exectuves for on Baord to celebrate this acheivement. Hpowever, prior to the blowout, multiple warning signals—including pressure anomalies, cement integrity concerns, and procedural deviations—were present but not e ectively captured or escalated (National Commission on the BP Deepwater Horizon Oil Spill and Oshore Drilling, 2011; Hopkins, 2012; U.S. Chemical Safety Board, 2016). Decision-making was conducted under cost and schedule pressures within a fragmented structure involving multiple contractors. KPI systems emphasized personal safety indicators, such as low lost-time injury rates, creating a perception of robust safety performance while process risks accumulated. A critical danger highlighted by the Macondo disaster was the direct linkage of safety KPIs to bonus and incentive systems. Tying financial rewards to the achievement of specific metrics, such as low injury rates or compliance scores, created powerful pressures for individuals and teams to prioritize the appearance of safety over its substance. This alignment encouraged the underreporting of incidents, superficial documentation, and the concealment of emerging problems—practices which undermined genuine risk awareness and learning. This dynamic contributed to a culture where meeting KPI targets became an end in itself, overshadowing the need to address underlying process safety issues or escalate concerns about abnormal conditions. Ultimately, the bonus-linked KPI system fostered complacency and a dangerous illusion of control, masking the true state of operational risk and contributing to the catastrophe. The focus on easily measurable KPIs, such as injury rates, diverted attention from systemic process safety risks. Critical warning signs—such as failed pressure tests and reports of abnormal conditions—were not systematically integrated into performance management or decision-making. Fragmentation of responsibility and emphasis on compliance with established metrics created blind spots: actors were incentivized to meet targets, not to interrogate the adequacy of those targets or seek out unmeasured risks. Investigations revealed that KPI systems contributed to a culture of complacency and false confidence. The illusion of safety was maintained by high scores on selected indicators, while underlying vulnerabilities intensified (Hopkins, 2012; U.S. Chemical Safety Board, 2016).

Three Mile Island (Nuclear Power, 1979)

The Three Mile Island accident was preceded by multiple warning signals that were not adequately captured by performance metrics (Perrow, 1984; Reason, 1997). The focus on compliance and measurable indicators led to the neglect of qualitative concerns and operator intuition, contributing to the escalation of the crisis. Operators relied on instrument readings and procedural compliance, while system complexity and emergent risks were overlooked. The incident exposed the limitations of metric-driven assurance and the need for integrating expert judgment and weak signal detection (IAEA, 2012; Perrow, 1984). Further analysis by the International Atomic Energy Agency (IAEA, 2012) shows that KPI systems in nuclear power often prioritize regulatory compliance and easily measurable indicators, while qualitative concerns such as safety culture, communication, and adaptability are marginalized. The literature on resilience engineering emphasizes the importance of learning from near-misses, anomalies, and qualitative signals (Hollnagel, 2014; Woods, 2006).

Challenger Launch Decision (NASA, 1986)

The Challenger disaster was the result of organizational epistemic blindness, where risks associated with O-ring failures were not adequately recognized or escalated (Vaughan, 1996). NASA’s performance management systems emphasized schedule adherence, compliance with procedures, and quantitative metrics. Dissenting voices and weak signals were suppressed, and the illusion of safety was maintained by high scores on established indicators. The incident underscores the dangers of partial measurement and the need for critical inquiry and adaptive learning (Vaughan, 1996; Reason, 1997).

Fukushima Daiichi (Nuclear Power, 2011)

The Fukushima disaster exposed the limitations of KPI systems in managing emergent and systemic risks. Regulatory compliance and measurable indicators were prioritized, while vulnerabilities related to design, emergency preparedness, and external threats (tsunami risk) were marginalized (IAEA, 2012; Dekker, 2014). Investigations revealed that performance management systems failed to capture the complexity and dynamic nature of risk, and that epistemic blindness was reinforced by organizational silos and regulatory pressures.

Aviation Incidents

In aviation, the emphasis on punctuality, on-time departures, and safety compliance metrics has sometimes led to the underreporting of near-misses and procedural deviations. While metrics improved, underlying risks persisted (Stolzer, Halford, & Goglia, 2016). The Air France Flight 447 crash (2009) and other incidents illustrate how KPI systems can mask weak signals and emergent threats, and how organizational routines can suppress critical inquiry (Stolzer et al., 2016; Reason, 1997). Recent research on aviation safety management systems (SMS) emphasizes the importance of integrating qualitative assessments, expert judgment, and weak signal detection. KPIs should be complemented by learning from near-misses, anomalies, and qualitative concerns (ICAO, 2013; Stolzer et al., 2016).

Healthcare Failures

Hospitals often use KPIs such as infection rates, readmission rates, and patient satisfaction scores. However, these can incentivize gaming behaviors, such as selective reporting or avoidance of high-risk patients, while systemic safety issues—such as communication breakdowns or cultural factors—remain unaddressed (Mannion & Braithwaite, 2012; Dixon-Woods et al., 2014). The Mid Sta ordshire NHS Trust scandal in the UK revealed how metric-driven assurance can foster complacency, concealment of problems, and neglect of qualitative concerns (Francis Report, 2013). Empirical studies show that performance management systems in healthcare often crowd out attention to process, culture, and learning. The literature on patient safety and resilience emphasizes the need for integrating qualitative and quantitative knowledge, fostering critical inquiry, and learning from failures (Wears & Vincent, 2013; Hollnagel, 2014).

Chemical Manufacturing Accidents

In chemical manufacturing, KPI systems often prioritize environmental compliance and safety audits, but fail to recognize process safety hazards that are not included in the metrics (Center for Chemical Process Safety, 2007). The Texas City refinery explosion (BP, 2005) and other major accidents illustrate how metric-driven assurance can mask weak signals and emergent risks (Hopkins, 2000; U.S. Chemical Safety Board, 2007). Investigations revealed that organizations met their KPI targets, but underlying vulnerabilities intensified. The literature on process safety management emphasizes the importance of integrating qualitative assessments, weak signal detection, and learning from anomalies (Center for Chemical Process Safety, 2007; Hopkins, 2000).

Other Real-World Examples

• Rail Transport: The Ladbroke Grove rail crash (UK, 1999) was preceded by multiple weak signals and procedural deviations that were not captured by KPI systems. The focus on punctuality and compliance crowded out attention to safety culture and learning (Rail Accident Investigation Branch, 2009). • Mining: The Pike River mine disaster (New Zealand, 2010) revealed how KPI systems focused on production targets and compliance metrics, while process safety and weak signals were ignored (Royal Commission on the Pike River Coal Mine Tragedy, 2012). • Financial Services: The global financial crisis (2007–2008) exposed the limitations of risk management systems that prioritized quantitative metrics (e.g., Value at Risk), while systemic vulnerabilities and qualitative concerns were marginalized (Taleb, 2007; Power, 2004). These examples illustrate the widespread nature of the problem: when KPIs dominate attention and incentives, organizations can become blind to risks that are not easily measured. The illusion of safety persists until reality intrudes, often with devastating consequences.

Implications for Practice: Designing Resilient KPI Systems and Fostering Organizational Learning

The limitations of KPI systems demand a critical re-evaluation of performance management in high-risk industries. To mitigate the dangers of the illusion of safety and epistemic blindness, organizations must adopt more resilient and adaptive approaches to measurement, incentives, and learning (Hollnagel, 2014; Woods, 2006; Dekker, 2011). 1. Broaden the Scope of Measurement: KPIs should be complemented by qualitative assessments, expert judgment, and weak signal detection. Organizations must recognize that not all critical risks can be quantified, and that a balanced approach is necessary (Hollnagel, 2014; ICAO, 2013). 2. Foster a Culture of Critical Inquiry: Leadership should encourage questioning of metrics, exploration of blind spots, and the identification of unmeasured risks. Dissenting voices and alternative perspectives should be valued, not suppressed (Vaughan, 1996; Francis Report, 2013). 3. Integrate Learning and Adaptation: Performance management systems should be designed to support continuous learning, feedback, and adaptation. Near misses, anomalies, and failures should be treated as opportunities for improvement, not as threats to reputation (Woods, 2006; Hollnagel, 2014). 4. Align Incentives with Broader Goals: Incentive structures should reward behaviors that contribute to organizational resilience, not merely the attainment of specific metrics. This may involve recognizing contributions to process safety, culture, and innovation (Holmström & Milgrom, 1991; Center for Chemical Process Safety, 2007). 5. Engage Stakeholders in Measurement Design: Regulators, auditors, and stakeholders should be involved in the design and review of KPIs to ensure that metrics reflect the realities of risk and performance, not just compliance (Power, 2004; Mannion & Braithwaite, 2012). 6. Embrace Complexity and Uncertainty: High-risk industries must accept that safety is dynamic and emergent. Measurement systems should be flexible, adaptive, and capable of capturing evolving threats and opportunities (Hollnagel, 2014; Dekker, 2011). 7. Integrate Qualitative and Quantitative Knowledge: Performance management should bridge organizational silos and integrate multiple forms of knowledge, including expert judgment, frontline experience, and stakeholder perspectives (Wears & Vincent, 2013; Weick, 1995). 8. Develop Leading Indicators: Use leading indicators that measure activities and behaviors supporting resilience, such as training, communication, and proactive risk identification (Center for Chemical Process Safety, 2007; ICAO, 2013). 9. Enhance Transparency and Accountability: Reporting structures should support transparency, accountability, and critical reflection. Metrics should be subject to review, challenge, and continuous improvement (Power, 2004; Francis Report, 2013). 10. Support Organizational Learning: Encourage learning from near-misses, incidents, and failures. Use incident reviews, after-action reports, and feedback loops to drive improvement and adaptation (Woods, 2006; Hollnagel, 2014). By adopting these practices and using leading indicators measuring these activities, organizations can move beyond the illusion of safety and develop more robust approaches to risk management. The goal is not to abandon KPIs, but to use them wisely, critically, and in conjunction with other forms of knowledge (Hollnagel, 2014; Dekker, 2011; Woods, 2006).

Conclusion

KPIs are powerful instruments for organizational governance, but their limitations are profound. When measurement systems are narrow, incentives are misaligned, and epistemic blindness takes hold, organizations become vulnerable to the illusion of safety and the escalation of unrecognized risks. Case studies from Macondo, Three Mile Island, Challenger, Fukushima, aviation, healthcare, chemical manufacturing, and other sectors illustrate the dangers of partial measurement and over-reliance on metrics. To foster organizational learning and resilience, leaders must critically interrogate their KPI systems, embrace complexity, and cultivate cultures of inquiry and adaptation. Future research should explore new approaches to performance management that integrate quantitative and qualitative knowledge, bridge organizational silos, and enhance the capacity to detect and respond to emerging threats. Only by moving beyond the illusion of safety can high-risk industries achieve genuine progress in risk management and organizational performance.