Learning/ AI Healthcare Quality & Safety/ Lesson 06

Lesson 06 of 10AI Healthcare Quality & Safety

Clinical Decision Support
Promise, Peril & Practice

Clinical decision support is the most pervasive form of AI in healthcare — embedded in every electronic health record, shaping clinical behavior at the point of care. Understanding how CDS works, when it helps, when it harms, and how to govern it is foundational knowledge for every quality and safety professional.

What you will learn

Define clinical decision support and describe the spectrum from simple alerts to AI-powered CDS

Explain the evidence base for clinical decision support effectiveness and its limitations

Describe the mechanisms by which CDS causes alert fatigue and unintended consequences

Apply governance principles to the design, deployment, and monitoring of clinical decision support

Distinguish between AI-enhanced CDS and traditional rule-based CDS and the governance differences between them

Start Lesson Back to Course

What clinical decision support is
and the spectrum of its forms

Clinical decision support encompasses any system that provides clinicians, patients, or other healthcare stakeholders with knowledge and person-specific information, presented at appropriate times, to enhance health and health care. This broad definition encompasses a spectrum from simple reminder systems to sophisticated AI-powered recommendation engines.

At the simpler end: allergy alerts that fire when a clinician prescribes a drug to which the patient has a documented allergy. Drug-drug interaction alerts that flag potentially harmful medication combinations. Dosing guidance embedded in the prescribing interface. Preventive care reminders that prompt vaccination or screening recommendations. These rule-based forms of CDS are the most prevalent and have the strongest evidence base.

At the more complex end: AI-powered diagnostic support systems that analyze multiple data streams and suggest differential diagnoses. Sepsis prediction systems that generate early warning alerts based on real-time vital sign and laboratory trends. Natural language processing tools that extract structured information from clinical notes to surface clinically relevant data. The governance requirements increase with complexity, because the reasoning behind complex CDS outputs is harder to verify and its failure modes are less predictable.

CDS Is Everywhere

Most clinicians interact with clinical decision support hundreds of times per shift without recognizing it as AI or algorithmic output. The alert that fires when they prescribe, the suggested diagnosis code in the documentation template, the risk score in the patient header — these are all forms of CDS shaping clinical decisions at scale.

Alert fatigue
the governance failure hiding in plain sight

The most comprehensively documented failure of clinical decision support is alert fatigue — the habituation of clinical users to frequent alerts that results in reflexive dismissal, including of clinically important alerts. Studies consistently show that clinical teams override 49–96% of all drug alerts in common EHR systems. The rate at which clinicians override even high-priority alerts has been measured at over 75% in multiple healthcare settings.

Alert fatigue is not a technology failure. It is a governance failure. It arises when the volume of alerts exceeds the capacity of clinical workflows to accommodate meaningful responses — when alerts fire for conditions that are already known and managed, when the clinical significance of alerts is unclear, when the recommended action is ambiguous, or when the alert interrupts a workflow at a point where acting on it is impractical.

The governance response to alert fatigue is not simply reducing alert volume. It is redesigning the alert system based on evidence of clinical utility — measuring which alerts change clinical behavior in beneficial ways, which are systematically overridden without investigation, and which generate harmful unintended consequences. Alert governance is an ongoing clinical quality activity, not a one-time implementation task.

AI-enhanced CDS
and the governance differences it requires

Traditional rule-based CDS fires based on explicit, auditable logic — if drug A is prescribed to a patient with allergy B, fire an alert. The rule can be reviewed, validated, and updated. AI-enhanced CDS fires based on learned patterns that may not be directly interpretable. This fundamental difference changes the governance requirements.

For AI-powered CDS, clinicians need to understand what the system is detecting, how confident it is, and what its known performance characteristics are in the local patient population. Governance frameworks must include mechanisms for ongoing performance monitoring, bias assessment, and transparent communication to clinical users about the nature and limitations of AI-generated recommendations.

The integration of AI-enhanced CDS with existing rule-based systems creates additional complexity — alert stacking, where AI-generated and rule-based alerts compete for clinical attention simultaneously. Organizations deploying AI-enhanced CDS must audit the total alert burden across all CDS systems and actively manage the combined experience rather than treating each system in isolation.

Alert Governance

Every alert in a clinical system should be able to answer three questions: What is the evidence that this alert changes clinical behavior in beneficial ways? What is the current override rate and has it changed since deployment? What happens when this alert is dismissed? If these questions cannot be answered, the alert governance program needs attention.

Key concepts
from this lesson

Key Concept

Clinical Decision Support

Any system providing clinicians or patients with knowledge and person-specific information to enhance health decisions.

Key Concept

Rule-Based CDS

Decision support based on explicit, auditable if-then logic — allergy alerts, drug interaction checks, dosing guidance.

Key Concept

Alert Override Rate

The proportion of alerts dismissed without the recommended action — a primary metric for alert fatigue surveillance.

Key Concept

Alert Governance

The ongoing organizational process of measuring, evaluating, and optimizing clinical decision support to maximize benefit and minimize burden.

Key Concept

Alert Stacking

The cumulative alert burden when multiple CDS systems generate simultaneous or competing alerts — a governance concern in complex EHR environments.

Key Concept

Interruptive vs Passive CDS

Interruptive CDS requires active response before workflow can continue. Passive CDS presents information without interrupting workflow. Both have clinical utility and governance requirements.

Case Study

96% of alerts ignored — and one that mattered

An academic medical center audit reveals that physicians in one department are overriding 96% of drug-drug interaction alerts without changing the prescription or documenting a clinical rationale. The quality team flags this as an alert fatigue concern.

Analysis of the 96% reveals: 71% of overridden alerts are for interactions of theoretical concern but no documented clinical significance at therapeutic doses; 19% are for medications the patient is already taking on a chronic basis; 6% are for interactions that have been previously reviewed and accepted for this patient. The remaining 4% — the actually clinically important alerts — cannot be distinguished from the others in the alert interface. They look identical.

Three months after redesigning the alert system — removing the lowest-priority theoretical interactions, implementing a different visual hierarchy for high-priority alerts, and adding a documentation pathway for chronic medication exceptions — the override rate for high-priority alerts drops from 82% to 31%. Two medication harm events attributable to missed high-priority drug interactions are prevented in the following quarter.

What this illustrates

The problem was not that clinicians were dismissing alerts carelessly. The problem was that the alert system had no mechanism for distinguishing important alerts from background noise. Alert governance — measuring utility, redesigning based on evidence, and differentiating by clinical priority — produced measurable patient safety benefit without requiring any technology replacement.

Reflection Prompt

What is the override rate for your organization's most important alerts?

Do you know the override rate for your organization's most critical clinical decision support alerts — sepsis warnings, high-risk medication alerts, critical laboratory value notifications? If not, that gap in your safety intelligence is itself a governance concern. Override rate is one of the most informative leading safety indicators available in any organization using electronic clinical decision support. The question is whether anyone is looking at it.

↗

Further Learning

IHI resources on clinical decision support and medication safety governance are available at ihi.org. AHRQ also publishes extensive guidance on CDS governance and the evidence base for specific alert types.

Knowledge Check — Lesson 06

1. A physician receives a high-priority sepsis alert and clicks 'override' without reviewing the patient's clinical status because she has overridden this alert for the same patient three times in the past 12 hours. This is best described as:

AA clinical error caused by physician negligence

BAlert fatigue — a governance failure resulting from an alert system that fires with insufficient precision

CAutomation bias — the physician is over-relying on past override decisions

DAn appropriate clinical judgment — the physician knows the patient's status

Correct. Correct. This is alert fatigue — the physician has developed habituated dismissal because the alert fires repeatedly without clear new clinical utility. The governance failure is the alert system design, not the individual physician's behavior.

Review the lesson. Review the lesson. Alert fatigue is a system design failure — when alert volume exceeds clinical capacity for meaningful response, habituation is the predictable human behavioral response. Attributing this to individual negligence misses the governance problem.

2. Which metric is most useful for ongoing governance of a clinical decision support alert system?

AThe total number of alerts generated per month

BThe override rate stratified by alert priority level and trend over time

CThe number of alerts that result in a changed prescription

DThe time clinicians spend reviewing each alert on average

Correct. Correct. Override rate stratified by priority level — and tracked over time — is the most informative governance metric. It reveals whether clinicians are responding differently to high and low priority alerts, whether alert fatigue is developing or resolving, and whether specific alert types have become clinically meaningful.

Review the lesson. Review the lesson. Total alert volume tells you scale but not effectiveness. Override rate by priority, tracked over time, is the governance metric that reveals behavioral patterns and allows targeted intervention.

3. An AI-powered diagnostic support system recommends including pulmonary embolism in the differential diagnosis for a patient. The key governance difference between this recommendation and a rule-based allergy alert is:

AThe AI recommendation is more likely to be correct because it uses more data

BThe AI recommendation is based on learned patterns that may not be directly interpretable, requiring different transparency and oversight standards

CRule-based alerts are held to a higher evidence standard than AI-powered CDS

DThere is no governance difference — all clinical decision support requires the same oversight approach

Correct. Correct. AI-powered CDS fires based on learned patterns that may not be directly interpretable, unlike rule-based CDS where the logic is explicit and auditable. This opacity requires additional governance — transparency to clinical users, ongoing performance monitoring, and bias assessment.

Review the lesson. Review the lesson. The fundamental governance difference between rule-based and AI-enhanced CDS is interpretability. Rule-based logic is auditable. AI-learned patterns may not be. This changes the transparency requirements, monitoring approach, and the information clinicians need to appropriately calibrate their response.

4. An organization deploys three new AI-powered clinical decision support tools in the same quarter — sepsis prediction, fall risk, and deterioration detection. The primary governance concern unique to this combined deployment is:

AThe three systems may use different underlying machine learning architectures

BAlert stacking — the combined alert burden may exceed clinical workflow capacity even if each system is individually calibrated

CThe three systems require three separate governance committees with overlapping membership

DAI-powered tools require quarterly retraining that must be coordinated across all three systems simultaneously

Correct. Correct. Alert stacking — where multiple CDS systems generate simultaneous or competing alerts — is a governance concern that arises specifically when multiple systems are deployed together. The combined alert burden may be clinically unmanageable even when each individual system is calibrated appropriately.

Review the lesson. Review the lesson. Alert stacking is the governance risk unique to deploying multiple CDS systems simultaneously. Organizations must audit the total alert experience — not just each system individually.

5. Which approach to reducing alert fatigue is most likely to improve patient safety while maintaining clinical utility?

ARemoving all low-priority alerts from the system immediately

BRequiring a physician co-signature for all alert overrides to increase accountability

CAnalyzing override rates and clinical utility evidence to selectively remove or redesign low-utility alerts while maintaining high-priority visibility

DReplacing all rule-based alerts with AI-powered alerts that have higher specificity

Correct. Correct. Evidence-based alert governance — measuring utility, stratifying by priority, selectively removing low-utility alerts, and redesigning presentation of high-priority alerts — addresses alert fatigue while preserving clinical safety value. Blanket removal risks eliminating genuinely important alerts.

Review the lesson. Review the lesson. The governance response to alert fatigue is selective, evidence-based redesign — not blanket removal or increased bureaucratic burden. The goal is making high-priority alerts visible and actionable, not reducing all alert volume indiscriminately.

Clinical Decision SupportPromise, Peril & Practice

What clinical decision support isand the spectrum of its forms

Alert fatiguethe governance failure hiding in plain sight

AI-enhanced CDSand the governance differences it requires

Key conceptsfrom this lesson

Clinical Decision Support

Rule-Based CDS

Alert Override Rate

Alert Governance

Alert Stacking

Interruptive vs Passive CDS

96% of alerts ignored — and one that mattered

What is the override rate for your organization's most important alerts?

Clinical Decision Support
Promise, Peril & Practice

What clinical decision support is
and the spectrum of its forms

Alert fatigue
the governance failure hiding in plain sight

AI-enhanced CDS
and the governance differences it requires

Key concepts
from this lesson