Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system

Picture a primary care visit that's already running late. The clinician is juggling symptoms, past history, and a worried patient—and then the screen pings. Another alert. And another. Some are vital, like a dangerous drug-drug interaction. Others, not so much. After a while, those pings blur together. That blurring—alert fatigue—isn't just annoying; it's linked to overrides and potentially to missed or delayed care. Across the literature, clinicians click past alerts a lot. Depending on the study and the setting, override rates range from roughly half to almost all alerts. That's a sobering backdrop for software meant to keep patients safe. Why does this happen? Two stories compete. One is cognitive overload: too many alerts, too much noise, not enough time to sift the few important pieces from the gravel. The other is desensitization: the idea that, like a car alarm you've heard too many times, even a good alert loses its punch as you get used to it. Which one dominates matters because the fixes are different. If overload rules, you need fewer, sharper alerts. If desensitization rules, you need novelty—fresh phrasing, different timing, a reset of attention. Ancker and colleagues set out to tease those apart in the real world, not a lab. They turned to the Institute for Family Health, a network of safety-net clinics in and around New York City that's been using EpicCare since 2003. The team pulled three and a half years of electronic health record data—from January 2010 through June 2013—for 112 ambulatory primary care clinicians, consisting of 93 physicians and 19 nurse practitioners. The scale is big: 1.26 million best-practice advisories and 326 thousand drug-drug or drug-allergy alerts, spread over about 431 thousand encounters and just shy of 100 thousand patients. In other words, this is the kind of volume where small effects show up, and patterns stop being anecdotes. They tracked two families of alerts. Best-practice advisories are those reminders to give a vaccine, order a screening test, or manage a chronic condition. The high-stakes drug alerts include drug-drug interactions and drug-allergy contraindications. Notably, they focused the drug side only on interactions and allergies, because other medication alerts had such high override rates they weren't going to be interpretable. For acceptance, they used a practical definition: a best-practice advisory counted as accepted if the clinician clicked "accept" or opened the highlighted order set. Drug alerts were recorded with the clinician's usual accept-override response. Here's the clever part. Because there isn't a gold-standard scale for "workload" or "alert informativeness," the team built clinician-level markers that made sense to front-line doctors and nurses who co-authored the study. The amount of work was measured by the number of unique patients and the number of encounters per year. The complexity of work was determined by the number of alerts per encounter, plus the comorbidity load of the clinician's panel measured with the Johns Hopkins Aggregated Diagnosis Groups, a method that rolls diagnoses into a comorbidity count. Potential low-value or low-information alerts were captured as the proportion of alerts that repeated for the same clinician, same patient, within the same year. Think of a flu shot reminder that fires again and again on a patient who declines—plausibly less informative on the fifth try than the first. Statistically, they leaned into models suited to counts with a lot of dispersion: negative binomial regressions. Acceptance rates were the outcome, and the logarithm of the number of alerts fired was an offset, with the coefficients reported as incident rate ratios—how the acceptance rate changes for a clinician with, say, more repeats compared to one with fewer repeats. For the time-course question: does acceptance tail off as a new alert ages? They computed acceptance per provider per month from the date the alert first went live, smoothed the curves, and estimated slopes using Poisson or negative binomial variants as needed. It's a lot of technical plumbing, but in plain terms, they asked whether more alerts, more repeated alerts, and more complex patients go hand in hand with more clicking past, and whether attention erodes over time. The first finding is almost a character in its own right: repetition. In this system, repeats were everywhere. For best-practice advisories, the median clinician saw twenty-six point two percent of their advisories repeat for the same patient within a year. For drug alerts, that median was even higher at thirty-one point eight percent. That's not a stray edge case; that's a quarter to a third of the stream. What did those repeats, and the sheer density of alerts, do to acceptance? For best-practice advisories, the pattern is striking. Each extra advisory per encounter predicted a meaningful drop in the odds that the clinician would accept the next one. The incident rate ratio was 0.70 per additional advisory per encounter—about a thirty percent reduction—highly significant. As repeats stacked up, acceptance sagged further: every five percentage point increase in the share of repeats was linked to a ten percent drop in acceptance, with an incident rate ratio of 0.90. Those are clinician-level effects, not just one-off visits, and they held in multivariable models that controlled for other features of the job. If you're wondering whether this is just about how busy a clinician is—more patients per year, more visits per year—that's the interesting counterpoint. The traditional workload measures didn't predict acceptance once you accounted for alert structure. For best-practice advisories, the number of patients a clinician saw per year was essentially a wash, with an incident rate ratio around 1.00 and a p-value indicating no significant result. The same story holds for encounters per year. In other words, it wasn't the number of people walking through the door; it was what the software did during each of those visits. The complexity of the patient panel added a nuance. Clinicians whose patients had higher comorbidity loads tended to accept slightly fewer advisories. The estimate pointed toward lower acceptance as complexity rose—the incident rate ratio hovered around 0.50—but that trend narrowly missed the conventional cutoff for statistical significance. Still, it fits the picture: the more clinically complex the backdrop, the less bandwidth left for marginal alerts. Drug alerts live in a different neighborhood. They are more severe in concept, and acceptance is low overall, creating a floor effect—there's not much room to go lower. Even so, the direction of the relationship with repeats looks familiar. As the proportion of repeated drug alerts climbed, acceptance trended down; the estimate for a five-point increase in repeats was an incident rate ratio of about 0.84, with a p-value nudging significance in some models and missing in others. However, one robust and memorable difference popped: nurse practitioners were much more likely than physicians to accept drug alerts. In multivariable analysis, the incident rate ratio was 4.56, with a confidence interval that didn't overlap 1 and a p-value of 0.002. That's a big gap—one that held up even after adjusting for the complexity of the patients they cared for. If cognitive overload is driving the bus, what about desensitization—the slow fade in attention as an alert grows stale? Here the team did a focused time-series look at six newly launched best-practice advisories. They tracked each clinician's acceptance month by month from the alert's debut. If desensitization were strong, you'd expect to see clean, downward-sloping lines. That's not what they saw. Only one alert showed an early peak followed by a decline, and even that likely reflected seasonality—think flu-vaccine reminders swelling in the fall and ebbing after the winter push. Across the six, slope estimates ranged from a small positive bump to a modest negative drift, with p-values scattered from 0.03 to 0.24. No consistent downward trend. In short, habituation to a specific new alert didn't explain the fatigue they were seeing. It's worth pausing on what those statistics mean under the hood. In the negative binomial setup, an incident rate ratio below 1.0 says, "as this thing goes up, acceptance rates go down." So an incident rate ratio of 0.70 per extra advisory per encounter translates to a thirty percent reduction in acceptance for each step up in alert density during a visit. An incident rate ratio of 0.90 per five point increase in repeats says acceptance falls about ten percent as the share of repeats ticks up by that increment. Those are multiplicative effects; they stack. Taken together, the pattern has a shape. Alerts fire in dense clusters during complex visits. A sizeable share are repeats. As density and repeats rise, clinicians accept fewer alerts, especially the advisory-style nudges. Swapping in more patients per year or more visits per year doesn't move that needle. Over time, brand-new alerts don't reliably lose their potency. That shape looks a lot like cognitive overload and not much like desensitization. There are caveats. This is observational data from a single electronic health record in a specific care network. The team used reasonable, clinician-approved proxies for things like workload and informativeness, but they couldn't hand-label a million alerts to separate the gems from the gravel. They also didn't have severity gradings for each drug interaction, and they couldn't capture ambient workflow factors—like how many interruptions the clinician faced or what else was happening in the room. The desensitization analysis was constrained to six new advisories with enough time on the market to analyze, and the statistical models for monthly acceptance had to use robust errors rather than cluster-robust ones because some months had no responses. None of that invalidates the pattern, but it sets the boundaries for where we should be confident. So what does this mean if you're designing or tuning a clinical decision support system? Start with the repeats. In this dataset, one in four advisories and one in three drug alerts were repeats for the same patient within a year. That's a lot of déjà vu. The acceptance penalty attached to repeats was clear for advisories and directionally similar for drug alerts. De-duplicating within a patient-clinician dyad, throttling how often a reminder reappears after a recent dismissal, or bundling the information into a single, high-value summary could reclaim cognitive bandwidth. Next, respect the per-encounter alert budget. The thirty percent drop in advisory acceptance per additional advisory per visit isn't subtle. It tells you that the marginal alert in a crowded moment is likely to be ignored. That doesn't mean hiding critical safety warnings; it means front-loading the highest-value signals and pushing lower-value nudges into less interruptive channels or into times when the clinician's attention is less contested. And that nurse-practitioner finding? It's a reminder that "the clinician" isn't a single archetype. In this network, nurse practitioners accepted drug alerts at rates multiple times higher than physicians. The study didn't unpack why—training, workflow, risk tolerance, and scope of practice are all candidates—but it suggests that tailoring alert presentation by role could matter. This study also offers a small reframing. It says, "Don't chase total workload as the lever." Seeing more patients in a year didn't predict lower acceptance once you accounted for how the software delivered alerts. It's the local dose—the per-encounter density and the repetition—where the signal sits. If you're aiming to reduce alert fatigue, rebalancing that local dose seems more promising than trying to shave a percent off overall clinic volume. Zooming out, there's a nice symmetry here with how our brains handle signals in other domains. Pilots train with clear hierarchies of alarms because too many beeps at once can bury the one that matters. Your phone's lock screen sorts notifications for a reason. Clinical decision support is no different. When repetition and clutter rise, attention falls. Looking forward, it would be useful to layer severity and informativeness directly into these analyses—tag the life-saving alerts, separate the routine nudges, and see how the curves change. It would also help to test de-duplication strategies prospectively and measure not just acceptance, but downstream outcomes. But even without those, the shape of the current evidence is actionable. As Ancker and colleagues show, the core levers are close at hand. Trim the repeats. Control the per-encounter burst. Keep the high-signal alerts prominent. And don't assume clinicians slowly go numb to every new message; in this setting, they didn't. The problem wasn't time dulling the senses. It was too much noise in the moment when focus mattered most. This lecture was created by ennepō. Go to ennepo dot A I to Discover, Create and Follow the latest research in your field. Read when you can. Listen when you want to.