10
Worst week of my life - an AI triage system misread my patient's symptoms
Last Tuesday night, I was on shift when the hospital's new AI triage tool flagged a chest pain case as low priority because the patient's chart said they were 'anxious'. The algorithm ignored their actual complaint of crushing pressure because it matched a pattern from old ER notes. I argued with the charge nurse for 15 minutes before they let me override the system and call the doctor. Turned out it was a massive heart attack - the AI killed 2 hours of precious time. Has anyone else had a near-miss because a machine decided your patient wasn't sick enough?
3 comments
Log in to join the discussion
Log In3 Comments
aaron74011d ago
Yeah that chest pain thing is scary because AI just sees patterns not people. Had a similar thing happen with a kid who had a fever and stiff neck, AI slapped "viral syndrome" on it based on some checkbox from a previous visit. Nurse caught it quick but still lost an hour waiting on the override. The problem is these systems get trained on past notes which already have bias baked in. If a patient got labeled "anxious" once, the AI treats everything after that as less serious.
4
the_olivia11d agoTop Commenter
Nah I gotta disagree with the "bias baked in" part. The whole point of AI is that it's SUPPOSED to learn from patterns, and if a patient keeps coming in with the same vague stuff labeled as anxiety, maybe the system is actually right more often than not. You're acting like that nurse finding the exception proves the rule but odds are the AI catches tons of real cases that humans miss. Every time we override a correct AI call just because it feels wrong we're wasting time that could go to someone who actually needs it.
3
rowanhernandez11d ago
Exactly this. @aaron740 hit the nail on the head about how old notes poison the data. The system learns that 'anxious' equals 'not urgent' but misses the actual symptoms hiding underneath. That bias just keeps reinforcing itself every time the AI gets to make a call, making it harder for real cases to get caught until it's too late.
4