I thought the hiring algorithm at my old job was rigged, then I saw the 3 year data
I spent two years convinced the resume screening AI at my last company was quietly filtering out anyone over 50, mainly because my own dad got rejected from there six times. Last spring a former coworker sent me the internal audit from 2021 to 2024, and the rejection gap between age groups was under 2 percent, but the gap between zip codes was 31 percent. Has anyone else seen a bias study that pointed somewhere totally different than where you expected?
Did your old coworker say who ran the audit? A buddy of mine went through something like this at a warehouse chain. Everyone swore the screening tool was dinging people with gaps in their work history. Turns out it was mostly dinging people who lived more than 45 minutes away, because the system scored commute time as a "reliability risk." Folks in the next county over got cut at almost double the rate, and nobody in HR had any clue that field was even in the model. He only found out because a dev pulled the feature weights during a slow week. So yeah, the thing you're sure is the problem is often just the loudest guess.