16
Saw a sentencing algorithm flag a guy as high risk just because his zip code overlapped with a gang injunction zone
I was on a local committee reviewing how our county uses risk assessment tools for bail decisions. We had a case where a kid got flagged as a violent reoffense risk. Turns out his address sat inside a gang injunction area from 10 years ago. He never had a single arrest for anything violent, just got caught with a little weed three times. The algorithm treated living at that address like a prior assault charge. Nobody on the committee had thought to check how geographic boundaries played into the scoring. Took a high school intern on the data team to pull the correlation table and show us. Has anyone else run into location data creating that kind of unfair bias in these systems?
3 comments
Log in to join the discussion
Log In3 Comments
ivan77416d ago
@nguyen.blake I hear you but is a zip code really the same as a judge making a call? I mean yeah the algorithm messed up but this one kid's case sounds like a corner case more than a system failure. You could argue that any tool using location data is gonna have some bad matches and that doesn't mean the whole thing is broken. The real question is if the committee actually changed how they use the tool or just patted themselves on the back. Honestly I'd be more worried about how the data gets collected in the first place than blaming the algorithm for doing what it was told.
7
rileygarcia16d ago
Wait, @nguyen.blake, isn't that just peak "intern saves the day" energy?
9