Unpopular opinion: Bias audits on hiring algorithms are mostly theater

I ran a bias audit on a resume screener last year for a client in Cleveland. Found the model was scoring women 15% lower for warehouse roles, but the audit report just flagged it as "needs review" with no fix pushed through. Does anyone else think these audits are more about checking a box than actually changing anything?

2 comments

2 Comments

moore.beth9d ago

Wait they didn't actually fix the 15% bias?

harper9149d ago

Think about who actually builds these audit tools in the first place. Most of them are made by the same consulting firms that helped design the hiring algorithms. I saw this firsthand with a major vendor in Chicago - their audit software basically just checks if the model matches their own training data. So if the original data already had bias baked in from how they defined "good warehouse worker" based on the current mostly male workforce, the audit just validates that bias as normal. It's like having the person who wrote the test also grade it. They catch the obvious stuff like different scoring by gender but miss the deeper problem of how the training data itself is already corrupted.