Hot take: algorithm bias is actually a people problem first

I used to think bias in lending algorithms came down to bad code or evil tech executives. But I saw something at my school’s career fair last month that changed my mind. A local credit union rep showed me how their loan approval model flagged more applicants from certain zip codes as high risk. I asked why and they admitted the training data came from old loan officer decisions from the 90s. The people making those calls back then had their own prejudices baked in. So the algorithm just learned what humans already did wrong. How do we fix a machine when the original source material is flawed from the start? Has anyone else found a way to clean up biased training data without starting from scratch?

3 comments

3 Comments

roberts.leo1mo ago

Funny enough my old bank used to do that too, lol.

jamie_webb671mo ago

My buddy's fintech team fed their algorithm 90s data and it flagged every woman as high risk.

evaw181mo ago

You know, I'm genuinely shocked that anyone would think feeding an algorithm data from the 90s would be a good idea, @jamie_webb67. That era had some real problems with lending practices and bias that are well documented now. It's a wonder the system didn't just spit out a list of names and addresses from the first "glass ceiling" article it found.