AI bias is a systematic tendency in a system's results. Here we mean the harmful kind: treating a group of people worse or repeating a prejudice. It is not the machine's opinion: it comes from the data it learned from, how that data was labeled, what was chosen to measure and how the result is used.
A question from the app. Pick an answer.
An AI résumé filter rejects more women with the same profile. Why does it happen?
Learning from unequal past hiring can reproduce that inequality. How it was labeled, what it measures and how it is used also play a part.
It has no opinions: it repeats patterns from what it learned and how it was built. That is why bias gets measured and fixed, not argued with.
It computes with no intent, but it learns from human data and choices. If there is inequality there, the machine can repeat it.
From the data, mostly. A machine learning model learns patterns from its examples. If a speech recognizer was trained almost only on one accent, it understands others worse. If a résumé filter learned from past hiring where few women got in, it can learn that "good candidate" looks like a man and reproduce that inequality.
But not only from there. Labels are added by people, with their own criteria. What you choose to measure matters too: if you judge who needs more medical care by how much they spent on health before, people who had less access look like they need less. And how it is used: a model built for one city can fail in another.
Removing gender or origin from the data looks like the fix, but the model can find other clues that say almost the same thing. A zip code often gives away the neighborhood, and the neighborhood gives away origin or income. A gap of years in a career, the name of a school or a sport can also stand in for the column you deleted.
By measuring results per group, not just the average. A model can be 95% right overall and much more wrong with a small group. You compare how many it approves and how often it errs for each group. Evals with matched cases help too: several equivalent résumés, changing only the name, and repeating the test. If the result changes systematically by group, there is bias.
"A machine is neutral." It computes with no intent, but it learns from human decisions. "More data fixes it." More data of the same kind repeats the same bias. "Bias and hallucination are the same." A hallucination is making up a fact in one answer; bias is a pattern that keeps going against a group, even when each answer looks reasonable.