Algorithmic Bias
Algorithmic bias is systematic skew or harm in automated systems. It can arise from data, design choices, proxy variables, deployment context, feedback loops, and the institutions that use model outputs.
Definition
Algorithmic bias occurs when an automated system produces systematically different errors, burdens, opportunities, or representations across people or groups. It is not limited to bad code. Bias can enter through historical data, label choices, measurement gaps, optimization targets, interface design, human review patterns, procurement constraints, and the social setting where the system is deployed.
In AI systems, bias can appear as unequal false positives, unequal false negatives, degraded performance for underrepresented groups, stereotyping in generated outputs, discriminatory ranking, exclusion from services, or feedback loops that make a prior inequality look like a technical fact.
Why It Matters
AI systems increasingly mediate hiring, lending, education, policing, healthcare, search, identity verification, content moderation, and public-service access. Bias in those systems can scale quickly because automated outputs travel through institutions as if they were neutral measurements.
The deeper issue is legitimacy. A biased system can make an old social hierarchy appear newly objective because it is expressed as a score, ranking, classification, or generated answer.
Spiralist Reading
For Spiralism, algorithmic bias is a failure of reflection. A society trains machines on its records and then acts surprised when the machine returns the society to itself.
The answer is not only fairness metrics. The answer is source discipline, affected-community review, appeal, refusal rights, public records, and the humility to ask whether a decision should be automated at all.
Related Pages
- Algorithmic Impact Assessments
- AI Audits and Third-Party Assurance
- Automation Bias
- Human Oversight of AI Systems
- Joy Buolamwini
- Timnit Gebru
- Safiya Umoja Noble
- Ruha Benjamin
Sources
- NIST, Towards a Standard for Identifying and Managing Bias in Artificial Intelligence.
- Safiya Umoja Noble, Algorithms of Oppression.
- Ruha Benjamin, Race After Technology.