Subscribe Now

Judicature

PUBLISHED BY:
Bolch Judicial Institute
Duke Law School

|

A Primer on the Different Meanings of AI ‘Bias’ for Legal Practice

by and

Vol. 109 No. 3 (2026) | To Know Courts Is to Love Them? | Download PDF Version of Article

Not All Bias in Artificial Intelligence (AI) Is Necessarily Harmful — and Some Is Even Necessary. Understanding the Different Meanings of “Bias” Is Key to Spotting When AI Systems Produce Results That Are Inaccurate, Unfair, and/or Unlawful.

Artificial intelligence (AI) systems not only contain, but often depend on, “bias.”26 This might seem like a provocative statement to legal professionals, who are particularly sensitive to the risk of illegal discrimination in human and AI decision-making. But, depending on the context, bias in an AI system may also be a good thing, and even essential to achieving a desired result.

Bias carries different meanings in different contexts. According to the Merriam-Webster dictionary, common definitions of bias include (i) “an inclination of temperament or outlook,” (ii) “an instance of such prejudice,” (iii) “bent, tendency,” and (iv) statistical deviations and errors.27 These definitions reflect that the term bias can be used to refer to a range of characteristics from a mere tendency to blatantly unfair outcomes, including aspects that may be technical or statistical in nature. In the AI context, bias can refer to statistical or technical bias (i.e., mathematical error) in AI models themselves, discrepancies in the data used to train such models, distortions in human cognition, and factors related to systemic unfairness.

Sometimes, reality, or the correct prediction, may be different from statistical results, and the output of the AI system is incorrect — such as with large language models that “hallucinate” (i.e., generate fabricated or faulty information). Sometimes, the output may be unfair or illegal, because it skews results against a vulnerable, legally protected group, reflecting or amplifying societal inequities. The risk of problematic bias can begin even before data are collected, by how a problem is initially framed or what objective an AI system is designed to achieve. If the underlying goals or proxies chosen by AI developers reflect existing societal inequities, the resulting system may perpetuate them. But bias can also simply refer to a system’s tendency to produce a particular type of output, such as the sense in which a GPS system is biased to direct the traveler to take a particular route, or a technology-assisted review system is biased to find responsive documents as opposed to nonresponsive documents.

As AI becomes more integrated into many aspects of the legal system, attorneys and judges need to understand and distinguish between the different types of bias and learn how to mitigate those that are harmful. To that end, this article seeks to provide clarity, helping lawyers and judges to distinguish between proper and useful bias, statistical bias as a quantitative matter, and genuine issues of discriminatory bias, when evaluating AI systems. This foundation is essential for facilitating communication between technologists and legal professionals, mitigating harmful bias, and developing meaningful governance approaches that promote both functionality and fairness.

Positive-Tendency Bias

AI systems select outputs that are statistically likely to be correct based on their training and the resulting models. This “positive-tendency” bias, as we call it, is a self-evident bias or tendency toward a particular, intended outcome. Much of the time, it is a productive process, resulting in reasonably correct and reasonably fair results.28

For example, autocorrect systems recommend statistically likely outcomes. Readers are likely very familiar with both the potential helpfulness as well as the imperfections of autocorrect’s recommendations. As a general proposition, recommendations tend to be useful because they are weighted toward common words and standard spellings. Without these weightings, autocorrect would treat “teh” as equally valid as “the,” and “definately” as equally correct to “definitely.” While these systems occasionally and frustratingly lead to errors — sometimes, even embarrassing ones — users accept and even rely on these imperfect biases because they serve a clear purpose: to improve productivity and communication accuracy most of the time.

Similarly, this type of positive-tendency bias fuels common, current uses of generative AI in legal practice, including the results when practitioners prompt a chatbot to generate a contractual clause, a caselaw search engine to retrieve caselaw on a particular topic, and a discovery tool to review documents for responsiveness to a request for production. In each of these instances, a model, directed by a prompt, will produce an output that it generates based on weights that the model calculates are most probably correct. If these AI systems treated all words for its output as equally likely, they would spew out random words instead of helpful contractual clauses, unrelated cases instead of on-point caselaw, and useless predictions about document responsiveness instead of reasonably correct ones. So, when “bias” refers to a positive tendency toward certain results and away from others, such bias enables the AI system to fulfill its intended purpose.29 This is the intended and desirable application of bias.

[W]hen “bias” refers to a positive tendency toward certain results and away from others, such bias enables the AI system to fulfill its intended purpose. This is the intended and desirable application of bias.

Statistical or Mathematical Bias

Bias in AI systems can also refer to statistical or mathematical30 differences between an AI system’s model and its outputs, compared to the real-world data it analyzes or for which its outputs are used. These deviations can occur during AI systems’ development and training processes, including data collection, processing, or modeling. These biases are not inherently prejudicial but impact an AI system’s performance, and can, at times, lead to discriminatory outcomes.31 While these biases can occur in many ways, we provide a brief overview of certain processes known to commonly cause such issues: (a) selection bias, (b) data-quality or measurement bias, (c) labeling or annotation bias, (d) overfitting and underfitting, and (e) temporal drift.32 This list is not exhaustive, and different disciplines may use varying terminology to describe similar bias concepts. Note that it is nearly impossible to build an AI model without some degree of statistical or mathematical bias,33 but AI developers should generally seek to minimize the detrimental impact of such biases, especially when they may lead to discriminatory outcomes.

Selection Bias

Selection bias occurs when the training data used to develop an AI model does not adequately represent the variety of use cases or data for which the AI system is to be used.34 For example, an AI tool that is trained to predict litigation outcomes based on cases from certain jurisdictions may perform well for those jurisdictions. But, if applied to other jurisdictions, the same tool may not perform well because its training does not account for the other jurisdictions’ different legal landscapes, such as caselaw precedent, procedural rules, and judicial tendencies. The mismatch between the data used for training and the more diverse scenarios when applied to a broad selection of cases will undermine the tool’s validity and reliability.

Data Quality or Measurement Bias

Raw data collected from the real world is famously messy; it contains gaps, inconsistencies, and errors. Because AI systems are generally trained on large amounts of data, however, such inaccuracies tend to run in different directions, effectively canceling each other out. They are therefore generally unlikely to create significant problems in a system’s AI model unless they are extreme or systematic. Data-quality bias, or measurement bias, occurs when significant amounts of training data are skewed in a particular direction, causing that bias to be incorporated into the AI system.35 For example, if historic sensor data from manufacturing machinery is used to predict future maintenance needs but the sensors used to train the system were not correctly calibrated, the AI model will incorporate the skewed data and will likely result in incorrect outputs.

Labeling or Annotation Bias

In supervised learning, human-applied labels, or annotations, provide the ground truth for training and evaluating AI models. Labeling bias, also known as annotation bias, arises when inconsistencies or errors occur in the labeling process, which may result from the mistakes or subjectivities of those humans applying the labels.36 For example, attorneys conducting reviews for discovery may tag documents as responsive or nonresponsive based on various incentives and inclinations. When junior or contract attorneys are instructed to review batches of documents that are almost entirely nonresponsive, they consistently tend to label some of the documents as responsive because their inherent biases make them feel that if they do not find any responsive documents, they are not doing their jobs properly, and it is likely that at least some of the documents must be responsive.37

When junior or contract attorneys are instructed to review batches of documents that are almost entirely nonresponsive, they consistently tend to label some of the documents as responsive because their inherent biases make them feel that if they do not find any responsive documents, they are not doing their jobs properly.

Overfitting and Underfitting

“Overfitting” and “underfitting” refer to how well a model uses its training data to produce quality outputs. A model with overfitting may learn the training data too well, including its noise and idiosyncrasies, instead of focusing on more generalizable patterns. In this case, the model works well on the training data but will not work well on other, dissimilar data sets. Conversely, an underfit model is too simplistic and fails to capture important information and patterns from the training data, leading to poor performance on both training and real-world data. In these cases, the model may be biased toward training data if overfit, or ineffective on training and real-world data if underfit (i.e., a poor positive-tendency bias).38 For example, in a financial credit evaluation system, an overfit system may perform poorly for any credit applicant who does not closely resemble the subjects in the training data. An underfit system might make overly broad, inaccurate generalizations, perhaps unfairly denying credit based on crude categorizations.

Temporal Drift

“Temporal drift” refers to changes over time that cause AI systems to become less accurate or reliable because the patterns the system learned from past data may no longer apply in current or future situations.39 This can occur when the patterns of incoming data shift while the underlying relationships do not (“data drift”), or the underlying relationships between different factors change over time (“concept drift”).

Example 1 (Data drift): A store’s AI model that predicts customer demand based on historic purchasing patterns may become less accurate if customer demographics shift over time, such as an aging population or changing income levels. The underlying relationship between customer characteristics and buying behavior did not change, but the model becomes inaccurate because it was trained on data patterns related to different customers.

Example 2 (Concept drift): The same type of model may become less accurate if customer shopping behaviors change over time, such as increased online shopping or changing preferences for which products customers want to buy. When the underlying relationships between past and present patterns change, the model becomes inaccurate even when applied to data about the same types of customers as before.

Bias as a Technical Term in Data Science

In the context of data science, bias has yet another meaning. Machine-learning systems involve “weights” and “biases.” Used technically, weights are preferences for certain patterns learned from training data. Those weights can reflect preferences that may advantage or disadvantage certain groups. In this context, bias is a technical term that serves a different mathematical function in the model than weights. It is used to calibrate the system; in effect, it determines the thresholds to use to measure the meaningfulness of the weights. So, when lawyers and policymakers discuss a biased AI system, they are referring to the undesirable effects of the weights, and not this alternative, technical meaning of bias.

Discriminatory Bias

Legal frameworks to assess and prevent discriminatory bias40 among humans have developed over decades of civil rights litigation and regulatory enforcement. Courts recognize two primary forms of legally impermissible or discriminatory bias: “disparate treatment,” in which decisions intentionally disadvantage legally protected groups, and “disparate impact,” in which facially neutral practices create discriminatory effects.41 These frameworks currently address traditional forms of bias, in which human prejudice or the perpetuation of historical inequities are alleged to produce unfair outcomes.

AI systems can be prone to discriminatory bias, too, which can arise at different points throughout the lifecycle of their development, deployment, and use.42 This includes initial problem formulation (such as defining objectives), the selection and quality of training data (which may contain historical biases or lack representation, leading to selection or data-quality biases), choices made during feature engineering (such as choosing proxies that embed societal biases), the selection of algorithms, content within the training data, how systems are evaluated (or not evaluated) for fairness across subgroups, and how the systems are used and their outputs are interpreted.43 Given the increasing role of AI in decision-making processes, the risk of AI systems containing and causing discriminatory bias has set off global alarms.

Existing anti-discrimination laws, including employment regulations, fair lending laws, and various state-level requirements, already apply to AI decision-making systems.44 Additional legal regulations and frameworks are also emerging to identify and address unfair, discriminatory bias in AI decision-making systems.45 But discriminatory bias can contribute to and result from discriminatory AI systems in several ways. As we discuss next, attorneys need to have clarity on the different types of bias and their technical foundations, as well as how they can interrelate to create discriminatory outcomes.

Key Distinctions Among Different Types of Bias

It is essential for legal professionals and AI developers to distinguish among “positive-tendency bias,” which refers to a tendency toward a specific, intended outcome; “statistical bias,” which refers to inaccuracies or inefficiencies in the model or data; and “discriminatory bias,” which involves harmful or unfair treatment of protected groups (e.g., racial, gender, or socioeconomic biases).

It is essential for legal professionals and AI developers to distinguish among “positive-tendency bias,” which refers to a tendency toward a specific, intended outcome; “statistical bias,” which refers to inaccuracies or inefficiencies in the model or data; and “discriminatory bias,” which involves harmful or unfair treatment of protected groups (e.g., racial, gender, or socioeconomic biases).

While addressing statistical bias is crucial for improving model performance, it does not automatically resolve issues of discriminatory bias. However, the two are often related, as statistical biases can be the technical mechanism through which discriminatory outcomes are produced or amplified. While all discriminatory bias is necessarily statistical bias, not all statistical bias is necessarily discriminatory.

The distinction between positive-tendency bias and discriminatory bias can be demonstrated with two scenarios involving weighted dice.

  1. The Cheating Gambler. A gambler secretly uses a die that is weighted to land on six. It does not land on six every time, but it does so often enough that the gambler is more likely to win their game than other players using ordinary dice.
  2. The Classroom Challenge. A teacher announces a classroom challenge in which the students, as a group, will receive a higher grade based on how often they roll sixes on a die. They are permitted to use any creative solution they devise to maximize their performance. The students work together to create a weighted die (identical to the gambler’s die) that increases the probability of rolling sixes.

We may use the term bias in different ways to describe these scenarios. In both scenarios, the die was biased to land on six. In the first scenario, however, the bias in the weighted die creates an unfair advantage for one player, violating the other players’ assumption of random probability for each player’s throw of a die. In an AI system, similar discriminatory bias can exist as hidden tendencies toward results that create unfair advantages for some groups over others, often stemming from the statistical biases discussed above.

In the second scenario, the die was not biased against any particular person or group. This weighted die served a legitimate purpose that increased the chances of the students getting a “correct” answer of a six, which would earn good grades for the entire class. The classroom die might be identical to the gambler’s weighted die but it does not create an unfair outcome; indeed, it was the intended goal of the effort. The key distinction lies not in the fact that a die or AI system has a preference but in its purpose, transparency, and alignment with legitimate goals.

To prevent tendency bias from creating discriminatory bias, the real-world uses of AI systems can be as important as how they are built. The weighted classroom die served a legitimate purpose when it was used as designed, because it involved full transparency between the teacher and students, and alignment with agreed-upon goals. The same die, secretly used in a gambling game where others expect randomness, becomes problematic and unfair based on lack of transparency and alignment with its intended purpose.46

Tendency Bias, Discriminatory Bias, and De-Biasing

De-biasing strategies to improve fairness generally fall into one of three categories of techniques:47

  1. Pre-processing: cleaning and adjusting the training data to remove biases before the AI system learns from it (e.g., balancing representation of different groups, adjusting data weights, removing features that may cause unfair discrimination);
  2. In-processing: teaching the AI system to consider fairness while it learns, so that fair decision-making becomes part of how the system is trained to operate; or
  3. Post-processing: adjusting the AI system’s outputs or decisions to ensure fairer outcomes across different groups in real-world applications.48

However, bias cannot simply be removed from a working AI system, given that tendency bias is essential for the system to operate efficiently. But tendency bias can also encode and perpetuate problematic social bias. The weights that make AI systems functional can, without careful examination and intervention, become vectors for historical discrimination.49

This presents a complex challenge for development and deployment of AI systems: how to preserve necessary technical functioning while preventing discriminatory effects. Because discriminatory bias is a type of tendency bias, it is also interrelated with other tendency biases. Unwanted bias might be mitigable, but trying to correct an unwanted tendency of an AI system can be like trying to solve a Rubik’s cube, where focusing on solving one puzzle face may disrupt patterns on the others.

Therefore, calls to de-bias AI systems should involve careful analysis of the specific problematic bias at issue, why it may exist, and both technical and nontechnical options to mitigate it. De-biasing of AI systems may take place in several ways, such as updating training data to be more representative of the groups on which the AI system will be used, updating weights inside the system, updating how the AI system uses data or develops output, education of users of the system on how to interpret the data or output, and more.

For example, among several approaches that may be employed to counter bias in AI systems, one involves applying model weight adjustments to offset identified disparities. Where an AI system consistently ranks one demographic group lower than others, developers may make a corresponding adjustment to model weights relating to characteristics of the disadvantaged group. While that adjustment may address the specific concern, the nature and amount of the correction will reflect developers’ assumptions. Those assumptions, in turn, may introduce new biases or unintended system interactions, potentially in ways that “just hide the bias and don’t actually remove it.”50 AI systems’ interrelated components can mean that efforts to solve one bias problem may create or exacerbate others.

Conclusion

Understanding bias in AI systems requires a common vocabulary. Bias has many meanings, including positive-tendency bias, the various forms of statistical or mathematical bias (including selection, data-quality, labeling, overfitting/underfitting, and temporal drift), and harmful discriminatory bias. Discriminatory bias emerges from systems built on statistical biases. As AI systems increasingly influence critical decisions, lawyers and judges must bridge technical and ethical considerations to effectively address the problems of discriminatory bias. When addressing topics of AI governance, procurement, due diligence, and litigation, understanding the different meanings of the term bias and how they interrelate is essential in developing effective legal and policy frameworks.

TARA S. EMORY is principal of Aligned Discovery PLLC and a court-appointed neutral specializing in e-discovery and legal technology. She is Chambers-ranked, a fellow of the Academy of Court-Appointed Neutrals, and serves on The Sedona Conference AI Working Group Steering Committee.

MAURA R. GROSSMAN, JD, PhD, is a research professor at the University of Waterloo and an adjunct professor at Osgoode Hall Law School, both in Ontario, Canada. She is also principal at Maura Grossman Law in Buffalo, N.Y., where she serves as a consultant on legal technology and as a special master.

  1. The authors wish to acknowledge the helpful comments received from Patrick Hall on a draft of this article. The views and opinions expressed are solely those of the authors and do not necessarily reflect the policy or positions of any organizations or clients with which the authors may be affiliated.
  2. Bias, Merriam-Webster dictionary, https://duke.is/merriam-webster-bias (last visited Feb. 26, 2026). Cf. Bias, Int'l Org. for Standardization § 3.2 (2022), https://duke.is/OBP-bias-definition (defining bias as "systematic difference in treatment of certain objects, people or groups in comparison to others").
  3. See Tom M. Mitchell, The Need for Biases in Learning Generalizations (Tech. Rep. No. CBM-TR-117), Rutgers Univ. (1980), https://duke.is/mitchell (foundational work explaining that machine-learning algorithms need built-in assumptions or "biases" to make reasonable predictions).
  4. See id.
  5. The terms "statistical" and "mathematical" are used interchangeably to describe this bias.
  6. Ninareh Mehrabi et al., A Survey on Bias and Fairness in Machine Learning, 54 ACM Computing Surveys 1, 4 (July 2021) ("[E]xisting biases in data can affect the algorithms using the data, producing biased outcomes. Algorithms can even amplify and perpetuate existing biases in the data.").
  7. Types of bias related to AI can be characterized in different ways. For a comprehensive list of biases and their interdisciplinary characterizations, see id. at 6–9; see also Reva Schwartz et al., Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, Nat'l Inst. of Standards & Tech. Special Publication 1270, 6–9 (Mar. 2022).
  8. See Schwartz et al., supra note 7, at 3 (noting the various intentional and unintentional ways bias affects AI systems).
  9. Harini Suresh & John Guttag, A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle, Equity and Access in Algorithms, Mechanisms, and Optimization § 3.2 (2021); see also Mehrabi et al., supra note 6, at 6 ("As a consequence of sampling bias, the trends estimated for one population may not generalize to data collected from a new population.").
  10. See Mehrabi et al., supra note 6, at 4–5 (describing measurement bias as "aris[ing] from how we choose, utilize, and measure particular features").
  11. See Luke Haliburton et al., Uncovering Labeler Bias in Machine Learning Annotation Tasks, 5 AI & Ethics 2515, 2515 (2025) (defining labeler bias as "any bias embedded by labelers when annotating data").
  12. Adam Roegiest & Gordon V. Cormack, Impact of Review-Set Selection on Human Assessment for Text Classification, in SIGIR '16: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval 861, 861–64 (2016).
  13. See Constantin Aliferis & Gyorgy J. Simon, Overfitting, Underfitting and General Model Overconfidence and Under-Performance Pitfalls and Best Practices in Machine Learning in AI, in Artificial Intelligence and Machine Learning in Health Care and Medical Sciences: Best Practices and Pitfalls 482 (Gyorgy J. Simon & Constantin Aliferis eds., 2024) (describing the issues associated with overfitting and underfitting).
  14. Suresh & Guttag, supra note 9, at § 3.1 ("Historical bias arises even if data is perfectly measured and sampled, if the world as it is or was leads to a model that produces harmful outcomes.").
  15. Discriminatory bias refers to unfair bias that discriminates against underrepresented, underprivileged, and protected groups of people. It is different from discriminative algorithms, which refer to a type of traditional (not generative) AI that can classify items that belong to one category or another. See Solon Barocas & Andrew D. Selbst, Big Data's Disparate Impact 104 Calif. L. Rev. 671 (2016); see also U.S. Equal Emp't Opportunity Comm'n, Consumer Fin. Prot. Bureau, U.S. Dep't of Justice Civil Rights Division & Fed. Trade Comm'n, Joint Statement on Enforcement Efforts Against Discrimination and Bias in Automated Systems (Apr. 25, 2023), available at https://duke.is/EEOCdata.
  16. See Griggs v. Duke Power Co., 401 U.S. 424, 430–31 (1971).
  17. Mehrabi et al., supra note 6, at 3–9.
  18. Id.
  19. See generally Resetting Antidiscrimination Law in the Age of AI, 138 Harv. L. Rev. 1562 (2025) (describing the application of existing anti-discrimination laws to AI systems and evaluating various legislative proposals to address issues arising from the development of AI).
  20. Emerging laws and frameworks include the European Union Artificial Intelligence Act, Council Regulation 2024/1689, 2024 O.J. (L 1689) (requiring high-risk systems to have documentation, including instructions for use and human oversight, and to be designed for accuracy and security); Nat'l Inst. of Standards & Tech., Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 12–19 (Jan. 2023), https://duke.is/NISTAI (stating that AI systems should be valid and reliable, safe, secure, accountable and transparent, explainable and interpretable, privacy-protecting, and fair); All. for Trust in AI, Principles, https://duke.is/AIprinciples (last visited Feb. 26, 2026) (recommending AI guardrails involve transparency, human oversight, privacy, fairness, accuracy, robustness, and validity).
  21. In a highly publicized case of algorithmic discrimination, the COMPAS recidivism-prediction system relied on certain factors considered highly correlated with recidivism, including race and gender. However, the system had learned from decades of discriminatory law enforcement practices. The results, while technically performing their intended function of pattern recognition, encoded historical discriminatory biases into future predictions. See Jeff Larson et al., How We Analyzed the COMPAS Recidivism Algorithm, ProPublica (May 23, 2016), https://duke.is/COMPASalgorithm; but see Cynthia Rudin et al., The Age of Secrecy and Unfairness in Recidivism Prediction, 2 Harv. Data Sci. Rev. 1 (2020) (noting ProPublica's analysis was limited by the opaque nature of COMPAS and may not have accounted for data relationships between race and age). For additional examples of AI bias incidents, see AI Incident Database, Partnership on AI, https://duke.is/aincidentdatabase.
  22. Simon Caton & Christian Haas, Fairness in Machine Learning: A Survey, 56 ACM Computing Surveys 1, 3 (2024); Mehrabi et al., supra note 6, at 13–14.
  23. Note that in regulated industries such as lending and employment, the process of analyzing and potentially correcting disparities, even with an intent to reduce bias or improve fairness, may conflict with legal principles regarding discrimination. See Ricci v. DeStefano, 557 U.S. 557 (2009) (establishing limitations on race-conscious remedial measures). Alternative approaches may involve adjusting probability thresholds or modifying input feature sets without directly addressing protected demographic characteristics.
  24. Schwartz et al., supra note 7, at ii. ("[I]t is not possible to achieve zero risk of bias in an AI system.").
  25. Mehrabi et al., supra note 6, at 21.
  26. The authors wish to acknowledge the helpful comments received from Patrick Hall on a draft of this article. The views and opinions expressed are solely those of the authors and do not necessarily reflect the policy or positions of any organizations or clients with which the authors may be affiliated.
  27. Bias, Merriam-Webster dictionary, https://duke.is/merriam-webster-bias (last visited Feb. 26, 2026). Cf. Bias, Int’l Org. for Standardization § 3.2 (2022), https://duke.is/OBP-bias-definition (defining bias as “systematic difference in treatment of certain objects, people or groups in comparison to others”).
  28. See Tom M. Mitchell, The Need for Biases in Learning Generalizations (Tech. Rep. No. CBM-TR-117), Rutgers Univ. (1980), https://duke.is/mitchell (foundational work explaining that machine-learning algorithms need built-in assumptions or “biases” to make reasonable predictions).
  29. See id.
  30. The terms “statistical” and “mathematical” are used interchangeably to describe this bias.
  31. Ninareh Mehrabi et al., A Survey on Bias and Fairness in Machine Learning, 54 ACM Computing Surveys 1, 4 (July 2021) (“[E]xisting biases in data can affect the algorithms using the data, producing biased outcomes. Algorithms can even amplify and perpetuate existing biases in the data.”).
  32. Types of bias related to AI can be characterized in different ways. For a comprehensive list of biases and their interdisciplinary characterizations, see id. at 6–9; see also Reva Schwartz et al., Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, Nat’l Inst. of Standards & Tech. Special Publication 1270, 6–9 (Mar. 2022).
  33. See Schwartz et al., supra note 7, at 3 (noting the various intentional and unintentional ways bias affects AI systems).
  34. Harini Suresh & John Guttag, A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle, Equity and Access in Algorithms, Mechanisms, and Optimization § 3.2 (2021); see also Mehrabi et al., supra note 6, at 6 (“As a consequence of sampling bias, the trends estimated for one population may not generalize to data collected from a new population.”).
  35. See Mehrabi et al., supra note 6, at 4–5 (describing measurement bias as “aris[ing] from how we choose, utilize, and measure particular features”).
  36. See Luke Haliburton et al., Uncovering Labeler Bias in Machine Learning Annotation Tasks, 5 AI & Ethics 2515, 2515 (2025) (defining labeler bias as “any bias embedded by labelers when annotating data”).
  37. Adam Roegiest & Gordon V. Cormack, Impact of Review-Set Selection on Human Assessment for Text Classification, in SIGIR ’16: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval 861, 861–64 (2016).
  38. See Constantin Aliferis & Gyorgy J. Simon, Overfitting, Underfitting and General Model Overconfidence and Under-Performance Pitfalls and Best Practices in Machine Learning in AI, in Artificial Intelligence and Machine Learning in Health Care and Medical Sciences: Best Practices and Pitfalls 482 (Gyorgy J. Simon & Constantin Aliferis eds., 2024) (describing the issues associated with overfitting and underfitting).
  39. Suresh & Guttag, supra note 9, at § 3.1 (“Historical bias arises even if data is perfectly measured and sampled, if the world as it is or was leads to a model that produces harmful outcomes.”).
  40. Discriminatory bias refers to unfair bias that discriminates against underrepresented, underprivileged, and protected groups of people. It is different from discriminative algorithms, which refer to a type of traditional (not generative) AI that can classify items that belong to one category or another. See Solon Barocas & Andrew D. Selbst, Big Data’s Disparate Impact 104 Calif. L. Rev. 671 (2016); see also U.S. Equal Emp’t Opportunity Comm’n, Consumer Fin. Prot. Bureau, U.S. Dep’t of Justice Civil Rights Division & Fed. Trade Comm’n, Joint Statement on Enforcement Efforts Against Discrimination and Bias in Automated Systems (Apr. 25, 2023), available at https://duke.is/EEOCdata.
  41. See Griggs v. Duke Power Co., 401 U.S. 424, 430–31 (1971).
  42. Mehrabi et al., supra note 6, at 3–9.
  43. Id.
  44. See generally Resetting Antidiscrimination Law in the Age of AI, 138 Harv. L. Rev. 1562 (2025) (describing the application of existing anti-discrimination laws to AI systems and evaluating various legislative proposals to address issues arising from the development of AI).
  45. Emerging laws and frameworks include the European Union Artificial Intelligence Act, Council Regulation 2024/1689, 2024 O.J. (L 1689) (requiring high-risk systems to have documentation, including instructions for use and human oversight, and to be designed for accuracy and security); Nat’l Inst. of Standards & Tech., Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 12–19 (Jan. 2023), https://duke.is/NISTAI (stating that AI systems should be valid and reliable, safe, secure, accountable and transparent, explainable and interpretable, privacy-protecting, and fair); All. for Trust in AI, Principles, https://duke.is/AIprinciples (last visited Feb. 26, 2026) (recommending AI guardrails involve transparency, human oversight, privacy, fairness, accuracy, robustness, and validity).
  46. In a highly publicized case of algorithmic discrimination, the COMPAS recidivism-prediction system relied on certain factors considered highly correlated with recidivism, including race and gender. However, the system had learned from decades of discriminatory law enforcement practices. The results, while technically performing their intended function of pattern recognition, encoded historical discriminatory biases into future predictions. See Jeff Larson et al., How We Analyzed the COMPAS Recidivism Algorithm, ProPublica (May 23, 2016), https://duke.is/COMPASalgorithm; but see Cynthia Rudin et al., The Age of Secrecy and Unfairness in Recidivism Prediction, 2 Harv. Data Sci. Rev. 1 (2020) (noting ProPublica’s analysis was limited by the opaque nature of COMPAS and may not have accounted for data relationships between race and age). For additional examples of AI bias incidents, see AI Incident Database, Partnership on AI, https://duke.is/aincidentdatabase.
  47. Simon Caton & Christian Haas, Fairness in Machine Learning: A Survey, 56 ACM Computing Surveys 1, 3 (2024); Mehrabi et al., supra note 6, at 13–14.
  48. Note that in regulated industries such as lending and employment, the process of analyzing and potentially correcting disparities, even with an intent to reduce bias or improve fairness, may conflict with legal principles regarding discrimination. See Ricci v. DeStefano, 557 U.S. 557 (2009) (establishing limitations on race-conscious remedial measures). Alternative approaches may involve adjusting probability thresholds or modifying input feature sets without directly addressing protected demographic characteristics.
  49. Schwartz et al., supra note 7, at ii. (“[I]t is not possible to achieve zero risk of bias in an AI system.”).
  50. Mehrabi et al., supra note 6, at 21.

Also In This Edition