Algorithmic Justice: Legal Challenges of AI Bias and The Right to Equality

“Equality Should Be a Right, Not an Algorithmic Accident.” Artificial Intelligence (AI) has become increasingly prominent in 21st century decisions across core sectors such as finance, work, health care, law enforcement and the courts. While AI has the potential for efficiency, accuracy, and scale, AI bases its operations on human choices and biases, constitutively undermining equality rights in our Constitution. Biased hiring algorithms, racially discriminatory credit scoring models that have emerged out of FinTech, and predictive policing software that targets unfairly are all examples of how “neutral” coding May in fact, perpetuate and exacerbate inequality. This article examines the legal context of algorithmic bias and its implications on Article 14 equality in the Constitution of India and international human rights law, while framing the discussion around algorithmic bias. Ultimately this article advances the conclusion that for equality rights to be maintained there must be a new legal structure embedded with accountability, fairness and transparency into AI systems.

INTRODUCTION TO ALGORITHMIC BIAS

Algorithmic bias refers to patterns of repeatable and systematic error in AI systems where the AI has fair or discriminatory outcomes for groups of people and unfair outcomes for others.[2] In contrast to human bias, which is either purposeful or implicit human judgement, algorithmic bias is derived either from subjects or processes related to the design, data, or the specifics of the deployment of machine learning models. Given the nature of algorithmic bias, the bias is not as obvious as the bias of a human being and thus it is not always taken seriously. Further complicating the bias of AI systems is the tendency to view these systems as tools devoid of bias because they are relying on mathematical computations instead of the inherent subjectivity and discretion of a human being. This assumption is false. When historical patterns of discrimination are included in training data, or when informed by one group of individuals, AI models will produce biased results. Lastly, today AI model outputs are not applicable to under-represented populations subsequently producing biased systemic outcomes once again. Algorithmic bias poses a serious threat to the ideals of fairness, equity, and ultimately, the Right to Equality. Furthermore, algorithmic bias puts at risk as a broader societal impact, the trust and confidence society puts in algorithmic decision-making to make individual decisions.

LEGAL FRAMEWORK AND CONSTITUTIONAL PROTECTIONS

Despite being a game-changer in the efficiency and scalability AI technology brings to industries such as FinTech, employment, and law enforcement, it poses unique challenges to the right to equality. Having negative, biased outcomes toward disenfranchised communities, algorithmic bias threatens equity and fundamental rights most perilously in AI-driven financial services. The Indian Constitution in Article 14 provides for the right to equality for each individual throughout the territory of India.[3] This also includes the concept of non-arbitrariness. Equal treatment for all individuals in equal situations.[4] In the judiciary’s interpretation of Article 14, two types of equality must be taken into account. Firstly, the equality of the law is the same for all, which is termed as ‘formal equality’, and the other, the law considering individual situations to be equal, which is termed ‘substantive equality’. [5]Article 14 is quite significant in the world of AI-driven FinTech. Credit scoring algorithms and digital lending system which are biased and discriminatory triggers the equality violation of the Constitution and economically, gender, and socio-region biased discrimination against certain communities.[6] Courts are aware of the need for a mandate of non-discrimination in public and private sectors to address the constitutional constraints of contemporary technology.

The legal and regulatory framework for algorithmic justice in fintech, specifically the issues of AI bias and the right to equality, is rapidly becoming one of the more challenging issues for legislatures and courts around the world. Discrimination on specified grounds, such as religion, caste, sex and place of birth are prohibited by Article 15. Article 21 secures a person’s right to life and personal liberty and the right to life and personal liberty was held by the Supreme Court of India in Justice K.S. Puttaswamy v. Union of India (2017) to also include the right to privacy.[7] Therefore, any fintech systems that process the personal and financial data of persons should be subject to fair, just and reasonable standards and provide assurances of transparency and non-arbitrariness in their analysis. Article 7 of the Universal Declaration of Human Rights (UDHR) and the regulatory environment of algorithmic justice within fintech is one of the convergences of equality, data protection and financial regulation. In particular, Article 26 of the ICCPR currently states that, “Everyone is entitled to equal protection of the law against any discrimination.”[8]In India, both the Digital Personal Data Protection Act, 2023, and the Reserve Bank of India’s Master Directions on Digital Lending (2022) are designed to promote algorithmic equality, accountability and transparency in automated digital credit scoring.[9] At the global level, the General Data Protection Regulation (GDPR) of the European Union prohibits decisions solely on the basis of automated processing as stated in Article 22.[10] Likewise, the EU AI Act (2024) is currently regulating credit scoring AI as one type of “high-risk” which would include prescribed audit and transparency obligations. In the United States, and in the context of fintech, laws regulating bias include the Equal Credit Opportunity Act (1974) and the Fair Housing Act (1968),[11] with court cases reinforcing the legislative protection such as Bhopal Sugar Industries (1964), E.P. Royappa (1974) and Maneka Gandhi (1978) in India having been interpreted as establishing principles of fairness and equality in the context of administrative decision-making at every stage. In the EU in particular, the General Data Protection Regulation (GDPR) 2018, combined with article 22 of the GDPR, restricts decisions made solely based on automated processing with legal or other significant effects to identifiable individuals, thereby requiring transparency and eventually, human involvement.[12]

CONCLUSION AND WAY FORWARD

A strong framework for algorithmic accountability in fintech must safeguard constitutional values of equality, fairness, and innovation through multi-layered safeguards. First, fintech systems should be reviewed for algorithm audits and impact assessments to evaluate for potential bias and discriminatory outcomes and compliance with regulatory standards. For example, independently scheduled reviews of algorithms used in credit scoring, lending, or in insurance should be necessary. Second, explain ability, transparency, and due process must be part of the design and governance model. Third, equality propositions must be part of governance and mechanisms for fintech regulation, and particularly those aligned with constitutional guarantees of equality found in Articles 14, 15 and 21 of the Indian Constitution, and obligations for international norms found in the ICCPR[13] and GDPR[14]. To counterbalance the role of innovation in their underwriting procedures, there will need to be an even greater audit and oversight related to whether the fintech is being used transparently and legally. One must also stave against the exclusion of unfairly exploiting underrepresented communities. It will be vital to demand accountability and regulation of algorithmic decision-making to prevent AI from operating in counter to financial inclusion. Ultimately this article advances the conclusion that for equality rights to be maintained there must be a new legal structure embedded with accountability, fairness and transparency into AI systems. “In the end, justice must not only be coded,it must be constitutional.”

[1] The author is currently persuing BA.LLB,4^th year at Vel Tech School of Law,Avadi,Chennai-64.

Emailid:sowjanyasai04@gmail.com contactno:8838196590.

[2] Suresh Venkatasubramanian et al., Algorithmic Bias in Artificial Intelligence, 331 Comm. ACM 65, 66 (2018).

[3] Indian Constitution,Article 14 (Right to equality).

[4] M. Nagaraj v. Union of India, (2006) 8 SCC 212 (India) (discussing equality and non-arbitrariness under Article 14).

[5] E.P. Royappa v. State of Tamil Nadu, (1974) 4 SCC 3, 9 (India) (distinguishing formal and substantive equality).

[6] Arvind Narayanan et al., How to Recognize AI Bias in Financial Services, 32 J. FinTech L. & Pol’y 112, 115–16 (2021).

[7] Justice K.S. Puttaswamy v. Union of India, (2017) 10 SCC 1 (India).

[8] G.A. Res. 217 (III) A, Universal Declaration of Human Rights, art. 7 (Dec. 10, 1948).

[9] Digital Personal Data Protection Act, No. 3 of 2023, INDIA; Reserve Bank of India, Master Directions on Digital Lending (2022), https://www.rbi.org.in/Scripts/NotificationUser.aspx?Id=12260&Mode=0

[10] Regulation (EU) 2016/679, General Data Protection Regulation, art. 22, 2016 O.J. (L 119) 1.

[11] Equal Credit Opportunity Act, 15 U.S.C. § 1691 (1974); Fair Housing Act, 42 U.S.C. § 3601 (1968); Consumer Financial Protection Bureau (CFPB), Fair Lending Enforcement (2022), https://www.consumerfinance.gov/policy-compliance/guidance/fair-lending/

[12] Regulation (EU) 2016/679, General Data Protection Regulation, art. 22, 2016 O.J. (L 119) 1.

[13] International Covenant on Civil and Political Rights, art. 26, Dec. 16, 1966, 999 U.N.T.S. 171.

[14] Regulation (EU) 2016/679, General Data Protection Regulation, arts. 5–6, 22, 2016 O.J. (L 119) 1.