Senators propose AI Civil Rights Act to address 'sinister side' of artificial intelligence

AI algorithms have a dark side, said Senator Markey, second from right, “one that has real consequences for everyday people, especially marginalized communities.” Supporting him at the announcement were, left to right: Craig Aaron, president and co-CEO of Free Press Action; Laura MacCleery, senior policy director, UnidosUS; and Damon Hewitt, president and executive director of the Lawyers’ Committee for Civil Rights Under Law.

A bill introduced in the U.S. Senate on Tuesday aims to outlaw discrimination by corporate algorithms and ensure consumers have the option to have a decision made by a human being instead of artificial intelligence.

The AI Civil Rights Act would force companies that develop AI algorithms to test their algorithms for signs of discrimination. The civil legislation, if it passes, could force banks to rethink the way they use AI in areas like loan and hiring decisions. Banks already are not allowed to illegally discriminate in their lending and hiring practices. But the rules around transparency and consumers’ right to opt out of AI could require major process and technology changes. 

“Artificial intelligence is the hottest topic of the year, but these complex algorithms have a darker side as well, one that has real consequences for everyday people, from setting mortgage and insurance rates to screening job applicants to determine eligibility for benefits,” sponsor Sen. Edward Markey, D-Mass., said Tuesday. “Companies have already been using black box algorithms for years to make major decisions on Americans’ lives. Yet these algorithms are often biased against marginalized communities.” Sen. Mazie Hirono, D-Hawaii, is a co-sponsor on the bill.

Markey proposed the Eliminating Bias in Algorithmic Systems Act of 2023 in December, to require government agencies that use, fund or oversee algorithms to have an office of civil rights focused on AI bias. That bill was referred to the Committee on Homeland Security and Governmental Affairs, where it awaits consideration.

In Tuesday’s announcement, Markey said the bill’s first principle is equity: It prohibits companies from developing or using an AI-powered algorithm for a critical decision that discriminates on the basis of race, sex or other protected characteristics. 

The second principle is transparency: Companies would have to independently test and evaluate their algorithms to identify and mitigate any discrimination. 

“This provision sends a message to Silicon Valley,” Markey said. “The AI era cannot be driven by the ‘move fast and break things’ mindset that Big Tech foolishly adopted over the last two decades.”

The third principle is choice: giving every consumer the right to have major decisions made by a human.

“We know there’s a Dickensian quality to AI,” Markey said. “It’s the best of technologies and the worst of technologies simultaneously. It can enable, but it can degrade and it can do both simultaneously. So we have to protect against the sinister side of cyberspace as AI makes this inevitable powerful technology, with equity, transparency and choice.”

Companies that break this law would be subject to civil penalties of $15,000 per violation, or 4% of the defendant’s average gross annual revenue over the preceding three years, whichever is greater, according to the text of the legislation.

There haven’t been many high-profile cases involving banks or fintechs using biased AI.

Two years ago, a law firm that monitors fair-lending compliance found lower loan approval rates for Black applicants at online consumer lender Upstart Holdings. The law firm, Relman Colfax, said the disparities did not by themselves “demonstrate a fair lending violation” and were not unusual, but it recommended a “less discriminatory” model for Upstart to adopt.

The banking industry’s regulators are already on the lookout for bias. In 2022, the Consumer Financial Protection Bureau declared that discrimination associated with any financial product — not just credit — is illegal. In 2023, Fed Vice Chair for Supervision Michael Barr said the central bank would incorporate screening for discriminatory practices into its supervision framework, including evaluating applications for mergers and acquisitions.

Consumer advocacy leaders at the announcement voiced support for the bill.

“As AI has been increasingly adopted and proliferated throughout industries, it’s made an old but hard earned lesson very clear: equal opportunity and civil rights cannot be outweighed by innovation and technological advancement,” said Damon Hewitt, president and executive director of the Lawyers Committee for Civil Rights. “They can and should be linked.”

Algorithms are used to make decisions about all aspects of Americans’ daily lives, including “who gets bailed, who can rent a home, where we can go to school and who’s determined to be a threat,” Hewitt said. “And all too often, those decisions have not been made equitably or fairly and certainly not transparently.”

Craig Aaron, co-CEO of Free Press Action, said the bill would require those designing AI tools to conduct audits for potential harms and publicly share the results.

“For too long, there has been no public input or public recourse when these hidden systems cause real-world harms,” Aaron said. “The AI Civil Rights Act takes a major step toward addressing these urgent issues and empowering federal regulators to keep up with evolving technologies.”