Listener
Opinion

Aaron Smale: AI’s racism problem is quietly getting worse

Aaron Smale
Opinion by
Aaron Smale is a journalist specialising in te ao Māori issues.
AI has made horrendous mistakes in identifying people and amplified the aggressive policing of minority communities. Photo / Getty Images
AI has made horrendous mistakes in identifying people and amplified the aggressive policing of minority communities. Photo / Getty Images

Internet company Voyager has done a complete branding faceplant with an ad generated by artificial intelligence. It seems that the humans involved were devoid of any actual intelligence at all.

The ad showed three mates watching a UFC fight when the internet connection started getting laggy and showing a buffering screen.

Unhappy at this, the two large brown men are depicted making animalistic noises and then attacking the not-so-large white guy, who is hospitalised as a result.

“The advertisement was created using AI-generated video as part of our marketing activity,” a spokesperson told RNZ Pacific, adding Voyager was now reviewing the prompts and instructions used to generate the video “so we can understand how they contributed to the final content”.

A couple of questions: once the AI had generated the ad, did anyone human think this had missed the mark for humour and crossed into something else before they posted it? And why did it take reporters’ questions to provoke some self reflection?

There was a bit of tut-tutting about representation, but the ad shows that these machines operate on the principle of garbage in, garbage out.

Global hysteria about AI seems to have reached a peak, which is understandable when these machines are regularly going rogue and getting up to all sorts of dangerous mischief.

But it also highlights the power of nimbyism. Now that white middle-class people are getting uptight about it, it’s become a thing, even though AI has been used for some time against brown and black people. AI has long had a racism problem that has been quietly entrenching itself without comparable outrage. AI is not neutral because the people creating it are not neutral and it is being trained on data and information that is not neutral. AI is being fed analogue data that is inherently prejudiced and it doesn’t magically neutralise that human prejudice. It amplifies institutional racism and reproduces it on an industrial scale.

Take prisons and the criminal justice system, where we seem to blindly follow the US down disastrous paths. I once spoke to a senior gang member who said he found out his risk classification when he was in jail had him as a high escape risk. He was puzzled and questioned it. He found out the classification was based on him running away from Hokio Beach School when he was a kid. From this an automated system had predicted he had a propensity to a specific type of crime because he’d tried to escape sexual predators when he was a child. AI didn’t ask him what he was running away from.

Or the example of Māori kids being photographed by police in the Wairarapa, and elsewhere. When the cops got busted for this breach of privacy and laws on the treatment of children, the politicians were quick to defend them. But these photos were feeding into AI systems to train them and the people being photographed were brown kids who had not been accused of any crimes. If the cops had tried that on kids in Remuera or Khandallah they’d get their arses kicked by an army of KCs.

An abundance of research and writing out of the US in recent years shows how AI has made horrendous mistakes in identifying people and amplified the aggressive policing of minority communities. Because the systems are supposedly “scientific”, it is extremely difficult to mitigate or halt these trends. Because the companies running them are privately owned, they are effectively in a legal black box that even the courts have had difficulty penetrating. Furthermore, these companies are owned by some of the richest men in the world and like to get their own way. Palantir is suing the London Mayor’s Office for Policing and Crime after the city blocked a £50m (NZ$117m) artificial intelligence contract with the Metropolitan Police.

AI companies often trial digital surveillance on vulnerable populations and communities: first, because military and police contracts are highly profitable and second, because there is very little concern from the general population when things go wrong, such as people getting bombed, shot or wrongly locked up.

Before she retired from politics, former attorney-general Judith Collins was also minister for digitising government, which suggested to me she was quite keen to make it legally easier for AI to take over the state’s infrastructure. Her former colleague Anne Tolley wasn’t so enamoured with techno hocus-pocus. A decade ago, as Minister of Social Development, she put a stop to a controversial risk-modelling trial that aimed to run a predictive tracking model on newborns to assess their risk of abuse. Tolley scribbled a note in a draft cabinet paper: “Not on my watch, these are children, not lab rats.”

If you’re worried about AI bots accidentally escaping their computer kennels and roaming around your neighbourhood biting people and shitting on your lawn, be assured AI has already been deliberately let off the leash to meddle in a lot of other people’s lives.

You’re next.