Lawyers and academics have heavily criticised Australia’s first live AI face scanning trial, warning there are serious concerns around privacy, misidentification, and even potential for racial profiling.
Western Australia Police began the five-month trial in June, where marked vans, mounted with cameras, scanned faces of people in real time and matched them against a watchlist of around 4,000 people.
During the trial, 130,000 faces were scanned in the first week, leading to 33 alerts and 18 arrests in the state’s capital. Two false alerts were also recorded during the same week.
Legal experts argued there are serious ethical and legal concerns with the use of a technology in the public space that accesses sensitive personal data on a large scale.
“There is a tendency on the part of police to then use it to surveil and monitor activity of people who might be going about their lawful business,” Australian Lawyers Alliance spokesperson Greg Barns SC warned.
“For example, people attending protests, people who are simply congregating with other people in a public setting.
“It’s sort of surveillance without a need for a warrant, which you would normally need in order to surveil a person.”
What makes Australia more vulnerable compared to other countries that are using this technology is weaker privacy laws, Mr Barns told news.com.au.
“If your privacy is breached, it’s far from clear that the law will protect you and that you can seek damages, which of course, in the UK, you can,” Mr Barns said.
If for any reason, innocent people are misidentified and wrongly accused of criminal offences, Australian laws are too restricted to give them any chance of suing for damages.
‘Lack of transparency’
The lawyer also slammed the “lack of transparency” around the program, saying there were questions to be asked.
“This is a sort of mass surveillance gathering exercise with very little accountability attached to it,” he said.
University of Western Australia legal academic Anna Zenz found it “striking” that WA Police did not even formally consult the state’s own privacy watchdog, the Office of the Information Commissioner, before starting the trial.
Ms Zenz echoed privacy concerns, saying technologies like this not only eroded personal privacy but also our collective privacy.
“If you feel like you are constantly being watched in a public space and essentially the amount of public spaces that are not impacted by these technologies are becoming less and less, society’s behaviour changes,” she said.
“Your behaviour in moving around in public changes, and trust levels fall.”
These technologies have a tendency to creep outside their initial use, something Ms Zenz described as “Function Creep”, where a trial could become permanent and overt use could become covert use, and the context of use could change.
Despite WA Police saying no data is recorded or kept, Ms Zenz argued data security and hacking were still valid concerns.
“Because it is such sensitive, highly personal data security concerns, cyber security risks are really consequential,” Ms Zenz warned.
Experts are also concerned that the AI model is not trained on the diversity of the Australian population and, based on evidence, puts minority groups at a higher risk of misidentification.
If a specific group is already overrepresented within policing and the criminal justice system, this technology will merely replicate and potentially amplify the already existing biases, Ms Zenz said.
People from diverse or historically overpoliced backgrounds are forced to bear an added layer of fear in their daily life.
Evidence from countries like the USA and the UK has shown the technology targets particular minority groups like black people.
In Australia, that could translate into targeting and overpolicing of the indigenous communities.
Do the concerns outweigh the benefits of this tool?
WA police commissioner Col Blanch has argued the system is a policing tool rather than a general surveillance network.
“Live Facial Recognition is about protecting people, not watching people. It is another tool to help police find the right people, at the right time, for the right reasons,” the commissioner said in a statement.
This technology is often sold to organisations, particularly police forces, as a cost-effective solution to better use their resources, Australian National University digital sociologist Jessamy Perriam said.
“I think there is a desire in the community for the police to be seen… and public places feel safe for people, but it has to be done in a way that’s appropriate and respects people’s privacy and boundaries,” she said.
The experts have all called for greater transparency from WA Police and an independent oversight into the use of this technology to protect public rights.