AI agents aren’t legally responsible for any harm that they cause, experts say. So who is?

The law is clear, says Prof Jeannie Paterson. “If I deploy an AI agent and it causes harm to someone else, I am responsible for that harm.

“Even if I didn’t intend for that to happen, it was foreseeable, and I should be taking responsibility.”

However, the director of the University of Melbourne’s Centre for AI and Digital Ethics acknowledges there is also a good deal of scope for legal and ethical “murkiness” around actions perpetrated by automated agents, in the wake of Australia’s first known agentic AI “accident”.

An agent is an autonomous software system that, tasked with pursuing a goal, can do it without human oversight – every step of the way.

This week the ABC reported on the case of Andrew, an AI expert who went only by his first name, who asked his agentic program to book some gym classes for him.

After being informed he was fourth on a waitlist for a class, Andrew asked the agent if it was possible to move him up the list.

To his shock, his agent hacked his gym’s software system and booted another member off a waitlist so he could get into a class sooner.

“It could book classes months outside the intended booking window, before they were supposed to be available,” Andrew wrote.

“Worse, it could cancel other members’ reservations and bump them off the waitlist.”

The agent was being helpful, he wrote, but his experience showed that if you gave an AI permission to do something for you, it would “often discover paths you did not explicitly ask it to look for”.

Andrew asked the agent to undo its cancellation of the other person’s reservation, but it was unable to do so.

“Sorry about that – I should have been more careful with the test,” the AI agent responded.

He then tasked the AI with writing an email to the gym’s software provider about the software vulnerability it had exploited.

A spokesperson for Victoria police said Andrew’s matter “does not appear to involve any criminality”. However, experts say there will be more dramatic cases that may cause harm – and break the law.

Legal ambiguity

Australian law applies only to people, not virtual beings. Experts say the person – or business – that deploys the AI agent is legally responsible. And they warn that those “deployers” often have very little idea about their legal liability.

“We’re going to see a lot of cases like this,” says Dr Rebecca Johnson, an AI evaluation and governance expert at the University of Sydney.

Paterson says it appeared Andrew had been responsible in his case, by going public and trying to fix the problem, but elsewhere there was “a kind of gung-ho mentality”.

“As soon as you give people the capacity to create agents to do things for them, the likelihood is that there will be accidents like this,” she says.

Paterson gives the notional example of someone having a bad experience at a rented property, so they ask their agent to write a review.

“And the agent doesn’t just write one, it writes 10, it’s pumping out reviews. So the listing plummets. You could destroy a business,” she says.

“You’re probably responsible for engaging in a fraudulent activity, you may have defamed the owner.”

And it could be worse. “[What if] it engages in racist, sexist, misogynistic language?” Paterson asks. “Then you might be asking where the guardrails are that the developer has provided. You should be putting out a product that is reasonably safe.”

In that case, the developer could also be held responsible for not putting basic guardrails in place, which is where some legal ambiguity could creep in, she says.

Paterson said the law is different to, but informed by, ethics.

“It tells us what we should be aspiring to, and the law is often ruling out the worst conduct,” she says.

The federal government’s new AI office lists a (non-comprehensive) range of laws that apply to AI, including breaching privacy, consumer, online safety, defamation and criminal laws.

Both Paterson and Johnson dislike the word “rogue” – the idea that an AI agent is operating entirely on its own – because parameters and safeguards can be put in place.

“As soon as we allow AI agents to act for us, they’re acting on the goal we give them, and if we don’t give them a whole bunch of parameters, the agent’s just going to try to achieve that goal [in any way],” Johnson says.

“I hear a lot of people saying ‘Oh, I made an agent’, and they’re experimenting, and that’s fine, but they’re given these tools without a lot of guidance, and the guidance that’s out there is of highly variable quality.”

Eventually, Paterson says, cases will end up in court where legal precedents will be set, and the developers will have a duty to monitor incidents, evolve and improve their protocols in response.

Andrew wrote that his situation felt “less like a one-off bug story and more like a preview”.

“Things are getting weird. And a bit scarier.”