As AI Agents Go Rogue, Healthcare Braces For Impact

As more artificial intelligence agents break out of their sandboxes and do things their creators never intended, the healthcare industry is taking a beat.

Recent weeks have seen a barrage of news of AI agents autonomously breaking into company networks, infiltrating a government website for the first time, and carrying out hacks that were discovered only weeks or months later.

A researcher who recently left AI giant Anthropic gained widespread attention earlier this month for a viral post in which he claimed that the "people building AI earnestly believe that it could kill us all by the end of the decade" — a refrain that many prominent figures in the space have since echoed and wielded to argue for industry guardrails.

Some AI leaders have taken the extraordinary step of calling for an across-the-board slowdown of the technology.

"I think that it is fair to describe this moment in history as an AI revolution," said Margaret Hu, a professor at William & Mary Law School and data privacy expert. "Given how transformative this technology is, we need to grapple with the revolutionary impact of it."

The incidents have caused anxieties around the globe and across industries about the unknowns and challenges of AI. The healthcare industry, which has been eager to adopt the technology, has not been spared.

The government of Australia announced last Wednesday that a rogue OpenAI AI agent had recently hacked into one of its health portals, marking the first known public instance of an AI agent hacking a government system.

OpenAI on Monday apologized to the Australian government and said it was "working to do better in the future."

That same week, Epic CEO Judy Faulkner said that the healthcare software giant was pausing most of its technology developments for six weeks as it worked to secure its systems against cyberattacks.

In a statement, an Epic spokesperson told Law360 that the company's development road map hasn't changed since it presented it at its August 2026 Users Group Meeting.

"We're participating in Project Glasswing and using AI tools to stay ahead of cybersecurity threats that are growing across all industries," the spokesperson said, referring to an Anthropic initiative that gives participating organizations access to advanced AI models that help strengthen cybersecurity.

"We're also continuing rapid progress in many areas: expanded AI capabilities, Agent Factory, MyChart, EpicOps, interoperability to speed up prior authorization, and more," the spokesperson said.

They did not respond to questions about the pause itself.

AI experts and attorneys advising the healthcare industry say that all of this is amounting to a growing caution in the space.

"We're seeing a growing number of health systems slowing down the procurement process of agentic solutions in increasingly autonomous clinical use cases," said Brian Anderson, CEO of Coalition for Health AI, a nonprofit focused on developing guidelines and best practices for AI within the healthcare space.

"There's concern," he added. "I mean, you don't want an agent doing anything that comes close to the things we're seeing in the news."

Pointing to the recent OpenAI hack in Australia, Hu said a rogue AI agent breaching another government's databases has historical significance.

"It opens the door to all sorts of questions of, 'What's next?'" she said.

Both OpenAI and the government of Australia have said that the breach did not impact sensitive health records. But Hu pointed out that protected health information, known as PHI, has historically been a huge target for hackers and data breaches.

Cyberattacks in healthcare are commonplace. Giant hacks, like the one on UnitedHealth Group's Change Healthcare in 2024, exposed the sensitive information of millions of patients. Smaller breaches happen on a daily basis.

An FBI report on internet crime in April noted that the healthcare and public health sector reported 460 ransomware attacks and 182 data breaches in 2025 — the highest of any of the 16 sectors the agency tracks.

While human perpetrators of cybercrimes can be prosecuted under current laws, there's little to address autonomous cyberattacks.

"If you have a company that just says, 'We don't know,' and they shrug … there's no accountability for that type of wrongdoing," Hu said. "We're in a wild wild west situation." 

Anderson noted that the speed at which AI agents operate is generating new anxieties about cyber defenses in healthcare.

While cyberattacks can take days to weeks when carried out by human actors, the new generation of highly autonomous AI models being developed today have the ability to drastically compress those timelines.

In light of this new reality, Anderson said that cyber experts at health systems are thinking about how they can and should use agentic AI tools in-house to defend their sensitive data against AI-driven cyberattacks — essentially fighting fire with fire.

"The fact that we have to create these kinds of cybersecurity-specific efforts is testimony to the fact that we have not solved the alignment problem on the use of these AI models," he said. "It speaks to the cybersecurity concerns we have. It speaks to clinical harm that might come to our patients."

Alignment is a term used within the AI space to describe how closely an AI system adheres to the intended goals and purposes of the humans who developed it.

Over the past few years, the healthcare industry has readily but carefully adopted AI into its workflows. The technology is front and center in administrative work at providers across the country. It's also making its way into clinical work, from AI scribes that summarize doctor-patient conversations to AI-powered tools used by radiologists to analyze X-rays and MRIs.

According to a 2025 report on AI in healthcare from consulting firm McKinsey & Co. that surveyed 150 healthcare leaders in the U.S., 85% of respondents said that they were pursuing or have already implemented AI initiatives within their organizations.

Agentic AI systems, which can act with some level of autonomy, are far less common in the clinic.

Anderson said that news stories about rogue AI agents are halting developments even further, as healthcare leaders grapple with the reality that we don't yet know how to control the technology.

They're asking: "If we don't know how to manage agentic AI, why would we deploy agents in a profoundly sensitive … and high-risk space like healthcare?" he said.

Attorneys who spoke to Law360 Healthcare Authority said the recent headlines are affecting how they're advising some clients.

Jordan Cohen, a healthcare partner at Akerman LLP and the firm's digital health team lead, told Law360 in an email that he's advising his HIPAA-regulated clients to revisit their Security Rule risk analysis and risk management plan if they're looking to implement agentic AI tools.

"Because these tools may access electronic protected health information and take actions across systems with limited human involvement, organizations should evaluate whether they introduce new or changed risks and implement appropriate safeguards," he said.

Cohen added that healthcare organizations also need to think about whether their AI governance policies account for tools "that can take actions, not just generate recommendations" today.

"That may include clearly defining approved uses, access permissions, human oversight requirements, and procedures for monitoring and responding to unexpected behavior," he said.

Aaron Maguregui, a partner at Foley & Lardner LLP focused on AI and digital health, described the current state of AI in healthcare as "an exercise in maturity."

"I think healthcare is, like many other ecosystems, attempting to figure out where AI fits and where it doesn't," he said. "Healthcare companies are figuring out some of the benefits, but also some of the drawbacks."

Maguregui added that from a legal perspective, companies are starting to figure out where and how their risks increase or decrease with certain AI tools.

"When you see something in the press about the risk that AI is creating or a lawsuit that was filed because of AI, you usually are trying to figure out where you fit in that spectrum of risk," he said.

"Maybe people feel that the temperature is rising," Maguregui added. "I don't think it's hit a boiling point yet."

Across the industry, providers are grappling with the role of AI, said Tara Sklar, faculty director of the health law and policy program at the University of Arizona's James E. Rogers College of Law.

Sklar told Law360 in an email that there's a broad consensus that a coherent framework to oversee and manage such tools is necessary, but that "how to get there is lagging."

At a recent conference she attended, Sklar said she saw clinicians frustrated and concerned about the safety and transparency of AI tools and when and where they should be used within the context providing care.

But at the same time, they were seeing first-hand patients coming to their medical appointments prepared with information they've gleaned from consumer chatbots.

There's a "false binary of slowing down versus moving forward with AI," she said. "Rather, a more useful question is how to proceed deliberately."

--Additional reporting by Dan McKay. Editing by Abbie Sarfo.

For a reprint of this article, please contact reprints@law360.com.