Dr. Joy Bhosai, founder and CEO of Pluto Health
Photo: Courtesy Pluto Health
AI regulation efforts are still coming into focus as current federal and state efforts are a patchwork of what's needed, according to Dr. Joy Bhosai, founder and CEO of Pluto Health.
"Regulations are nebulous," Bhosai said.
International efforts include a recent open letter on AI from celebrities and notables in the field, and in September, the United Nations launched two government bodies for AI.
As AI becomes more prevalent, ethical safeguards must evolve for hospitals and other healthcare organizations, Bhosai said.
This includes securing voluntary cooperation from Big Tech companies to agree on how to use and leverage the data, closing policy loopholes, especially for gray areas, and increasing transparency with users, she said.
WHY THIS MATTERS
Bhosai founded Pluto Health in 2022 as a tech-enabled company that uses AI to close gaps in care and connect patients to services and clinical trials.
Pluto analyzes medical data, identifying missed appointments or unfulfilled prescriptions. It can send a phlebotomist to the house or schedule a telehealth appointment.
Pluto uses hospitals' data to develop its models.
Pluto looked at the source of inputs and determined it couldn't be second-hand, Bhosai said.
To mitigate the risk of getting inaccurate data, Pluto decided to go directly to the source. The company reviews information on 5,000 to 10,000 patients a month.
"We need to acknowledge that the data can be inaccurate," Bhosai said. "AI models can't be run on incomplete information."
For example, she said, a patient will tell them they're not on a certain medication that's listed in the medical record.
"We see that the hospital has written it in wrong," Bhosai said. "It's better when we ask the patient if it is correct."
Credibility, the first risk of AI, according to Bhosia, relies on data.
Another category of AI risk is in the foundation of the model.
"I worry about models out there that will auto-deny a patient their claim because it didn't incorporate context from the medical record," she said. "I think those concerns are very real."
More context is needed from what physicians are writing into the medical record, she said.
"It's nearly impossible to train a model on the 100 different ways a physician can chart," Bhosai said.
Another risk is around what is done with the outputs of the data.
"Once you get an output, I think there is a very real component figuring out what you do with that data," Bhosai said. "I worry for patients and their privacy. I don't want things to be exploited for them."
The integration of AI into the clinical setting needs guardrails, she said.
"I like regulations," she said. "Appropriate use needs to be addressed. Just like how the FDA approves medical devices, there needs to be pathways to outline what is a credible model or not and a reviewing board that can verify you've created a legitimate model."
In August, Pluto launched C.A.R.E. on Deck (Connect, Assess, Review and Engage) to unify patient data.
In September, the Duke University spinoff announced it was expanding its payer relationships to extend its care model to patients with a wide range of insurance coverage. This includes Medicare, Medicaid and major commercial plans, such as UnitedHealthcare, Aetna, Blue Cross Blue Shield, Ambetter and Cigna in select states.
THE LARGER TREND
In an October open letter, AI scientists, celebrities and others called for a ban on the development of "superintelligence" – a hypothetical AI technology that could exceed the intelligence of all of humanity – until the technology is reliably safe and controllable, according to Forbes.
The open letter urged policymakers to "define what AI should never be allowed to do" and enact laws by the end of 2026, given the rapid expansion of the technology.
At an AI Governance meeting at the United Nations in September, discussion centered on creating a scientific panel to inform models. Two AI governance bodies were launched: The Global Dialogue on AI Governance and the Independent International Scientific Panel on AI.
The new architecture is intended to usher in a more inclusive form of international governance for AI, according to the World Economic Forum.
Email the writer: SMorse@himss.org