Diagnosis ChatGPT

Vijay Mago Chris Robinson

Imagine going to your doctor and getting a medical diagnosis from a large language model (LLM) like ChatGPT. It may seem futuristic, but it is closer than most people realize.

“This kind of agentic AI [artificial intelligence] is the next big thing,” says Associate Professor Vijay Mago, director of the DaTALab at York University and Chair of the School of Health Policy & Management, Faculty of Health.

“Right now, a clinician reviews test results such as CT scans, gathers the patient’s medical history, and uses that information to make a diagnosis or decide on treatment. In the future, AI agents will be capable of performing many of these tasks – analyzing data, integrating medical information, and supporting or even making clinical decisions. We’re heading toward a much more advanced stage of health-care intelligence.”

It is the stuff of science fiction and York researchers are on the cutting edge of helping to make it happen in a safe and ethical way. As Mago points out, LLMs are becoming increasingly sophisticated, and the emergence of agentic AI – systems capable of autonomous reasoning and decision making in health care – appears to be imminent.

As a type of LLM, these agentic AI models involve several agents working together to accomplish complex tasks, memorize and collect data, plan, reason and learn.

Mago is part of a project developing an AI-powered doctor’s assistant for patients with chest pain to enhance diagnostic support in First Nations communities in northern Ontario, as well as other rural areas.

“AI models are becoming more intelligent every day.”

If a patient presents with chest pain, a doctor would ask all sorts of questions, including medical history and symptoms, but Mago says the AI assistant could say: “‘Hey, doctor or nurse practitioner, you missed asking this question,’ or recommend care approaches or suggest an ambulance be called.” The ultimate goal is to improve diagnostic accuracy, patient outcomes and safety.

“For rural emergency departments, where there is limited access to critical care, these AI-based approaches can help alleviate a lot of pressure,” says Mago, a member of Connected Minds and the Centre for AI & Society at York.

The model, once complete, still needs to undergo testing in a clinical setting, but down the line these types of AI models could help manage and diagnose any number of ailments, including strokes or diabetes.

“There are some very exciting things happening right now in the field and a rush to leverage the potential of these systems to improve health care, which would eventually include treatment options and predicting disease progression and outcome,” says Mago, whose health related research has garnered some $3.5 million in funding, including from the Natural Sciences and Engineering Research Council of Canada, the Social Sciences and Humanities Research Council and the Public Health Agency of Canada (PHAC).

With such a surge in interest, Mago acknowledges, comes the necessity to ensure AI is unbiased, ethical and adding benefit, rather than harm, to patients.

“AI models are becoming more intelligent every day. The goal is to figure out how to infuse them with emotional intelligence and cognition, and to make sure they are safe,” says York Research Chair in Safe AI for Health Equity, Elham Dolatabadi, who recently received funding through the Canada Foundation for Innovation’s John R. Evans Leaders Fund to start the Health Equity and AI Lab (HEAL).

Over the next five years, she will be part of a team building a human-AI complementary system that combines human brain power with cognitively robust and emotionally intelligent AI for use in health care and mental health.

She agrees agentic AI holds the promise of being a game changer for improved health care, which is why much of her current focus is on creating toolkits and pipelines to evaluate these systems before being deployed. These multi-agent models are more complicated to assess than non-agentic generative AI models where it is easier to see if the outputs align with expectations.

When dealing with several agents in a model, if one is biased, perhaps toward a certain demographic, it could throw off the accuracy and safety of the output, the diagnosis or prognosis.

“Hackers are another issue. They can attack one of the agents in the group or infuse a faulty agent into the system, which may corrupt how the system thinks, or push it into hallucinating. That’s something many will find surprising, but AI not only lies; it hallucinates,” she says. In both cases, the model outputs something that looks factual, but is not.

“Hallucination is very complicated to understand, but we are working on a dynamic pipeline for hallucination evaluation, as well as pipelines for AI agentic models in mental health, acute care and outpatient care. These are across different dimensions, clinical values, behavioural values or cognition, and emotional intelligence, so they align with human values,” says Dolatabadi of York’s Faculty of Health and a faculty affiliate of the Vector Institute.

There can be different agentic systems for each health-care application. What will work for mental health care will not work for acute care. “We also need to ensure the output is not something AI made up because it didn’t know the answer.”

Gender, ethnicity, race and skin colour can all affect accuracy. Anyone who falls outside of the average parameters is not always served well.

“That’s the risk,” says York Associate Professor Ian Stedman of the School of Public Policy & Administration, Faculty of Liberal Arts & Professional Studies. He is cross-appointed to York’s Osgoode Hall Law School where he graduated with a PhD. “Different subpopulations are not always captured as well with AI models. The question becomes, as a doctor do you still deploy the model if you know that it’s going to catch 80 per cent of European diseases and 10 per cent of sub-Saharan African diseases?” It’s a question Dolatabadi ponders often as she develops evaluation tools, suggesting the doctor needs to be aware of any limitations so they can make adjustments.

Despite the current shortcomings, Stedman too holds out great hope in the power of AI for the future of health care, specifically in how AI can help better leverage genome sequencing. As someone who lives with a rare genetic condition, he is a prime example of a person whose health data might not yet be captured by larger AI models, as is his daughter who inherited the same gene mutation.

It took 32 years of searching before he finally had a diagnosis and learned the name of his rare disease, which allowed him to access appropriate medicine. Stedman says that getting a diagnosis felt, at the time, like winning a lottery.

There are more than 8,000 known rare diseases that affect one in 12 Canadians, but only about five per cent of them have an effective therapy, with even fewer having access to available therapy. Stedman really is among the lucky.

In his vision of the future, he says, AI plays an outsized role in health-care systems. With its ability to speed up genomic data interpretation, unpack clues to rare disease diagnoses and generally help in the understanding of each individual’s needs better, he believes that getting AI right will be non-negotiable in moving toward the ideal of having personalized health-care systems.

“If you look at the power of genomics in the context of its ability to improve care, it comes from unlocking the data through genomics sequencing. If we can do that with big data analytics and in an everyday clinical setting, we can change health care,” he says. “The challenge is all the legal stuff, which is where my unique hat comes in.”

Stedman brings a particular set of skills, knowledge and experience as a patient to the realm of AI and health. He is on the executive of the Centre for AI & Society and Connected Minds at York and the Chair of the advisory board for the Canadian Institutes of Health Research’s (CIHR) Institute of Genetics.

“The potential of AI is bigger than people imagine.”

Ensuring rare disease health-care data and genome sequencing data is available across the country is a big part of the equation. That was the impetus for the Pan-Canadian Genome Library (PCGL), funded by the CIHR and Genome Canada’s Canadian Precision Health Initiative, with the goal to sequence 100,000 genomes and deposit them in the PCGL. “We’ve done a lot of infrastructure work behind the scenes to build a secure, privacy-protected, properly governed repository,” says Stedman, a member of the PCGL leadership team.

He is confident the repository will lead to more equitable precision health and result in faster rare disease diagnoses. “AI, genetics, precision therapy, it all fits under the umbrella of – how do we build a personalized health-care system where data analytics has an actual bedside impact. When you walk into the ER or your family doctor’s office and data analytics has some impact on you getting quicker, better, more accurate care, people will understand the value of the infrastructure,” he says.

He was also instrumental in the creation of the Canadian Rare Disease Network, which got its start in 2023 with funding from One Child Every Child, a Canada First Research Excellence Fund grant hosted by the University of Calgary. It helps connect the country’s rare disease scientists and clinicians with patient expertise to advance care and research.

“I believe the rare disease patient’s experience will teach us how to create a personalized health-care system. If you build a system that treats every individual as an individual, you’ll create a system that cares for more. I think we’re going to find we’re all rare, even if we don’t call ourselves rare disease patients,” he says. “The potential of AI is bigger than people imagine.”

It is already exploding into so many aspects of the healthcare field, even if patients are unaware, adds Mago. In addition to his work on LLM-based doctor assistants, he is partnering with the Northern Ontario Academic Medicine Association to use LLMs for text simplification for things like medical summaries in highly technical research articles.

It is another way AI is removing geographical boundaries to medical knowledge, he says. “It’s making research a lot more understandable and accessible, not only to lay people, but also to medical practitioners.” The continuing challenge is ensuring the summaries are ethical and sensitive, including to Indigenous and Black communities. With an ill family member in India, Mago knows first-hand the value of these sorts of AI-assisted summaries to bridge the gap not only between layperson knowledge and medical jargon, but also between different countries.

He is also in the second stage of a project that monitors substance-related issues in real time using an LLM-based surveillance system to analyze social media, mainstream news items, including images and videos, and hospital reports across the country. As part of a larger team, the work could result in earlier intervention and more targeted healthcare action. It has already expanded into a multi-institutional collaboration with the Canadian Centre on Substance Use and Addiction and the Urban Data Centre at the University of Toronto, funded by PHAC Enhanced Surveillance for Chronic Disease Program.

“This is the time for us to embrace AI, especially in the medical domain because there is the promise of huge benefits,” says Mago, who is excited about the opening of York’s new School of Medicine saying it will help further accelerate research outputs.

He also believes Canada should design and develop its own AI technology and software rather than use technologies made elsewhere that are then customized for Canadians. It’s a matter of AI sovereignty, he says.