A revolutionary AI tool developed by University at Buffalo outperformed most physicians and other AI systems on the USMLE exams, showcasing the immense potential of AI to enhance medical decision-making and patient care.
In a groundbreaking development in biomedical informatics, a clinical artificial intelligence tool from the University at Buffalo has demonstrated exceptional accuracy in all three parts of the United States Medical Licensing Exam (USMLE). According to a study published in the prestigious JAMA Network Open, the tool, known as Semantic Clinical Artificial Intelligence (SCAI), has outperformed most physicians and all other AI tools tested to date.
SCAI, pronounced “Sky,” scored an impressive 95.2% on Step 3 of the USMLE, surpassing the 90.5% achieved by the GPT-4 Omni tool. This remarkable achievement underscores the potential of SCAI to become a valuable partner for physicians, significantly enhancing their decision-making processes.
“Artificial intelligence isn’t going to replace doctors, but a doctor who uses AI may replace a doctor who does not,” lead author Peter L. Elkin, the chair of the Department of Biomedical Informatics in the Jacobs School of Medicine and Biomedical Sciences at UB and a physician with UBMD Internal Medicine, said in a news release.
Elkin emphasized that SCAI stands out from other AI tools due to its advanced capability for semantic reasoning. Unlike generative AI models that rely heavily on online data to draw associations, SCAI is designed to think and reason similarly to how medical professionals are trained in medical school. This foundational difference enables SCAI to answer complex medical questions with greater accuracy and depth.
The AI’s development began with a natural language processing software previously designed by the UB team. They expanded this foundation by integrating vast amounts of authoritative clinical information, including recent medical literature, clinical guidelines, genomic data and patient safety information. In total, SCAI encompasses 13 million medical facts, structured into semantic triples (subject-relation-object) to create a comprehensive knowledge network.
Additionally, SCAI employs knowledge graphs to uncover new links and hidden patterns in medical data, along with retrieval-augmented generation techniques. This sophisticated approach allows the AI to access and incorporate information from external databases, reducing the risk of fabricating responses when data is insufficient.
“SCAI is different from other large language models because it can have a conversation with you and as a human-computer partnership can add to your decision-making and thinking based on its own reasoning,” Elkin added.
The development team, comprising experts from the University at Buffalo and notable contributions from Roswell Park Comprehensive Cancer Center and the Department of Veterans Affairs, believes that SCAI has the potential to revolutionize patient care. SCAI’s capabilities could democratize specialty care, making specialized medical knowledge more accessible to primary care providers and patients alike, thereby improving patient safety and access to care.
Source: University at Buffalo