Article ; Online: Artificial Intelligence for Anesthesiology Board-Style Examination Questions: Role of Large Language Models.
Journal of cardiothoracic and vascular anesthesia
2024 Volume 38, Issue 5, Page(s) 1251–1259
Abstract: New artificial intelligence tools have been developed that have implications for medical usage. Large language models (LLMs), such as the widely used ChatGPT developed by OpenAI, have not been explored in the context of anesthesiology education. ... ...
| Abstract | New artificial intelligence tools have been developed that have implications for medical usage. Large language models (LLMs), such as the widely used ChatGPT developed by OpenAI, have not been explored in the context of anesthesiology education. Understanding the reliability of various publicly available LLMs for medical specialties could offer insight into their understanding of the physiology, pharmacology, and practical applications of anesthesiology. An exploratory prospective review was conducted using 3 commercially available LLMs--OpenAI's ChatGPT GPT-3.5 version (GPT-3.5), OpenAI's ChatGPT GPT-4 (GPT-4), and Google's Bard--on questions from a widely used anesthesia board examination review book. Of the 884 eligible questions, the overall correct answer rates were 47.9% for GPT-3.5, 69.4% for GPT-4, and 45.2% for Bard. GPT-4 exhibited significantly higher performance than both GPT-3.5 and Bard (p = 0.001 and p < 0.001, respectively). None of the LLMs met the criteria required to secure American Board of Anesthesiology certification, according to the 70% passing score approximation. GPT-4 significantly outperformed GPT-3.5 and Bard in terms of overall performance, but lacked consistency in providing explanations that aligned with scientific and medical consensus. Although GPT-4 shows promise, current LLMs are not sufficiently advanced to answer anesthesiology board examination questions with passing success. Further iterations and domain-specific training may enhance their utility in medical education. |
|---|---|
| MeSH term(s) | Humans ; Anesthesiology ; Artificial Intelligence ; Prospective Studies ; Reproducibility of Results ; Language |
| Language | English |
| Publishing date | 2024-02-01 |
| Publishing country | United States |
| Document type | Journal Article ; Review |
| ZDB-ID | 1067317-9 |
| ISSN | 1532-8422 ; 1053-0770 |
| ISSN (online) | 1532-8422 |
| ISSN | 1053-0770 |
| DOI | 10.1053/j.jvca.2024.01.032 |
| Database | MEDical Literature Analysis and Retrieval System OnLINE |
Full text online
More links
Kategorien
In stock of ZB MED Cologne/Königswinter
| Zs.A 2203: Show issues | Location: Je nach Verfügbarkeit (siehe Angabe bei Bestand) bis Jg. 1994: Bestellungen von Artikeln über das Online-Bestellformular Jg. 1995 - 2021: Lesesall (1.OG) ab Jg. 2022: Lesesaal (EG) |
|||
| Zs.MO 31: Show issues |
Order via subito
This service is chargeable due to the Delivery terms set by subito. Orders including an article and supplementary material will be classified as separate orders. In these cases, fees will be demanded for each order.