{"doi":"10.3390/ijerph20043378","title":"Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study","abstract":"<jats:p>The diagnostic accuracy of differential diagnoses generated by artificial intelligence (AI) chatbots, including the generative pretrained transformer 3 (GPT-3) chatbot (ChatGPT-3) is unknown. This study evaluated the accuracy of differential-diagnosis lists generated by ChatGPT-3 for clinical vignettes with common chief complaints. General internal medicine physicians created clinical cases, correct diagnoses, and five differential diagnoses for ten common chief complaints. The rate of correct diagnosis by ChatGPT-3 within the ten differential-diagnosis lists was 28/30 (93.3%). The rate of correct diagnosis by physicians was still superior to that by ChatGPT-3 within the five differential-diagnosis lists (98.3% vs. 83.3%, p = 0.03). The rate of correct diagnosis by physicians was also superior to that by ChatGPT-3 in the top diagnosis (53.3% vs. 93.3%, p &lt; 0.001). The rate of consistent differential diagnoses among physicians within the ten differential-diagnosis lists generated by ChatGPT-3 was 62/88 (70.5%). In summary, this study demonstrates the high diagnostic accuracy of differential-diagnosis lists generated by ChatGPT-3 for clinical cases with common chief complaints. This suggests that AI chatbots such as ChatGPT-3 can generate a well-differentiated diagnosis list for common chief complaints. However, the order of these lists can be improved in the future.</jats:p>","journal":"International Journal of Environmental Research and Public Health","year":2023,"id":610170,"datarank":0.8858042772081858,"base_score":5.905361848054571,"endowment":5.905361848054571,"self_citation_contribution":0.8858042772081858,"citation_network_contribution":0.0,"self_endowment_contribution":0.8858042772081858,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":366,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":2,"is_dataset":false,"is_dataset_confidence":null,"is_data_producer":false,"deposit_databanks":null,"is_oa":false,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":null,"fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":1568650,"name":"Yukinori Harada","orcid":"0000-0001-6042-7397","position":1,"is_corresponding":false},{"id":1568652,"name":"Masashi Yokose","orcid":"0000-0002-2685-0658","position":2,"is_corresponding":false},{"id":1568654,"name":"Tetsu Sakamoto","orcid":null,"position":3,"is_corresponding":false},{"id":1568656,"name":"Ren Kawamura","orcid":null,"position":4,"is_corresponding":false},{"id":1568657,"name":"Taro Shimizu","orcid":null,"position":5,"is_corresponding":false},{"id":1568648,"name":"Takanobu Hirosawa","orcid":"0000-0002-3573-8203","position":0,"is_corresponding":false}],"reference_count":0,"raw_metadata":{"has_enrichment":true,"resolved":true,"title":"Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study","abstract":"<jats:p>The diagnostic accuracy of differential diagnoses generated by artificial intelligence (AI) chatbots, including the generative pretrained transformer 3 (GPT-3) chatbot (ChatGPT-3) is unknown. This study evaluated the accuracy of differential-diagnosis lists generated by ChatGPT-3 for clinical vignettes with common chief complaints. General internal medicine physicians created clinical cases, correct diagnoses, and five differential diagnoses for ten common chief complaints. The rate of correct diagnosis by ChatGPT-3 within the ten differential-diagnosis lists was 28/30 (93.3%). The rate of correct diagnosis by physicians was still superior to that by ChatGPT-3 within the five differential-diagnosis lists (98.3% vs. 83.3%, p = 0.03). The rate of correct diagnosis by physicians was also superior to that by ChatGPT-3 in the top diagnosis (53.3% vs. 93.3%, p &lt; 0.001). The rate of consistent differential diagnoses among physicians within the ten differential-diagnosis lists generated by ChatGPT-3 was 62/88 (70.5%). In summary, this study demonstrates the high diagnostic accuracy of differential-diagnosis lists generated by ChatGPT-3 for clinical cases with common chief complaints. This suggests that AI chatbots such as ChatGPT-3 can generate a well-differentiated diagnosis list for common chief complaints. However, the order of these lists can be improved in the future.</jats:p>","is_dataset_classified":null,"base_score":5.905361848054571,"endowment":5.905361848054571,"datacite_reuse_total":2,"file_count":0,"downloads":0,"views":0,"has_version_chain":false,"is_dataset":false,"is_oa":false,"pmid":"36834073","pmcid":"PMC9967747","openalex_id":"https://openalex.org/W4320920036","authors":[],"funders":[],"total_grants":0,"fwci":10.5956,"citation_percentile":0.9853934,"influential_citations":0,"citation_trend":[{"year":2022,"count":1},{"year":2023,"count":107},{"year":2024,"count":145},{"year":2025,"count":73},{"year":2026,"count":39}],"oa_status":"gold","license":"cc-by","oa_locations":[{"url":"https://www.mdpi.com/1660-4601/20/4/3378/pdf?version=1676439341","host_type":"journal"},{"url":"https://www.mdpi.com/1660-4601/20/4/3378/pdf?version=1676439341","host_type":"publisher"},{"url":"https://www.mdpi.com/1660-4601/20/4/3378/pdf","host_type":"publisher"},{"url":"https://doi.org/10.3390/ijerph20043378","host_type":"journal"},{"url":"https://pubmed.ncbi.nlm.nih.gov/36834073","host_type":"repository"},{"url":"https://www.ncbi.nlm.nih.gov/pmc/articles/9967747","host_type":"repository"},{"url":"https://www.mdpi.com/1660-4601/20/4/3378/","host_type":"repository"},{"url":"https://dx.doi.org/10.3390/ijerph20043378","host_type":"repository"},{"url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC9967747/pdf/ijerph-20-03378.pdf","host_type":"repository"},{"url":"https://europepmc.org/articles/PMC9967747","host_type":"Europe_PMC"},{"url":"https://europepmc.org/articles/PMC9967747?pdf=render","host_type":"Europe_PMC"}],"fields_of_study":["Artificial Intelligence in Healthcare and Education","Machine Learning in Healthcare"],"mesh_terms":["Artificial Intelligence","Diagnosis, Differential","Humans","Pilot Projects","Software","General Practitioners"],"keywords":["Differential diagnosis","Medical diagnosis","Medicine","Diagnostic accuracy","Pediatrics","Internal medicine","Radiology","Pathology","Artificial intelligence","Diagnosis","Clinical Decision Support","Natural Language Processing","Ai Chatbot","Generative Pretrained Transformers"],"sdg_mappings":[],"linked_datasets":[{"doi":"10.6084/m9.figshare.26720949.v1","title":"Additional file 1 of Expert assessment of ChatGPT’s ability to generate illness scripts: an evaluative study","publisher":"figshare","resource_type":"JournalArticle"},{"doi":"10.6084/m9.figshare.26720949","title":"Additional file 1 of Expert assessment of ChatGPT’s ability to generate illness scripts: an evaluative study","publisher":"figshare","resource_type":"JournalArticle"}],"clinical_trials":[],"software_tools":[],"database_accessions":[],"source":"live","citation_network_status":"fetched"},"created_at":"2026-07-31T20:07:29.822888Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}