{"doi":"10.1053/j.gastro.2023.04.033","title":"ChatGPT Answers Common Patient Questions About Colonoscopy","abstract":"See editorial on page 336. See editorial on page 336. ChatGPT (OpenAI) is a 175 billion–parameter large language model (LLM) artificial intelligence (AI) that was released in November 2022. ChatGPT is developed based on the generative pretrained transformer (GPT) 3.5 natural language processing technology and provides a conversational text response to a given prompt.1OpenAI.https://openai.com/blog/chatgptDate accessed: February 8, 2023Google Scholar One potential application of ChatGPT is answering patients’ medical questions. With more than 70 million procedures annually in the United States,2Ladabaum U. et al.Gastroenterology. 2019; 157: 137-148Abstract Full Text Full Text PDF PubMed Scopus (122) Google Scholar screening colonoscopies are frequently the subject of questions in gastroenterology. In this study, we examine the quality of ChatGPT-generated answers to common questions (CQs) about colonoscopy. We retrieved 8 CQs and answers about colonoscopy from the publicly available webpages of 3 randomly selected hospitals from the top-20 list of the US News & World Report Best Hospitals for Gastroenterology and GI Surgery (Supplementary Methods). We input these questions as prompts for ChatGPT (January 30, 2023, version) twice on the same day and recorded the ChatGPT-generated answers as AI1 and AI2, respectively. We compared the text similarity among all answers using a plagiarism detection software (Supplementary Table 1). To objectively interpret the quality of ChatGPT-generated answers, 4 gastroenterologists (2 senior gastroenterologists and 2 fellows) rated 36 pairs of CQs and answers, randomly displayed, for the following quality indicators on a 7-point Likert scale: (1) ease of understanding, (2) scientific adequacy, and (3) satisfaction with the answer (Table 1) . Raters were also requested to interpret whether the answers were AI generated or not.Table 1Quality Indicators (Ease of Understanding, Scientific Adequacy, Satisfaction) for Answers From AI and From Non-AI SourcesCommon questions (CQ)Source of answers“The answers are easy to understand.”“The answers are scientifically adequate.”“I am satisfied with the answers.”MeanPMeanPMeanPCQ1What is a colonoscopy?AI6.4.545.9.945.6.78Non-AI5.75.75.7CQ2Why is a colonoscopy performed?AI5.9.205.8.655.6.39Non-AI4.85.54.9CQ3How to prepare for a colonoscopy?AI5.9.596.1.725.81Non-AI5.85.95.6CQ4What to expect during the colonoscopy procedure?AI5.9.785.6.605.3.66Non-AI5.66.05.5CQ5What to expect after the colonoscopy procedure?AI6.3.055.9.166.1.12Non-AI5.35.15.0CQ6What to do after a negative colonoscopy result?AI6.4.445.9.286.1.77Non-AI5.86.35.8CQ7What to do after a positive colonoscopy result?AI5.0.875.4.114.9.83Non-AI4.66.04.8CQ8What to expect about complications?AI6.1.406.5.016.3.02Non-AI5.65.44.8Interpreted by four physicians with 7-points Likert Scale. (7 = Strongly agree, 4 = Neutral, 1 = Strongly disagree)Interpretation 1: “The answers are easy to understand.”Interpretation 2: “The answers are scientifically adequate.”Interpretation 3: “I am satisfied with the answers.”Statistical analysis by Mann Whitney U test. After adjustment for multiple comparison, new Bonferroni corrected alpha value = 0.05 / 56 = 0.00089 ∗: P < .00089 as significant. Open table in a new tab Interpreted by four physicians with 7-points Likert Scale. (7 = Strongly agree, 4 = Neutral, 1 = Strongly disagree) Interpretation 1: “The answers are easy to understand.” Interpretation 2: “The answers are scientifically adequate.” Interpretation 3: “I am satisfied with the answers.” Statistical analysis by Mann Whitney U test. After adjustment for multiple comparison, new Bonferroni corrected alpha value = 0.05 / 56 = 0.00089 ∗: P < .00089 as significant. We found that ChatGPT answers had extremely low text similarity (0%–16%) compared to answers on hospital webpages, and the text similarity ranged from 28% to 77% between the 2 ChatGPT answers, except for the CQ7 (Supplementary Table 1). Gastroenterologists rate","journal":"Gastroenterology","year":2023,"id":315570,"datarank":0.7525919755222388,"base_score":5.017279836814924,"endowment":5.017279836814924,"self_citation_contribution":0.7525919755222388,"citation_network_contribution":0.0,"self_endowment_contribution":0.7525919755222388,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":150,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.9474,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2023-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":50970,"name":"Kyle Staller","orcid":"0000-0003-4925-4290","position":1,"is_corresponding":false},{"id":1017619,"name":"Vlaicu Botoman","orcid":null,"position":2,"is_corresponding":false},{"id":491931,"name":"Mythili P. Pathipati","orcid":"0000-0003-3580-7974","position":3,"is_corresponding":false},{"id":1016968,"name":"Sanskriti Varma","orcid":"0000-0001-5179-7388","position":4,"is_corresponding":false},{"id":323498,"name":"Braden Kuo","orcid":"0000-0002-7228-7317","position":5,"is_corresponding":false},{"id":762937,"name":"Tsung‐Chun Lee","orcid":"0000-0002-8352-3266","position":0,"is_corresponding":true}],"reference_count":9,"raw_metadata":{"citation_network_status":"fetched"},"created_at":"2026-07-19T01:06:25.560098Z","pmid":"37150470","pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}