{"doi":"10.1093/jamiaopen/ooae111","title":"Tree-based classification model for Long-COVID infection prediction with age stratification using data from the National COVID Cohort Collaborative","abstract":"Objectives: We propose and validate a domain knowledge-driven classification model for diagnosing post-acute sequelae of SARS-CoV-2 infection (PASC), also known as Long COVID, using Electronic Health Records (EHRs) data. Materials and Methods: We developed a robust model that incorporates features strongly indicative of PASC or associated with the severity of COVID-19 symptoms as identified in our literature review. The XGBoost tree-based architecture was chosen for its ability to handle class-imbalanced data and its potential for high interpretability. Using the training data provided by the Long COVID Computation Challenge (L3C), which was a sample of the National COVID Cohort Collaborative (N3C), our models were fine-tuned and calibrated to optimize Area Under the Receiver Operating characteristic curve (AUROC) and the F1 score, following best practices for the class-imbalanced N3C data. Results: Our age-stratified classification model demonstrated strong performance with an average 5-fold cross-validated AUROC of 0.844 and F1 score of 0.539 across the young adult, mid-aged, and older-aged populations in the training data. In an independent testing dataset, which was made available after the challenge was over, we achieved an overall AUROC score of 0.814 and F1 score of 0.545. Discussion: The results demonstrated the utility of knowledge-driven feature engineering in a sparse EHR data and demographic stratification in model development to diagnose a complex and heterogeneously presenting condition like PASC. The model's architecture, mirroring natural clinician decision-making processes, contributed to its robustness and interpretability, which are crucial for clinical translatability. Further, the model's generalizability was evaluated over a new cross-sectional data as provided in the later stages of the L3C challenge. Conclusion: The study proposed and validated the effectiveness of age-stratified, tree-based classification models to diagnose PASC. Our approach highlights the potential of machine learning in addressing the diagnostic challenges posed by the heterogeneity of Long-COVID symptoms.","journal":"JAMIA Open","year":2024,"id":462338,"datarank":0.0,"base_score":0.0,"endowment":0.0,"self_citation_contribution":0.0,"citation_network_contribution":0.0,"self_endowment_contribution":0.0,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":2,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.9486,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2024-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":986548,"name":"Hayoung Jeong","orcid":"0000-0002-0359-8218","position":1,"is_corresponding":false},{"id":473588,"name":"Leeor Hershkovich","orcid":"0000-0001-7361-4765","position":2,"is_corresponding":false},{"id":666753,"name":"Peter Cho","orcid":"0000-0001-8435-652X","position":3,"is_corresponding":false},{"id":871022,"name":"Karnika Singh","orcid":"0000-0003-1181-2090","position":4,"is_corresponding":false},{"id":1045213,"name":"Lauren Lederer","orcid":"0000-0002-5872-7697","position":5,"is_corresponding":false},{"id":871021,"name":"Ali R. Roghanizad","orcid":"0000-0002-3461-5092","position":6,"is_corresponding":false},{"id":304832,"name":"Md Mobashir Hasan Shandhi","orcid":"0000-0001-9541-529X","position":7,"is_corresponding":false},{"id":85421,"name":"Warren A. Kibbe","orcid":"0000-0001-5622-7659","position":8,"is_corresponding":false},{"id":17266,"name":"Jessilyn Dunn","orcid":"0000-0002-3241-8183","position":9,"is_corresponding":false},{"id":225275,"name":"Adam Wilcox","orcid":"0000-0002-6305-735X","position":10,"is_corresponding":false},{"id":878723,"name":"Adam M. Lee","orcid":"0000-0002-8311-581X","position":11,"is_corresponding":false},{"id":225283,"name":"Alexis Graves","orcid":"0000-0002-2193-9987","position":12,"is_corresponding":false},{"id":49884,"name":"Alfred J Anzalone","orcid":"0000-0002-3212-7845","position":13,"is_corresponding":false},{"id":225303,"name":"Amin Manna","orcid":"0000-0002-4711-2128","position":14,"is_corresponding":false},{"id":710159,"name":"Amit Saha","orcid":"0000-0002-3542-4319","position":15,"is_corresponding":false},{"id":571568,"name":"Amy L. Olex","orcid":"0000-0001-8064-521X","position":16,"is_corresponding":false},{"id":948422,"name":"Andrea Zhou","orcid":"0000-0003-1845-5620","position":17,"is_corresponding":false},{"id":225276,"name":"Andrew E. Williams","orcid":"0000-0002-0692-412X","position":18,"is_corresponding":false},{"id":584723,"name":"Andrew M. Southerland","orcid":"0000-0002-3646-358X","position":19,"is_corresponding":false},{"id":225302,"name":"Andrew T. Girvin","orcid":"0000-0002-2031-9209","position":20,"is_corresponding":false},{"id":225297,"name":"Anita Walden","orcid":"0000-0002-3327-7423","position":21,"is_corresponding":false},{"id":467322,"name":"Anjali Sharathkumar","orcid":"0000-0003-4574-6175","position":22,"is_corresponding":false},{"id":225301,"name":"Benjamin Amor","orcid":"0000-0002-7841-8672","position":23,"is_corresponding":false},{"id":848497,"name":"Benjamin Bates","orcid":"0000-0002-1221-4028","position":24,"is_corresponding":false},{"id":703790,"name":"Brian Hendricks","orcid":"0000-0001-6682-1694","position":25,"is_corresponding":false},{"id":689726,"name":"Brijesh Patel","orcid":"0000-0001-8973-8228","position":26,"is_corresponding":false},{"id":948792,"name":"Caleb Alexander","orcid":null,"position":27,"is_corresponding":false},{"id":49889,"name":"Carolyn T. Bramante","orcid":"0000-0001-5858-2080","position":28,"is_corresponding":false},{"id":334851,"name":"Cavin Ward‐Caviness","orcid":"0000-0002-6322-4349","position":29,"is_corresponding":false},{"id":49932,"name":"Charisse Madlock‐Brown","orcid":"0000-0002-3647-1045","position":30,"is_corresponding":false},{"id":27488,"name":"Christine Suver","orcid":"0000-0002-2986-385X","position":31,"is_corresponding":false},{"id":49893,"name":"Christopher G. Chute","orcid":"0000-0001-5437-2545","position":32,"is_corresponding":false},{"id":948880,"name":"Christopher Dillon","orcid":null,"position":33,"is_corresponding":false},{"id":24000,"name":"Chunlei Wu","orcid":"0000-0002-2629-6124","position":34,"is_corresponding":false},{"id":948881,"name":"Clare Schmitt","orcid":null,"position":35,"is_corresponding":false},{"id":948882,"name":"Cliff Takemoto","orcid":null,"position":36,"is_corresponding":false},{"id":948883,"name":"Dan Housman","orcid":null,"position":37,"is_corresponding":false},{"id":225282,"name":"Davera Gabriel","orcid":"0000-0001-9041-4597","position":38,"is_corresponding":false},{"id":225273,"name":"David Eichmann","orcid":"0000-0003-3150-8758","position":39,"is_corresponding":false},{"id":49907,"name":"Diego R. Mazzotti","orcid":"0000-0003-3924-9199","position":40,"is_corresponding":false},{"id":455046,"name":"Donald D. Brown","orcid":"0000-0002-2740-2654","position":41,"is_corresponding":false},{"id":49927,"name":"Eilis Boudreau","orcid":null,"position":42,"is_corresponding":false},{"id":266572,"name":"Elaine Hill","orcid":"0000-0003-2494-317X","position":43,"is_corresponding":false},{"id":788941,"name":"Elizabeth Zampino","orcid":"0000-0002-8233-5132","position":44,"is_corresponding":false},{"id":948884,"name":"Emily Carlson Marti","orcid":null,"position":45,"is_corresponding":false},{"id":49914,"name":"Emily Pfaff","orcid":"0000-0002-6840-9756","position":46,"is_corresponding":false},{"id":626296,"name":"Evan French","orcid":"0000-0002-5809-5810","position":47,"is_corresponding":false},{"id":49900,"name":"Farrukh M. Koraishy","orcid":"0000-0001-6974-5674","position":48,"is_corresponding":false},{"id":948885,"name":"Federico Mariona","orcid":null,"position":49,"is_corresponding":false},{"id":256030,"name":"Fred Prior","orcid":"0000-0002-6314-5683","position":50,"is_corresponding":false},{"id":542316,"name":"George Sokos","orcid":"0000-0001-7421-0148","position":51,"is_corresponding":false},{"id":24174,"name":"Greg S. Martin","orcid":"0000-0002-9684-7593","position":52,"is_corresponding":false},{"id":225289,"name":"Harold P. Lehmann","orcid":"0000-0002-7698-219X","position":53,"is_corresponding":false},{"id":49919,"name":"Heidi Spratt","orcid":"0000-0002-9420-5028","position":54,"is_corresponding":false},{"id":369851,"name":"Hemalkumar B. Mehta","orcid":"0000-0001-9134-6370","position":55,"is_corresponding":false},{"id":49903,"name":"Hongfang Liu","orcid":"0000-0003-2570-3741","position":56,"is_corresponding":false},{"id":52914,"name":"Hythem Sidky","orcid":"0000-0003-3532-4809","position":57,"is_corresponding":false},{"id":890770,"name":"J.W. Awori Hayanga","orcid":"0000-0001-5091-7069","position":58,"is_corresponding":false},{"id":948423,"name":"Jami Pincavitch","orcid":"0000-0003-1701-0005","position":59,"is_corresponding":false},{"id":948886,"name":"Jaylyn Clark","orcid":null,"position":60,"is_corresponding":false},{"id":621075,"name":"Jeremy Harper","orcid":"0000-0003-1244-8948","position":61,"is_corresponding":false},{"id":369799,"name":"Jessica Y. Islam","orcid":"0000-0002-3690-3848","position":62,"is_corresponding":false},{"id":1139998,"name":"Jin Ge","orcid":"0009-0007-9818-1717","position":63,"is_corresponding":false},{"id":49896,"name":"Joel Gagnier","orcid":"0000-0002-3162-3935","position":64,"is_corresponding":false},{"id":49915,"name":"Joel Saltz","orcid":"0000-0002-3451-2165","position":65,"is_corresponding":false},{"id":59360,"name":"Johanna J. Loomba","orcid":"0000-0003-3673-5423","position":67,"is_corresponding":false},{"id":34495,"name":"John B. Buse","orcid":"0000-0002-9723-3876","position":68,"is_corresponding":false},{"id":621080,"name":"Jomol Mathew","orcid":"0000-0002-5229-4552","position":69,"is_corresponding":false},{"id":3662,"name":"Joni L. Rutter","orcid":"0000-0002-6502-2361","position":70,"is_corresponding":false},{"id":49908,"name":"Julie A. McMurry","orcid":"0000-0002-9353-5498","position":71,"is_corresponding":false},{"id":225104,"name":"Justin Guinney","orcid":"0000-0003-1477-1888","position":72,"is_corresponding":false},{"id":488216,"name":"Justin Starren","orcid":"0000-0002-5403-1115","position":73,"is_corresponding":false},{"id":948887,"name":"Karen Crowley","orcid":null,"position":74,"is_corresponding":false},{"id":432001,"name":"Katie R. Bradwell","orcid":"0000-0002-9730-1808","position":75,"is_corresponding":false},{"id":225298,"name":"Kellie M Walters","orcid":"0000-0002-7473-1709","position":76,"is_corresponding":false},{"id":948888,"name":"Ken Wilkins","orcid":null,"position":77,"is_corresponding":false},{"id":225307,"name":"Kenneth Gersing","orcid":"0000-0003-3897-3409","position":78,"is_corresponding":false},{"id":617832,"name":"Kenrick Cato","orcid":"0000-0002-0704-3826","position":79,"is_corresponding":false},{"id":680273,"name":"Kimberly Murray","orcid":"0000-0001-5650-170X","position":80,"is_corresponding":false},{"id":225287,"name":"Kristin Kostka","orcid":"0000-0003-2595-8736","position":81,"is_corresponding":false},{"id":948889,"name":"Lavance Northington","orcid":null,"position":82,"is_corresponding":false},{"id":948890,"name":"Lee Allan Pyles","orcid":null,"position":83,"is_corresponding":false},{"id":431564,"name":"Leonie Misquitta","orcid":null,"position":84,"is_corresponding":false},{"id":948424,"name":"Lesley Cottrell","orcid":"0000-0001-9799-9727","position":85,"is_corresponding":false},{"id":229211,"name":"Lili Portilla","orcid":null,"position":86,"is_corresponding":false},{"id":622381,"name":"Mariam Deacy","orcid":null,"position":87,"is_corresponding":false},{"id":621059,"name":"Mark M. Bissell","orcid":"0000-0002-7134-9280","position":88,"is_corresponding":false},{"id":225279,"name":"Marshall Clark","orcid":"0000-0001-5827-4862","position":89,"is_corresponding":false},{"id":948891,"name":"Mary Emmett","orcid":null,"position":90,"is_corresponding":false},{"id":948425,"name":"Mary Saltz","orcid":"0000-0002-1554-5738","position":91,"is_corresponding":false},{"id":225294,"name":"Matvey B. Palchuk","orcid":"0000-0002-7737-8752","position":92,"is_corresponding":false},{"id":4026,"name":"Melissa A Haendel","orcid":"0000-0001-9114-8737","position":93,"is_corresponding":false},{"id":464923,"name":"Meredith E. Adams","orcid":"0000-0002-7962-3475","position":94,"is_corresponding":false},{"id":948892,"name":"Meredith Temple-O’Connor","orcid":null,"position":95,"is_corresponding":false},{"id":229210,"name":"Michael G. Kurilla","orcid":null,"position":96,"is_corresponding":false},{"id":225292,"name":"Michele Morris","orcid":"0000-0002-3255-5727","position":97,"is_corresponding":false},{"id":225304,"name":"Nabeel Qureshi","orcid":"0000-0002-3177-5314","position":98,"is_corresponding":false},{"id":226785,"name":"Nasia Safdar","orcid":"0000-0003-3946-0437","position":99,"is_corresponding":false},{"id":821295,"name":"Will Ke Wang","orcid":"0000-0003-1444-5468","position":0,"is_corresponding":true}],"reference_count":21,"raw_metadata":null,"created_at":"2026-07-19T02:04:20.631898Z","pmid":"39524607","pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}