{"doi":"10.1093/jamia/ocab018","title":"Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data","abstract":"OBJECTIVE: The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity. MATERIALS AND METHODS: Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site. RESULTS: The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability-up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review. DISCUSSION: We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions. CONCLUSIONS: We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.","journal":"Journal of the American Medical Informatics Association","year":2021,"id":156883,"datarank":1.6210817399382635,"base_score":3.970291913552122,"endowment":3.970291913552122,"self_citation_contribution":0.5955437870328184,"citation_network_contribution":1.0255379529054451,"self_endowment_contribution":0.5955437870328184,"citer_contribution":1.0255379529054451,"corpus_percentile":87.21281039684382,"corpus_rank":1654,"citation_count":52,"citer_count":23,"citers_with_citation_signal":17,"citers_with_endowment":17,"datacite_reuse_total":0,"is_dataset":true,"is_dataset_confidence":0.6969,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2021-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":415551,"name":"Hossein Estiri","orcid":"0000-0002-0204-8978","position":1,"is_corresponding":false},{"id":39725,"name":"Griffin M. Weber","orcid":"0000-0002-2597-881X","position":2,"is_corresponding":false},{"id":557176,"name":"Bertrand Moal","orcid":"0000-0001-9197-721X","position":3,"is_corresponding":false},{"id":74694,"name":"Paul Avillach","orcid":"0000-0002-0235-7543","position":4,"is_corresponding":false},{"id":465955,"name":"Chuan Hong","orcid":"0000-0001-7056-9559","position":5,"is_corresponding":false},{"id":554604,"name":"Amelia L.M. Tan","orcid":"0000-0003-0623-6623","position":6,"is_corresponding":false},{"id":7875,"name":"Brett K. Beaulieu‐Jones","orcid":"0000-0002-6700-1468","position":7,"is_corresponding":false},{"id":274491,"name":"Víctor M. Castro","orcid":"0000-0001-7390-6354","position":8,"is_corresponding":false},{"id":554596,"name":"Thomas Maulhardt","orcid":"0000-0003-3184-1366","position":9,"is_corresponding":false},{"id":427686,"name":"Alon Geva","orcid":"0000-0002-8574-0133","position":10,"is_corresponding":false},{"id":554594,"name":"Alberto Malovini","orcid":"0000-0003-2857-5773","position":11,"is_corresponding":false},{"id":284064,"name":"Andrew M. South","orcid":"0000-0002-3204-4142","position":12,"is_corresponding":false},{"id":225296,"name":"Shyam Visweswaran","orcid":"0000-0002-2079-8684","position":13,"is_corresponding":false},{"id":225292,"name":"Michele Morris","orcid":"0000-0002-3255-5727","position":14,"is_corresponding":false},{"id":662837,"name":"Malarkodi J Samayamuthu","orcid":null,"position":15,"is_corresponding":false},{"id":32989,"name":"Gilbert S. Omenn","orcid":"0000-0002-8976-6074","position":16,"is_corresponding":false},{"id":554599,"name":"Kee Yuan Ngiam","orcid":"0000-0001-5676-2520","position":17,"is_corresponding":false},{"id":18043,"name":"Kenneth D. Mandl","orcid":"0000-0002-9781-0477","position":18,"is_corresponding":false},{"id":554579,"name":"Martin Boeker","orcid":"0000-0003-2972-2042","position":19,"is_corresponding":false},{"id":18042,"name":"Karen L. Olson","orcid":"0000-0002-5124-6129","position":20,"is_corresponding":false},{"id":443323,"name":"Danielle L. Mowery","orcid":"0000-0003-3802-4457","position":21,"is_corresponding":false},{"id":512951,"name":"Robert W Follett","orcid":"0000-0003-1618-2817","position":22,"is_corresponding":false},{"id":226887,"name":"David A. Hanauer","orcid":"0000-0001-6931-3791","position":23,"is_corresponding":false},{"id":554574,"name":"Riccardo Bellazzi","orcid":"0000-0002-6974-9808","position":24,"is_corresponding":false},{"id":14812,"name":"Jason H. Moore","orcid":"0000-0002-5015-1099","position":25,"is_corresponding":false},{"id":554572,"name":"Ne Hooi Will Loh","orcid":"0000-0002-4114-1286","position":26,"is_corresponding":false},{"id":360565,"name":"Douglas S. Bell","orcid":"0000-0002-5063-8294","position":27,"is_corresponding":false},{"id":415550,"name":"Kavishwar B. Wagholikar","orcid":"0000-0002-6219-861X","position":28,"is_corresponding":false},{"id":554583,"name":"Luca Chiovato","orcid":"0000-0001-7457-7353","position":29,"is_corresponding":false},{"id":554606,"name":"Valentina Tibollo","orcid":"0000-0003-2687-3822","position":30,"is_corresponding":false},{"id":557177,"name":"Siegbert Rieg","orcid":"0000-0001-7493-7080","position":31,"is_corresponding":false},{"id":558106,"name":"Anthony L.L.J. Li","orcid":null,"position":32,"is_corresponding":false},{"id":554592,"name":"Vianney Jouhet","orcid":"0000-0001-5272-2265","position":33,"is_corresponding":false},{"id":443325,"name":"Emily Schriver","orcid":"0000-0003-4522-1029","position":34,"is_corresponding":false},{"id":554569,"name":"Zongqi Xia","orcid":"0000-0003-1500-2589","position":35,"is_corresponding":false},{"id":578299,"name":"Meghan R. Hutch","orcid":"0000-0001-8130-9429","position":36,"is_corresponding":false},{"id":58247,"name":"Yuan Luo","orcid":"0000-0003-0195-7456","position":37,"is_corresponding":false},{"id":3527,"name":"Isaac S. Kohane","orcid":"0000-0003-2192-5160","position":38,"is_corresponding":false},{"id":249691,"name":"Gabriel A. Brat","orcid":"0000-0003-3928-5931","position":39,"is_corresponding":false},{"id":225293,"name":"Shawn N. Murphy","orcid":"0000-0002-1905-8806","position":40,"is_corresponding":false},{"id":225286,"name":"Jeffrey G. Klann","orcid":"0000-0003-2043-1601","position":0,"is_corresponding":true}],"reference_count":28,"raw_metadata":null,"created_at":"2026-07-18T23:44:13.017332Z","pmid":"33566082","pmcid":"PMC7928835","fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}