{"doi":"10.1371/journal.pone.0288233","title":"External evaluation of the Dynamic Criticality Index: A machine learning model to predict future need for ICU care in hospitalized pediatric patients","abstract":"OBJECTIVE: To assess the single site performance of the Dynamic Criticality Index (CI-D) models developed from a multi-institutional database to predict future care. Secondarily, to assess future care-location predictions in a single institution when CI-D models are re-developed using single-site data with identical variables and modeling methods. Four CI-D models were assessed for predicting care locations >6-12 hours, >12-18 hours, >18-24 hours, and >24-30 hours in the future. DESIGN: Prognostic study comparing multi-institutional CI-D models' performance in a single-site electronic health record dataset to an institution-specific CI-D model developed using identical variables and modelling methods. The institution did not participate in the multi-institutional dataset. PARTICIPANTS: All pediatric inpatients admitted from January 1st 2018 -February 29th 2020 through the emergency department. MAIN OUTCOME(S) AND MEASURE(S): The main outcome was inpatient care in routine or ICU care locations. RESULTS: A total of 29,037 pediatric hospital admissions were included, with 5,563 (19.2%) admitted directly to the ICU, 869 (3.0%) transferred from routine to ICU care, and 5,023 (17.3%) transferred from ICU to routine care. Patients had a median [IQR] age 68 months (15-157), 47.5% were female and 43.4% were black. The area under the receiver operating characteristic curve (AUROC) for the multi-institutional CI-D models applied to a single-site test dataset was 0.493-0.545 and area under the precision-recall curve (AUPRC) was 0.262-0.299. The single-site CI-D models applied to an independent single-site test dataset had an AUROC 0.906-0.944 and AUPRC range from 0.754-0.824. Accuracy at 0.95 sensitivity for those transferred from routine to ICU care was 72.6%-81.0%. Accuracy at 0.95 specificity was 58.2%-76.4% for patients who transferred from ICU to routine care. CONCLUSION AND RELEVANCE: Models developed from multi-institutional datasets and intended for application to individual institutions should be assessed locally and may benefit from re-development with site-specific data prior to deployment.","journal":"PLoS ONE","year":2024,"id":475020,"datarank":0.0,"base_score":0.0,"endowment":0.0,"self_citation_contribution":0.0,"citation_network_contribution":0.0,"self_endowment_contribution":0.0,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":4,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.7365,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2024-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":733333,"name":"Eduardo Trujillo-Rivera","orcid":null,"position":1,"is_corresponding":false},{"id":376327,"name":"James M. Chamberlain","orcid":"0000-0003-0543-1814","position":2,"is_corresponding":false},{"id":570163,"name":"Hiroki Morizono","orcid":"0000-0002-9678-5564","position":3,"is_corresponding":false},{"id":285986,"name":"Murray M. Pollack","orcid":"0000-0003-0966-8261","position":4,"is_corresponding":false},{"id":24122,"name":"Anita Patel","orcid":"0000-0002-3791-7379","position":0,"is_corresponding":true}],"reference_count":31,"raw_metadata":null,"created_at":"2026-07-19T02:06:16.992586Z","pmid":"38285704","pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}