{"doi":"10.1007/s00330-024-11331-0","title":"Overlooked and underpowered: a meta-research addressing sample size in radiomics prediction models for binary outcomes","abstract":"<jats:title>Abstract</jats:title>\n          <jats:sec>\n            <jats:title>Objectives</jats:title>\n            <jats:p>To investigate how studies determine the sample size when developing radiomics prediction models for binary outcomes, and whether the sample size meets the estimates obtained by using established criteria.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Methods</jats:title>\n            <jats:p>We identified radiomics studies that were published from 01 January 2023 to 31 December 2023 in seven leading peer-reviewed radiological journals. We reviewed the sample size justification methods, and actual sample size used. We calculated and compared the actual sample size used to the estimates obtained by using three established criteria proposed by Riley et al. We investigated which characteristics factors were associated with the sufficient sample size that meets the estimates obtained by using established criteria proposed by Riley et al.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Results</jats:title>\n            <jats:p>We included 116 studies. Eleven out of one hundred sixteen studies justified the sample size, in which 6/11 performed a priori sample size calculation. The median (first and third quartile, Q1, Q3) of the total sample size is 223 (130, 463), and those of sample size for training are 150 (90, 288). The median (Q1, Q3) difference between total sample size and minimum sample size according to established criteria are −100 (−216, 183), and those differences between total sample size and a more restrictive approach based on established criteria are −268 (−427, −157). The presence of external testing and the specialty of the topic were associated with sufficient sample size.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion</jats:title>\n            <jats:p>Radiomics studies are often designed without sample size justification, whose sample size may be too small to avoid overfitting. Sample size justification is encouraged when developing a radiomics model.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Key Points</jats:title>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Question</jats:italic>\n              </jats:bold>\n              <jats:italic>Sample size justification is critical to help minimize overfitting in developing a radiomics model, but is overlooked and underpowered in radiomics research</jats:italic>.</jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Findings</jats:italic>\n              </jats:bold>\n              <jats:italic>Few of the radiomics models justified, calculated, or reported their sample size, and most of them did not meet the recent formal sample size criteria</jats:italic>.</jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Clinical relevance</jats:italic>\n              </jats:bold>\n              <jats:italic>Radiomics models are often designed without sample size justification. Consequently, many models are too small to avoid overfitting. It should be encouraged to justify, perform, and report the considerations on sample size when developing radiomics models</jats:italic>.</jats:p>\n          </jats:sec>","journal":"European Radiology","year":2025,"id":593823,"datarank":0.48283137373023016,"base_score":3.2188758248682006,"endowment":3.2188758248682006,"self_citation_contribution":0.48283137373023016,"citation_network_contribution":0.0,"self_endowment_contribution":0.48283137373023016,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":24,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":null,"is_data_producer":false,"deposit_databanks":null,"is_oa":false,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":null,"fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":1519907,"name":"Xianwei Liu","orcid":null,"position":1,"is_corresponding":false},{"id":1075017,"name":"Junjie Lu","orcid":"0000-0002-8370-7858","position":2,"is_corresponding":false},{"id":495345,"name":"Jiarui Yang","orcid":"0000-0003-2759-776X","position":3,"is_corresponding":false},{"id":1519909,"name":"Guangcheng Zhang","orcid":null,"position":4,"is_corresponding":false},{"id":851102,"name":"Shiqi Mao","orcid":"0000-0001-8036-7336","position":5,"is_corresponding":false},{"id":1519910,"name":"Haoda Chen","orcid":null,"position":6,"is_corresponding":false},{"id":266295,"name":"Qian Yin","orcid":"0000-0002-6535-9861","position":7,"is_corresponding":false},{"id":1519911,"name":"Qingqing Cen","orcid":null,"position":8,"is_corresponding":false},{"id":669948,"name":"Run Jiang","orcid":null,"position":9,"is_corresponding":false},{"id":308325,"name":"Yang Song","orcid":"0000-0001-8026-9293","position":10,"is_corresponding":false},{"id":1519913,"name":"Minda Lu","orcid":null,"position":11,"is_corresponding":false},{"id":1519914,"name":"Jingshen Chu","orcid":null,"position":12,"is_corresponding":false},{"id":336624,"name":"Yue Xing","orcid":"0000-0002-0665-2857","position":13,"is_corresponding":false},{"id":1519915,"name":"Yangfan Hu","orcid":null,"position":14,"is_corresponding":false},{"id":1519916,"name":"Defang Ding","orcid":null,"position":15,"is_corresponding":false},{"id":1519917,"name":"Xiang Ge","orcid":null,"position":16,"is_corresponding":false},{"id":693774,"name":"Huan Zhang","orcid":"0000-0003-0259-2735","position":17,"is_corresponding":false},{"id":1519918,"name":"Weiwu Yao","orcid":null,"position":18,"is_corresponding":false},{"id":1519906,"name":"Jingyu Zhong","orcid":"0000-0002-9817-2294","position":0,"is_corresponding":false}],"reference_count":0,"raw_metadata":{"has_enrichment":true,"resolved":true,"title":"Overlooked and underpowered: a meta-research addressing sample size in radiomics prediction models for binary outcomes","abstract":"<jats:title>Abstract</jats:title>\n          <jats:sec>\n            <jats:title>Objectives</jats:title>\n            <jats:p>To investigate how studies determine the sample size when developing radiomics prediction models for binary outcomes, and whether the sample size meets the estimates obtained by using established criteria.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Methods</jats:title>\n            <jats:p>We identified radiomics studies that were published from 01 January 2023 to 31 December 2023 in seven leading peer-reviewed radiological journals. We reviewed the sample size justification methods, and actual sample size used. We calculated and compared the actual sample size used to the estimates obtained by using three established criteria proposed by Riley et al. We investigated which characteristics factors were associated with the sufficient sample size that meets the estimates obtained by using established criteria proposed by Riley et al.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Results</jats:title>\n            <jats:p>We included 116 studies. Eleven out of one hundred sixteen studies justified the sample size, in which 6/11 performed a priori sample size calculation. The median (first and third quartile, Q1, Q3) of the total sample size is 223 (130, 463), and those of sample size for training are 150 (90, 288). The median (Q1, Q3) difference between total sample size and minimum sample size according to established criteria are −100 (−216, 183), and those differences between total sample size and a more restrictive approach based on established criteria are −268 (−427, −157). The presence of external testing and the specialty of the topic were associated with sufficient sample size.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion</jats:title>\n            <jats:p>Radiomics studies are often designed without sample size justification, whose sample size may be too small to avoid overfitting. Sample size justification is encouraged when developing a radiomics model.</jats:p>\n          </jats:sec>\n          <jats:sec>\n            <jats:title>Key Points</jats:title>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Question</jats:italic>\n              </jats:bold>\n              <jats:italic>Sample size justification is critical to help minimize overfitting in developing a radiomics model, but is overlooked and underpowered in radiomics research</jats:italic>.</jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Findings</jats:italic>\n              </jats:bold>\n              <jats:italic>Few of the radiomics models justified, calculated, or reported their sample size, and most of them did not meet the recent formal sample size criteria</jats:italic>.</jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Clinical relevance</jats:italic>\n              </jats:bold>\n              <jats:italic>Radiomics models are often designed without sample size justification. Consequently, many models are too small to avoid overfitting. It should be encouraged to justify, perform, and report the considerations on sample size when developing radiomics models</jats:italic>.</jats:p>\n          </jats:sec>","is_dataset_classified":null,"base_score":3.2188758248682006,"endowment":3.2188758248682006,"datacite_reuse_total":0,"file_count":0,"downloads":0,"views":0,"has_version_chain":false,"is_dataset":false,"is_oa":false,"pmid":"39789271","pmcid":"PMC11835977","openalex_id":"https://openalex.org/W4406199868","authors":[],"funders":[{"funder_name":"National Natural Science Foundation of China","grant_id":"82302183","title":null},{"funder_name":"National Natural Science Foundation of China","grant_id":"82471935","title":null},{"funder_name":"National Natural Science Foundation of China","grant_id":"82271934","title":null},{"funder_name":"Research Found of Health Commission of Shanghai Municipality","grant_id":"20244Y0214","title":null},{"funder_name":"Yangfan Project of Science and Technology Commission of Shanghai Municipality","grant_id":"22YF1442400","title":null},{"funder_name":"Research Found of Health Commission of Changing District, Shanghai Municipality","grant_id":"2023QN01","title":null},{"funder_name":"Laboratory Open Fund of Key Technology and Materials in Minimally Invasive Spine Surgery","grant_id":"2024JZWC-ZDA03","title":null},{"funder_name":"Laboratory Open Fund of Key Technology and Materials in Minimally Invasive Spine Surgery","grant_id":"2024JZWC-YBA07","title":null},{"funder_name":"Research Fund of Tongren Hospital, Shanghai Jiao Tong University School of Medicine","grant_id":"TRKYRC-XX202204","title":null},{"funder_name":"Research Fund of Tongren Hospital, Shanghai Jiao Tong University School of Medicine","grant_id":"TRYXJH28","title":null},{"funder_name":"Research Fund of Tongren Hospital, Shanghai Jiao Tong University School of Medicine","grant_id":"TRYJ2021JC06","title":null},{"funder_name":"Research Fund of Tongren Hospital, Shanghai Jiao Tong University School of Medicine","grant_id":"TRGG202101","title":null},{"funder_name":"Research Fund of Tongren Hospital, Shanghai Jiao Tong University School of Medicine","grant_id":"TRYXJH18","title":null}],"total_grants":13,"fwci":16.6955,"citation_percentile":0.99479996,"influential_citations":0,"citation_trend":[{"year":2025,"count":15},{"year":2026,"count":9}],"oa_status":"hybrid","license":"cc-by","oa_locations":[{"url":"https://link.springer.com/content/pdf/10.1007/s00330-024-11331-0.pdf","host_type":"journal"},{"url":"https://link.springer.com/content/pdf/10.1007/s00330-024-11331-0.pdf","host_type":"publisher"},{"url":"https://link.springer.com/article/10.1007/s00330-024-11331-0/fulltext.html","host_type":"publisher"},{"url":"https://doi.org/10.1007/s00330-024-11331-0","host_type":"journal"},{"url":"https://pubmed.ncbi.nlm.nih.gov/39789271","host_type":"repository"},{"url":"https://www.ncbi.nlm.nih.gov/pmc/articles/11835977","host_type":"repository"},{"url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC11835977/pdf/330_2024_Article_11331.pdf","host_type":"repository"},{"url":"https://europepmc.org/articles/PMC11835977","host_type":"Europe_PMC"},{"url":"https://europepmc.org/articles/PMC11835977?pdf=render","host_type":"Europe_PMC"}],"fields_of_study":["Radiomics and Machine Learning in Medical Imaging","Artificial Intelligence in Healthcare and Education","AI in cancer detection"],"mesh_terms":["Radiomics","Humans","Models, Statistical","Sample Size"],"keywords":["Sample size determination","Sample (material)","Quartile","Medicine","Statistics","Mathematics","Confidence interval","Methodology","Prediction model","Sample size","Radiomics"],"sdg_mappings":[{"sdg_number":0,"sdg_label":"Partnerships for the goals"}],"linked_datasets":[],"clinical_trials":[],"software_tools":[],"database_accessions":[],"source":"live","citation_network_status":"fetched"},"created_at":"2026-07-27T12:22:57.245519Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}