{"doi":"10.1016/j.jclinepi.2021.03.031","title":"Data-driven methods distort optimal cutoffs and accuracy estimates of depression screening tools: a simulation study using individual participant data","abstract":"<h4>Objective</h4>To evaluate, across multiple sample sizes, the degree that data-driven methods result in (1) optimal cutoffs different from population optimal cutoff and (2) bias in accuracy estimates.<h4>Study design and setting</h4>A total of 1,000 samples of sample size 100, 200, 500 and 1,000 each were randomly drawn to simulate studies of different sample sizes from a database (n = 13,255) synthesized to assess Edinburgh Postnatal Depression Scale (EPDS) screening accuracy. Optimal cutoffs were selected by maximizing Youden's J (sensitivity+specificity-1). Optimal cutoffs and accuracy estimates in simulated samples were compared to population values.<h4>Results</h4>Optimal cutoffs in simulated samples ranged from ≥ 5 to ≥ 17 for n = 100, ≥ 6 to ≥ 16 for n = 200, ≥ 6 to ≥ 14 for n = 500, and ≥ 8 to ≥ 13 for n = 1,000. Percentage of simulated samples identifying the population optimal cutoff (≥ 11) was 30% for n = 100, 35% for n = 200, 53% for n = 500, and 71% for n = 1,000. Mean overestimation of sensitivity and underestimation of specificity were 6.5 percentage point (pp) and -1.3 pp for n = 100, 4.2 pp and -1.1 pp for n = 200, 1.8 pp and -1.0 pp for n = 500, and 1.4 pp and -1.0 pp for n = 1,000.<h4>Conclusions</h4>Small accuracy studies may identify inaccurate optimal cutoff and overstate accuracy estimates with data-driven methods.","journal":"Journal of Clinical Epidemiology","year":2021,"id":4038,"datarank":0.8885590151530633,"base_score":3.044522437723423,"endowment":3.044522437723423,"self_citation_contribution":0.4566783656585135,"citation_network_contribution":0.43188064949454985,"self_endowment_contribution":0.4566783656585135,"citer_contribution":0.43188064949454985,"corpus_percentile":null,"corpus_rank":null,"citation_count":20,"citer_count":13,"citers_with_citation_signal":10,"citers_with_endowment":10,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.0482,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2021-09-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":2035,"name":"Brooke Levis","orcid":"0000-0002-4310-3689","position":1,"is_corresponding":false},{"id":59287,"name":"Tiffany Dal Santo","orcid":"0000-0001-8505-3969","position":2,"is_corresponding":false},{"id":2055,"name":"Scott B. Patten","orcid":"0000-0001-9871-4041","position":3,"is_corresponding":false},{"id":2056,"name":"Ian Shrier","orcid":"0000-0001-9914-3498","position":4,"is_corresponding":false},{"id":2034,"name":"Brett D. Thombs","orcid":"0000-0002-5644-8432","position":5,"is_corresponding":false},{"id":2083,"name":"Andrea Benedetti","orcid":"0000-0002-8314-9497","position":6,"is_corresponding":false},{"id":2040,"name":"Ying Sun","orcid":"0000-0002-8529-3521","position":7,"is_corresponding":false},{"id":2041,"name":"Chen He","orcid":"0000-0002-7308-0350","position":8,"is_corresponding":false},{"id":2046,"name":"Danielle B. Rice","orcid":"0000-0001-5615-7005","position":9,"is_corresponding":false},{"id":2042,"name":"Ankur Krishnan","orcid":"0000-0001-8671-7421","position":10,"is_corresponding":false},{"id":2039,"name":"Yin Wu","orcid":"0000-0001-5527-1746","position":11,"is_corresponding":false},{"id":2047,"name":"Marleine Azar","orcid":null,"position":12,"is_corresponding":false},{"id":16337,"name":"Tatiana A. Sanchez","orcid":null,"position":13,"is_corresponding":false},{"id":2048,"name":"Matthew J. Chiovitti","orcid":null,"position":14,"is_corresponding":false},{"id":2049,"name":"Nazanin Saadat","orcid":null,"position":15,"is_corresponding":false},{"id":2050,"name":"Kira E. Riehm","orcid":"0000-0002-9061-0560","position":16,"is_corresponding":false},{"id":2045,"name":"Mahrukh Imran","orcid":"0000-0002-2478-0182","position":17,"is_corresponding":false},{"id":2038,"name":"Zelalem Negeri","orcid":"0000-0002-3154-3855","position":18,"is_corresponding":false},{"id":2052,"name":"Pim Cuijpers","orcid":"0000-0001-5497-2743","position":20,"is_corresponding":false},{"id":2053,"name":"Simon Gilbody","orcid":"0000-0002-8236-6983","position":21,"is_corresponding":false},{"id":2054,"name":"Lorie A. Kloda","orcid":"0000-0003-1429-1497","position":23,"is_corresponding":false},{"id":2057,"name":"Roy C. Ziegelstein","orcid":"0000-0001-7671-1501","position":24,"is_corresponding":false},{"id":2058,"name":"Liane Comeau","orcid":null,"position":25,"is_corresponding":false},{"id":2059,"name":"Nicholas D. Mitchell","orcid":null,"position":26,"is_corresponding":false},{"id":2060,"name":"Marcello Tonelli","orcid":"0000-0002-0846-3187","position":27,"is_corresponding":false},{"id":2043,"name":"Simone N. Vigod","orcid":"0000-0002-2736-9639","position":28,"is_corresponding":false},{"id":40369,"name":"Franca Aceti","orcid":"0000-0002-8245-8466","position":29,"is_corresponding":false},{"id":5984,"name":"Rubén Alvarado","orcid":"0000-0002-8091-0324","position":30,"is_corresponding":false},{"id":40371,"name":"Muideen O. Bakare","orcid":"0000-0002-1820-1922","position":32,"is_corresponding":false},{"id":2061,"name":"Jacqueline Barnes","orcid":"0000-0001-5566-8997","position":33,"is_corresponding":false},{"id":40372,"name":"Amar D. Bavle","orcid":null,"position":34,"is_corresponding":false},{"id":2062,"name":"Cheryl Tatano Beck","orcid":"0000-0001-6130-2887","position":35,"is_corresponding":false},{"id":2063,"name":"Carola Bindt","orcid":"0000-0002-6516-540X","position":36,"is_corresponding":false},{"id":40373,"name":"Philip M. Boyce","orcid":null,"position":37,"is_corresponding":false},{"id":3092,"name":"Adomas Bunevicius","orcid":null,"position":38,"is_corresponding":false},{"id":40374,"name":"Tiago Castro e Couto","orcid":"0000-0003-1521-9489","position":39,"is_corresponding":false},{"id":40375,"name":"Linda H. Chaudron","orcid":null,"position":40,"is_corresponding":false},{"id":5993,"name":"Humberto Correa","orcid":null,"position":41,"is_corresponding":false},{"id":40376,"name":"Felipe Pinheiro de Figueiredo","orcid":"0000-0001-8616-0483","position":42,"is_corresponding":false},{"id":5996,"name":"Valsamma Eapen","orcid":"0000-0001-6296-8306","position":43,"is_corresponding":false},{"id":5998,"name":"Nicolas Favez","orcid":"0000-0003-1744-7602","position":44,"is_corresponding":false},{"id":5999,"name":"Ethel Felice","orcid":null,"position":45,"is_corresponding":false},{"id":6001,"name":"Michelle Fernandes","orcid":"0000-0002-0051-3389","position":46,"is_corresponding":false},{"id":2064,"name":"Barbara Figueiredo","orcid":null,"position":47,"is_corresponding":false},{"id":5064,"name":"Jane R. W. Fisher","orcid":null,"position":48,"is_corresponding":false},{"id":40377,"name":"Lluïsa Garcia-Esteve","orcid":null,"position":49,"is_corresponding":false},{"id":40378,"name":"Lisa Giardinelli","orcid":null,"position":50,"is_corresponding":false},{"id":2065,"name":"Nadine Helle","orcid":null,"position":51,"is_corresponding":false},{"id":2066,"name":"Louise M. Howard","orcid":"0000-0001-9942-744X","position":52,"is_corresponding":false},{"id":40379,"name":"Dina Sami Khalifa","orcid":"0000-0002-2950-9068","position":53,"is_corresponding":false},{"id":2067,"name":"Jane Kohlhoff","orcid":"0000-0001-6202-6685","position":54,"is_corresponding":false},{"id":2068,"name":"Zoltán Kozinszky","orcid":"0000-0001-7485-9895","position":55,"is_corresponding":false},{"id":40380,"name":"Laima Kusminskas","orcid":null,"position":56,"is_corresponding":false},{"id":40381,"name":"Lorenzo Lelli","orcid":null,"position":57,"is_corresponding":false},{"id":2069,"name":"Angeliki A. Leonardou","orcid":null,"position":58,"is_corresponding":false},{"id":6014,"name":"Michael Maes","orcid":null,"position":59,"is_corresponding":false},{"id":40382,"name":"Valentina Meuti","orcid":"0000-0003-4957-0461","position":60,"is_corresponding":false},{"id":2070,"name":"Sandra Nakić Radoš","orcid":"0000-0002-8330-8427","position":61,"is_corresponding":false},{"id":40383,"name":"Purificación Navarro García","orcid":null,"position":62,"is_corresponding":false},{"id":6022,"name":"Daisuke Nishi","orcid":"0000-0001-9349-3294","position":63,"is_corresponding":false},{"id":40384,"name":"Daniel Okitundu Luwa E-Andjafono","orcid":null,"position":64,"is_corresponding":false},{"id":6026,"name":"Susan J. Pawlby","orcid":null,"position":65,"is_corresponding":false},{"id":2071,"name":"Chantal Quispel","orcid":null,"position":66,"is_corresponding":false},{"id":40385,"name":"Emma Robertson-Blackmore","orcid":null,"position":67,"is_corresponding":false},{"id":2072,"name":"Tamsen J. Rochat","orcid":null,"position":68,"is_corresponding":false},{"id":5107,"name":"Heather J. Rowe","orcid":null,"position":69,"is_corresponding":false},{"id":6033,"name":"Deborah J. Sharp","orcid":null,"position":70,"is_corresponding":false},{"id":40386,"name":"Bonnie W.M. Siu","orcid":"0000-0003-4373-3223","position":71,"is_corresponding":false},{"id":6035,"name":"Alkistis Skalkidou","orcid":"0000-0002-4935-7532","position":72,"is_corresponding":false},{"id":2073,"name":"Alan Stein","orcid":"0000-0001-8207-2822","position":73,"is_corresponding":false},{"id":2074,"name":"Robert C. Stewart","orcid":"0000-0002-0413-1388","position":74,"is_corresponding":false},{"id":6039,"name":"Inger Sundström-Poromaa","orcid":null,"position":76,"is_corresponding":false},{"id":2075,"name":"Meri Tadinac","orcid":"0000-0002-3770-9000","position":77,"is_corresponding":false},{"id":2076,"name":"S. Darius Tandon","orcid":"0000-0002-3243-903X","position":78,"is_corresponding":false},{"id":2077,"name":"Iva Tendais","orcid":"0000-0002-4246-8133","position":79,"is_corresponding":false},{"id":40387,"name":"Pavaani Thiagayson","orcid":null,"position":80,"is_corresponding":false},{"id":2078,"name":"Annamária Töreki","orcid":"0000-0001-8901-3833","position":81,"is_corresponding":false},{"id":40388,"name":"Anna Torres-Giménez","orcid":null,"position":82,"is_corresponding":false},{"id":2079,"name":"Thach D. Tran","orcid":null,"position":83,"is_corresponding":false},{"id":2080,"name":"Kylee Trevillion","orcid":"0000-0001-5865-4726","position":84,"is_corresponding":false},{"id":2081,"name":"Katherine Turner","orcid":"0000-0001-7268-4681","position":85,"is_corresponding":false},{"id":2082,"name":"Johann M. Vega-Dienstmaier","orcid":"0000-0002-5686-4014","position":86,"is_corresponding":false},{"id":5122,"name":"Karen Wynter","orcid":"0000-0003-4620-7691","position":87,"is_corresponding":false},{"id":6048,"name":"Kimberly A. Yonkers","orcid":"0000-0001-7326-7636","position":88,"is_corresponding":false},{"id":6067,"name":"Kuan‐Pin Su","orcid":"0000-0002-4501-2502","position":89,"is_corresponding":false},{"id":16352,"name":"Tatiana Sanchez","orcid":"0000-0003-0804-0480","position":90,"is_corresponding":false},{"id":2084,"name":"Jill Boruff","orcid":"0000-0002-0338-7322","position":91,"is_corresponding":false},{"id":148,"name":"John P. A. Ioannidis","orcid":"0000-0003-3118-6859","position":92,"is_corresponding":false},{"id":40389,"name":"Cosme Alvarado‐Esquivel","orcid":"0000-0002-0367-6052","position":94,"is_corresponding":false},{"id":40390,"name":"Amar Bavle","orcid":null,"position":95,"is_corresponding":false},{"id":40391,"name":"Philip Boyce","orcid":null,"position":96,"is_corresponding":false},{"id":3159,"name":"Adomas Bunevičius","orcid":"0000-0003-0446-6898","position":97,"is_corresponding":false},{"id":6052,"name":"Humberto Corrêa","orcid":"0000-0003-4327-6413","position":98,"is_corresponding":false},{"id":2086,"name":"Bárbara Figueiredo","orcid":"0000-0002-8209-7445","position":99,"is_corresponding":false},{"id":5131,"name":"Jane Fisher","orcid":"0000-0002-1959-6807","position":100,"is_corresponding":false},{"id":40392,"name":"Lluïsa García-Esteve","orcid":"0000-0002-2893-4409","position":101,"is_corresponding":false},{"id":2087,"name":"Angeliki Leonardou","orcid":null,"position":102,"is_corresponding":false},{"id":6057,"name":"Michaël Maes","orcid":"0000-0002-2012-871X","position":103,"is_corresponding":false},{"id":6061,"name":"Susan Pawlby","orcid":"0000-0001-9109-3457","position":105,"is_corresponding":false},{"id":2088,"name":"Tamsen Rochat","orcid":"0000-0002-5223-770X","position":107,"is_corresponding":false},{"id":5141,"name":"Heather Rowe","orcid":"0000-0002-3664-6167","position":108,"is_corresponding":false},{"id":6064,"name":"Debbie Sharp","orcid":"0000-0003-3071-9860","position":109,"is_corresponding":false},{"id":6068,"name":"Inger Sundström Poromaa","orcid":"0000-0002-2491-2042","position":110,"is_corresponding":false},{"id":40395,"name":"A. Torres","orcid":"0000-0001-5178-1917","position":111,"is_corresponding":false},{"id":2089,"name":"Thach Tran","orcid":"0000-0002-4686-8601","position":112,"is_corresponding":false},{"id":2044,"name":"Parash Mani Bhandari","orcid":"0000-0002-4235-3029","position":0,"is_corresponding":true}],"reference_count":32,"raw_metadata":{"citation_network_status":"fetched"},"created_at":"2026-03-01T18:20:47.508186Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}