{"doi":"10.1001/jamanetworkopen.2024.29630","title":"Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool","abstract":"<h4>Importance</h4>Test accuracy studies often use small datasets to simultaneously select an optimal cutoff score that maximizes test accuracy and generate accuracy estimates.<h4>Objective</h4>To evaluate the degree to which using data-driven methods to simultaneously select an optimal Patient Health Questionnaire-9 (PHQ-9) cutoff score and estimate accuracy yields (1) optimal cutoff scores that differ from the population-level optimal cutoff score and (2) biased accuracy estimates.<h4>Design, setting, and participants</h4>This study used cross-sectional data from an existing individual participant data meta-analysis (IPDMA) database on PHQ-9 screening accuracy to represent a hypothetical population. Studies in the IPDMA database compared participant PHQ-9 scores with a major depression classification. From the IPDMA population, 1000 studies of 100, 200, 500, and 1000 participants each were resampled.<h4>Main outcomes and measures</h4>For the full IPDMA population and each simulated study, an optimal cutoff score was selected by maximizing the Youden index. Accuracy estimates for optimal cutoff scores in simulated studies were compared with accuracy in the full population.<h4>Results</h4>The IPDMA database included 100 primary studies with 44 503 participants (4541 [10%] cases of major depression). The population-level optimal cutoff score was 8 or higher. Optimal cutoff scores in simulated studies ranged from 2 or higher to 21 or higher in samples of 100 participants and 5 or higher to 11 or higher in samples of 1000 participants. The percentage of simulated studies that identified the true optimal cutoff score of 8 or higher was 17% for samples of 100 participants and 33% for samples of 1000 participants. Compared with estimates for a cutoff score of 8 or higher in the population, sensitivity was overestimated by 6.4 (95% CI, 5.7-7.1) percentage points in samples of 100 participants, 4.9 (95% CI, 4.3-5.5) percentage points in samples of 200 participants, 2.2 (95% CI, 1.8-2.6) percentage points in samples of 500 participants, and 1.8 (95% CI, 1.5-2.1) percentage points in samples of 1000 participants. Specificity was within 1 percentage point across sample sizes.<h4>Conclusions and relevance</h4>This study of cross-sectional data found that optimal cutoff scores and accuracy estimates differed substantially from population values when data-driven methods were used to simultaneously identify an optimal cutoff score and estimate accuracy. Users of diagnostic accuracy evidence should evaluate studies of accuracy with caution and ensure that cutoff score recommendations are based on adequately powered research or well-conducted meta-analyses.","journal":"JAMA Network Open","year":2024,"id":5835,"datarank":0.42498200160843247,"base_score":2.833213344056216,"endowment":2.833213344056216,"self_citation_contribution":0.42498200160843247,"citation_network_contribution":0.0,"self_endowment_contribution":0.42498200160843247,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":16,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.0513,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2024-11-22","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":2044,"name":"Parash Mani Bhandari","orcid":"0000-0002-4235-3029","position":1,"is_corresponding":false},{"id":59287,"name":"Tiffany Dal Santo","orcid":"0000-0001-8505-3969","position":2,"is_corresponding":false},{"id":5979,"name":"Suiqiong Fan","orcid":"0000-0002-4240-4164","position":3,"is_corresponding":false},{"id":2040,"name":"Ying Sun","orcid":"0000-0002-8529-3521","position":4,"is_corresponding":false},{"id":2041,"name":"Chen He","orcid":"0000-0002-7308-0350","position":5,"is_corresponding":false},{"id":2039,"name":"Yin Wu","orcid":"0000-0001-5527-1746","position":6,"is_corresponding":false},{"id":2042,"name":"Ankur Krishnan","orcid":"0000-0001-8671-7421","position":7,"is_corresponding":false},{"id":2038,"name":"Zelalem Negeri","orcid":"0000-0002-3154-3855","position":8,"is_corresponding":false},{"id":2045,"name":"Mahrukh Imran","orcid":"0000-0002-2478-0182","position":9,"is_corresponding":false},{"id":2046,"name":"Danielle B. Rice","orcid":"0000-0001-5615-7005","position":10,"is_corresponding":false},{"id":2050,"name":"Kira E. Riehm","orcid":"0000-0002-9061-0560","position":11,"is_corresponding":false},{"id":2047,"name":"Marleine Azar","orcid":null,"position":12,"is_corresponding":false},{"id":5049,"name":"Alexander W. Levis","orcid":"0000-0002-8678-0292","position":13,"is_corresponding":false},{"id":2084,"name":"Jill Boruff","orcid":"0000-0002-0338-7322","position":14,"is_corresponding":false},{"id":2052,"name":"Pim Cuijpers","orcid":"0000-0001-5497-2743","position":15,"is_corresponding":false},{"id":2053,"name":"Simon Gilbody","orcid":"0000-0002-8236-6983","position":16,"is_corresponding":false},{"id":148,"name":"John P. A. Ioannidis","orcid":"0000-0003-3118-6859","position":17,"is_corresponding":false},{"id":2054,"name":"Lorie A. Kloda","orcid":"0000-0003-1429-1497","position":18,"is_corresponding":false},{"id":2055,"name":"Scott B. Patten","orcid":"0000-0001-9871-4041","position":19,"is_corresponding":false},{"id":2057,"name":"Roy C. Ziegelstein","orcid":"0000-0001-7671-1501","position":20,"is_corresponding":false},{"id":5047,"name":"Daphna Harel","orcid":"0000-0001-7015-5989","position":21,"is_corresponding":false},{"id":5981,"name":"Yemisi Takwoingi","orcid":"0000-0002-5828-9746","position":22,"is_corresponding":false},{"id":5048,"name":"Sarah Markham","orcid":"0000-0002-8755-5935","position":23,"is_corresponding":false},{"id":56106,"name":"Sultan H. Alamri","orcid":"0000-0003-4991-6223","position":24,"is_corresponding":false},{"id":5050,"name":"Dagmar Amtmann","orcid":"0000-0003-1142-9729","position":25,"is_corresponding":false},{"id":5985,"name":"Bruce Arroll","orcid":"0000-0002-0451-9462","position":26,"is_corresponding":false},{"id":3089,"name":"Anna Beraldi","orcid":null,"position":29,"is_corresponding":false},{"id":5053,"name":"Charles N. Bernstein","orcid":null,"position":30,"is_corresponding":false},{"id":5054,"name":"Arvin Bhana","orcid":"0000-0003-4235-0108","position":31,"is_corresponding":false},{"id":5987,"name":"Charles H. Bombardier","orcid":"0000-0003-4758-6283","position":32,"is_corresponding":false},{"id":5055,"name":"Ryna Imma Buji","orcid":"0000-0003-2366-4400","position":33,"is_corresponding":false},{"id":5989,"name":"Peter Butterworth","orcid":"0000-0002-1531-3881","position":34,"is_corresponding":false},{"id":5057,"name":"Juliana C. N. Chan","orcid":null,"position":37,"is_corresponding":false},{"id":5058,"name":"Lai Fong Chan","orcid":"0000-0003-0958-6178","position":38,"is_corresponding":false},{"id":5059,"name":"Dixon Chibanda","orcid":"0000-0003-2505-8607","position":39,"is_corresponding":false},{"id":16342,"name":"Neerja Chowdhary","orcid":"0000-0001-7875-3254","position":40,"is_corresponding":false},{"id":5060,"name":"Aaron Conway","orcid":"0000-0002-9583-8636","position":41,"is_corresponding":false},{"id":5992,"name":"Yeates Conwell","orcid":"0000-0001-8517-2132","position":42,"is_corresponding":false},{"id":3101,"name":"Federico M. Daray","orcid":"0000-0003-2949-7252","position":43,"is_corresponding":false},{"id":5061,"name":"Janneke M. de Man-van Ginkel","orcid":null,"position":44,"is_corresponding":false},{"id":16343,"name":"Jaime Delgadillo","orcid":"0000-0001-5349-230X","position":45,"is_corresponding":false},{"id":3162,"name":"Felix H. Fischer","orcid":"0000-0002-9693-6676","position":46,"is_corresponding":false},{"id":5063,"name":"Sally Field","orcid":"0000-0002-7917-2598","position":47,"is_corresponding":false},{"id":5064,"name":"Jane R. W. Fisher","orcid":null,"position":48,"is_corresponding":false},{"id":56107,"name":"Daniel S. S. Fung","orcid":null,"position":49,"is_corresponding":false},{"id":5069,"name":"Leila Gholizadeh","orcid":"0000-0002-6711-3312","position":51,"is_corresponding":false},{"id":13504,"name":"Robert C. Grant","orcid":"0000-0002-9821-0377","position":53,"is_corresponding":false},{"id":6005,"name":"Catherine G. Greeno","orcid":"0009-0003-0863-9108","position":54,"is_corresponding":false},{"id":5072,"name":"Brian J. Hall","orcid":"0000-0001-9358-2377","position":55,"is_corresponding":false},{"id":5073,"name":"Liisa Hantsoo","orcid":"0000-0002-3402-9499","position":56,"is_corresponding":false},{"id":3113,"name":"Martin Härter","orcid":"0000-0001-7443-9890","position":57,"is_corresponding":false},{"id":5077,"name":"Stevan E. Hobfoll","orcid":"0000-0003-4136-0929","position":59,"is_corresponding":false},{"id":5078,"name":"Simone Honikman","orcid":"0000-0002-7045-5532","position":60,"is_corresponding":false},{"id":5080,"name":"Thomas N. Hyphantis","orcid":"0000-0002-7647-4312","position":61,"is_corresponding":false},{"id":5081,"name":"Masatoshi Inagaki","orcid":"0000-0002-9822-8649","position":62,"is_corresponding":false},{"id":6007,"name":"Maria Iglesias-González","orcid":null,"position":63,"is_corresponding":false},{"id":5082,"name":"Hong Jin Jeon","orcid":"0000-0002-6126-542X","position":64,"is_corresponding":false},{"id":3115,"name":"Nathalie Jetté","orcid":"0000-0003-1351-5866","position":65,"is_corresponding":false},{"id":5083,"name":"Mohammad E. Khamseh","orcid":"0000-0003-4313-8440","position":66,"is_corresponding":false},{"id":6009,"name":"Kim M. Kiely","orcid":"0000-0001-5876-3201","position":67,"is_corresponding":false},{"id":5084,"name":"Brandon A. Kohrt","orcid":"0000-0002-3829-4820","position":68,"is_corresponding":false},{"id":5087,"name":"Maria Asunción Lara","orcid":null,"position":70,"is_corresponding":false},{"id":5088,"name":"Holly F. Levin-Aspenson","orcid":null,"position":71,"is_corresponding":false},{"id":5089,"name":"Shen-Ing Liu","orcid":null,"position":72,"is_corresponding":false},{"id":2136,"name":"Bernd Löwe","orcid":"0000-0003-4220-3378","position":75,"is_corresponding":false},{"id":5092,"name":"Nagendra P. Luitel","orcid":"0000-0002-8291-0205","position":76,"is_corresponding":false},{"id":5093,"name":"Crick Lund","orcid":"0000-0002-5159-8220","position":77,"is_corresponding":false},{"id":5094,"name":"Ruth Ann Marrie","orcid":"0000-0002-1855-5595","position":78,"is_corresponding":false},{"id":83706,"name":"Guillermo Pita","orcid":"0000-0003-2393-0510","position":79,"is_corresponding":false},{"id":5095,"name":"Brian P. Marx","orcid":"0000-0001-8988-1796","position":80,"is_corresponding":false},{"id":6018,"name":"Anthony McGuire","orcid":"0000-0002-2423-8625","position":81,"is_corresponding":false},{"id":5096,"name":"Sherina Mohd Sidik","orcid":"0000-0001-6754-6145","position":82,"is_corresponding":false},{"id":5097,"name":"Tiago N. Munhoz","orcid":"0000-0003-1281-9542","position":83,"is_corresponding":false},{"id":5099,"name":"Juliet E. M. Nakku","orcid":null,"position":85,"is_corresponding":false},{"id":5100,"name":"Laura Navarrete","orcid":"0000-0001-5421-9794","position":86,"is_corresponding":false},{"id":5139,"name":"Flávia de Lima Osório","orcid":"0000-0003-1396-555X","position":87,"is_corresponding":false},{"id":6027,"name":"Brian W. Pence","orcid":"0000-0001-7321-9538","position":88,"is_corresponding":false},{"id":5102,"name":"Philippe Persoons","orcid":"0000-0002-4052-3127","position":89,"is_corresponding":false},{"id":6028,"name":"Inge Petersen","orcid":"0000-0002-3573-4229","position":90,"is_corresponding":false},{"id":5104,"name":"Stephanie L. Pugh","orcid":"0000-0002-5977-8912","position":92,"is_corresponding":false},{"id":3134,"name":"Terence J. Quinn","orcid":"0000-0003-1401-0181","position":93,"is_corresponding":false},{"id":5105,"name":"Elmars Rancans","orcid":null,"position":94,"is_corresponding":false},{"id":5106,"name":"Sujit D. Rathod","orcid":"0000-0001-8783-7724","position":95,"is_corresponding":false},{"id":3136,"name":"Katrin Reuter","orcid":null,"position":96,"is_corresponding":false},{"id":3137,"name":"Alasdair G. Rooney","orcid":"0000-0001-6755-4866","position":97,"is_corresponding":false},{"id":5108,"name":"Iná S. Santos","orcid":"0000-0003-1258-9249","position":98,"is_corresponding":false},{"id":5109,"name":"Miranda T. Schram","orcid":"0000-0001-8113-7604","position":99,"is_corresponding":false},{"id":5110,"name":"Eileen H. Shinn","orcid":"0000-0002-3220-7423","position":101,"is_corresponding":false},{"id":6034,"name":"Abbey Sidebottom","orcid":"0000-0002-5852-3539","position":102,"is_corresponding":false},{"id":5111,"name":"Lena Spangenberg","orcid":"0000-0001-8248-6734","position":104,"is_corresponding":false},{"id":3146,"name":"Lesley Stafford","orcid":"0000-0002-6460-9881","position":105,"is_corresponding":false},{"id":5113,"name":"Keiko Suzuki","orcid":"0000-0002-4285-0658","position":107,"is_corresponding":false},{"id":5114,"name":"Pei Lin Lynnette Tan","orcid":"0009-0001-1716-6563","position":108,"is_corresponding":false},{"id":53976,"name":"Martin Scherer","orcid":"0000-0003-3448-9679","position":109,"is_corresponding":false},{"id":2079,"name":"Thach D. Tran","orcid":null,"position":110,"is_corresponding":false},{"id":3151,"name":"Alyna Turner","orcid":"0000-0001-7389-2546","position":111,"is_corresponding":false},{"id":5144,"name":"Henk van Weert","orcid":"0000-0001-6370-4724","position":112,"is_corresponding":false},{"id":5117,"name":"Thandi van Heyningen","orcid":"0000-0002-5812-6170","position":113,"is_corresponding":false},{"id":16351,"name":"Paul A. Vöhringer","orcid":"0000-0002-2460-4680","position":114,"is_corresponding":false},{"id":5119,"name":"Lynne I. Wagner","orcid":"0000-0001-9685-4796","position":115,"is_corresponding":false},{"id":5120,"name":"Jian Li Wang","orcid":null,"position":116,"is_corresponding":false},{"id":5121,"name":"David Watson","orcid":"0000-0003-2125-2373","position":117,"is_corresponding":false},{"id":3155,"name":"Jennifer White","orcid":"0000-0003-1235-0916","position":118,"is_corresponding":false},{"id":6044,"name":"Mary A. Whooley","orcid":"0000-0002-5943-0078","position":119,"is_corresponding":false},{"id":5122,"name":"Karen Wynter","orcid":"0000-0003-4620-7691","position":121,"is_corresponding":false},{"id":5123,"name":"Mitsuhiko Yamada","orcid":"0000-0002-5649-2310","position":122,"is_corresponding":false},{"id":5124,"name":"Qing Zhi Zeng","orcid":null,"position":123,"is_corresponding":false},{"id":2034,"name":"Brett D. Thombs","orcid":"0000-0002-5644-8432","position":125,"is_corresponding":false},{"id":2083,"name":"Andrea Benedetti","orcid":"0000-0002-8314-9497","position":126,"is_corresponding":false},{"id":56109,"name":"Depression Screening Data (DEPRESSD) PHQ Group","orcid":null,"position":127,"is_corresponding":false},{"id":5126,"name":"Hamid Reza Baradaran","orcid":"0000-0002-5070-5864","position":128,"is_corresponding":false},{"id":5128,"name":"Marcos Hortes Nisihara Chagas","orcid":"0000-0003-3752-7984","position":129,"is_corresponding":false},{"id":1143,"name":"Juliana C.N. Chan","orcid":"0000-0003-1325-1194","position":130,"is_corresponding":false},{"id":5129,"name":"Janneke M. de Man‐van Ginkel","orcid":"0000-0002-3702-3711","position":131,"is_corresponding":false},{"id":5131,"name":"Jane Fisher","orcid":"0000-0002-1959-6807","position":133,"is_corresponding":false},{"id":5065,"name":"Daniel Fung","orcid":"0000-0003-0718-9363","position":134,"is_corresponding":false},{"id":6053,"name":"Felicity Goodyear‐Smith","orcid":"0000-0002-6657-9401","position":135,"is_corresponding":false},{"id":5133,"name":"Eric Green","orcid":"0000-0003-2188-2244","position":136,"is_corresponding":false},{"id":6054,"name":"María Iglesias-González","orcid":"0000-0002-7794-1533","position":137,"is_corresponding":false},{"id":5134,"name":"Ma. Asunción Lara","orcid":"0000-0002-0536-8520","position":138,"is_corresponding":false},{"id":5135,"name":"Holly Frances Levin-Aspenson","orcid":"0000-0002-3980-8299","position":139,"is_corresponding":false},{"id":5136,"name":"Shen‐Ing Liu","orcid":null,"position":140,"is_corresponding":false},{"id":5137,"name":"Sônia Regina Loureiro","orcid":"0000-0001-9423-2897","position":141,"is_corresponding":false},{"id":5138,"name":"Juliet Nakku","orcid":"0000-0002-0611-1102","position":142,"is_corresponding":false},{"id":5140,"name":"Elmārs Rancāns","orcid":"0000-0002-8329-7887","position":143,"is_corresponding":false},{"id":5142,"name":"Martin Taylor‐Rowan","orcid":"0000-0002-3027-5369","position":144,"is_corresponding":false},{"id":2089,"name":"Thach Tran","orcid":"0000-0002-4686-8601","position":145,"is_corresponding":false},{"id":5143,"name":"Christina M. van der Feltz‐Cornelis","orcid":"0000-0001-6925-8956","position":146,"is_corresponding":false},{"id":5145,"name":"JianLi Wang","orcid":"0000-0002-1329-914X","position":147,"is_corresponding":false},{"id":2035,"name":"Brooke Levis","orcid":"0000-0002-4310-3689","position":0,"is_corresponding":true}],"reference_count":34,"raw_metadata":null,"created_at":"2026-03-01T18:20:47.508186Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}