{"doi":"10.1016/j.ebiom.2023.104783","title":"Machine learning augmentation reduces prediction error in collective forecasting: development and validation across prediction markets with application to COVID events","abstract":"<h4>Background</h4>The recent COVID-19 pandemic highlighted the challenges for traditional forecasting. Prediction markets are a promising way to generate collective forecasts and could potentially be enhanced if high-quality crowdsourced inputs were identified and preferentially weighted for likely accuracy in real-time with machine learning.<h4>Methods</h4>We aim to leverage human prediction markets with real-time machine weighting of likely higher accuracy trades to improve performance. The crowd sourced Almanis prediction market longitudinal platform (n = 1822) and Next Generation Social Science (NGS2) platform (n = 103) were utilised.<h4>Findings</h4>A 43-feature model predicted accurate forecasters, those with top quintile relative Brier accuracy, with subsequent replication in two out-of-sample datasets (p<sub>both</sub> <1 × 10<sup>-9</sup>). Trades graded by this model as having higher accuracy scores than others produced a greater AUC temporal gain in the overall market after vs before trade. Accuracy score-weighted forecasts had higher accuracy than market forecasts alone, particularly when the two systems disagreed by 5% or more for binary event prediction: the hybrid system demonstrating substantial % AUC gains of 13.2%, p = 1.35 × 10<sup>-14</sup> and 13.8%, p = 0.003 in two out-of-sample datasets. When discordant, the hybrid model was correct for COVID-19 event occurrence 72.7% of the time vs 27.3% for market models, p = 0.007. This net classification benefit was replicated in the separate Almanis B dataset, p = 2.4 × 10<sup>-7</sup>.<h4>Interpretation</h4>Real-time machine classification followed by weighting human trades according to likely accuracy improves collective forecasting performance. This could provide improved anticipation of and thus response to emerging risks.<h4>Funding</h4>This work was supported by an AusIndustry R and D tax incentive program from the Department of Industry, Science, Energy and Resources, Australia, to SlowVoice Pty Ltd. (IR 2101990) and Fellowship (GNT 1110200) and Investigator grant (GNT 1197234) to A-L Ponsonby by the National Health and Medical Research Council of Australia.","journal":"eBioMedicine","year":2023,"id":11618,"datarank":0.10397207708399181,"base_score":0.6931471805599453,"endowment":0.6931471805599453,"self_citation_contribution":0.10397207708399181,"citation_network_contribution":0.0,"self_endowment_contribution":0.10397207708399181,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":1,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.0519,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2023-10-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":40711,"name":"Karl R Mattingly","orcid":"0000-0001-5676-4032","position":1,"is_corresponding":false},{"id":40712,"name":"Ellen Morwitch","orcid":"0000-0001-7141-3134","position":2,"is_corresponding":false},{"id":40713,"name":"Frederik Bossaerts","orcid":"0000-0002-5004-9211","position":3,"is_corresponding":false},{"id":40714,"name":"Manning Clifford","orcid":null,"position":4,"is_corresponding":false},{"id":40715,"name":"Chad Nash","orcid":null,"position":5,"is_corresponding":false},{"id":40716,"name":"Anne-Louise Ponsonby","orcid":"0000-0002-6581-3657","position":7,"is_corresponding":false},{"id":148,"name":"John P. A. Ioannidis","orcid":"0000-0003-3118-6859","position":9,"is_corresponding":false},{"id":40710,"name":"Alexander Gruen","orcid":null,"position":0,"is_corresponding":true}],"reference_count":47,"raw_metadata":null,"created_at":"2026-03-01T18:20:47.508186Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}