{"doi":"10.1101/2025.02.08.25321587","title":"PH-LLM: Public Health Large Language Models for Infoveillance","abstract":"Background: The effectiveness of public health intervention, such as vaccination and social distancing, relies on public support and adherence. Social media has emerged as a critical platform for understanding and fostering public engagement with health interventions. However, the lack of real-time surveillance on public health issues leveraging social media data, particularly during public health emergencies, leads to delayed responses and suboptimal policy adjustments. Methods: To address this gap, we developed PH-LLM (Public Health Large Language Models for Infoveillance)-a novel suite of large language models (LLMs) specifically designed for real-time public health monitoring. We curated a multilingual training corpus comprising 593,100 instruction-output pairs from 36 datasets, covering 96 public health infoveillance tasks and 6 question-answering datasets based on social media data. PH-LLM was trained using quantized low-rank adapters (QLoRA) and LoRA plus, leveraging Qwen 2.5, which supports 29 languages. The PH-LLM suite includes models of six different sizes: 0.5B, 1.5B, 3B, 7B, 14B, and 32B. To evaluate PH-LLM, we constructed a benchmark comprising 19 English and 20 multilingual public health tasks using 10 social media datasets (totaling 52,158 unseen instruction-output pairs). We compared PH-LLM's performance against leading open-source models, including Llama-3.1-70B-Instruct, Mistral-Large-Instruct-2407, and Qwen2.5-72B-Instruct, as well as proprietary models such as GPT-4o. Findings: Across 19 English and 20 multilingual evaluation tasks, PH-LLM consistently outperformed baseline models of similar and larger sizes, including instruction-tuned versions of Qwen2.5, Llama3.1/3.2, Mistral, and bloomz, with PH-LLM-32B achieving the state-of-the-art results. Notably, PH-LLM-14B and PH-LLM-32B surpassed Qwen2.5-72B-Instruct, Llama-3.1-70B-Instruct, Mistral-Large-Instruct-2407, and GPT-4o in both English tasks (>=56.0% vs. <=52.3%) and multilingual tasks (>=59.6% vs. <= 59.1%). The only exception was PH-LLM-7B, with slightly suboptimal average performance (48.7%) in English tasks compared to Qwen2.5-7B-Instruct (50.7%), although it outperformed GPT-4o mini (46.9%), Mistral-Small-Instruct-2409 (45.8%), Llama-3.1-8B-Instruct (45.4%), and bloomz-7b1-mt (27.9%). Interpretation: PH-LLM represents a significant advancement in real-time public health infoveillance, offering state-of-the-art multilingual capabilities and cost-effective solutions for monitoring public sentiment on health issues. By equipping global, national, and local public health agencies with timely insights from social media data, PH-LLM has the potential to enhance rapid response strategies, improve policy-making, and strengthen public health communication during crises and beyond. Funding: This study is supported in part by NIH grants R01LM013337 (YL).","journal":"medRxiv","year":2025,"id":554805,"datarank":0.0,"base_score":0.0,"endowment":0.0,"self_citation_contribution":0.0,"citation_network_contribution":0.0,"self_endowment_contribution":0.0,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":7,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.9669,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2025-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":659799,"name":"Jiaqi Zhou","orcid":"0009-0005-4139-9606","position":1,"is_corresponding":false},{"id":1453112,"name":"Chiyu Wang","orcid":null,"position":2,"is_corresponding":false},{"id":24397,"name":"Qianqian Xie","orcid":"0000-0002-9588-7454","position":3,"is_corresponding":false},{"id":1453113,"name":"Kaize Ding","orcid":null,"position":4,"is_corresponding":false},{"id":554595,"name":"Chengsheng Mao","orcid":"0000-0002-1515-9626","position":5,"is_corresponding":false},{"id":829353,"name":"Yuntian Liu","orcid":"0000-0001-6571-6112","position":6,"is_corresponding":false},{"id":1272475,"name":"Zhiyuan Cao","orcid":null,"position":7,"is_corresponding":false},{"id":1453114,"name":"Huangrui Chu","orcid":null,"position":8,"is_corresponding":false},{"id":226027,"name":"Xi Chen","orcid":"0000-0002-2058-0351","position":9,"is_corresponding":false},{"id":12534,"name":"Hua Xu","orcid":"0000-0002-5274-4672","position":10,"is_corresponding":false},{"id":22299,"name":"Heidi J. Larson","orcid":"0000-0002-8477-7583","position":11,"is_corresponding":false},{"id":58247,"name":"Yuan Luo","orcid":"0000-0003-0195-7456","position":12,"is_corresponding":false},{"id":1324921,"name":"Xinyu Zhou","orcid":"0000-0003-0968-4229","position":0,"is_corresponding":true}],"reference_count":27,"raw_metadata":null,"created_at":"2026-07-19T02:54:54.542303Z","pmid":"39990576","pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}