{"doi":"10.1016/j.ebiom.2025.105663","title":"Deep learning informed multimodal fusion of radiology and pathology to predict outcomes in HPV-associated oropharyngeal squamous cell carcinoma","abstract":"BACKGROUND: We aim to predict outcomes of human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma (OPSCC), a subtype of head and neck cancer characterized with improved clinical outcome and better response to therapy. Pathology and radiology focused AI-based prognostic models have been independently developed for OPSCC, but their integration incorporating both primary tumour (PT) and metastatic cervical lymph node (LN) remains unexamined. METHODS: We investigate the prognostic value of an AI approach termed the swintransformer-based multimodal and multi-region data fusion framework (SMuRF). SMuRF integrates features from CT corresponding to the PT and LN, as well as whole slide pathology images from the PT as a predictor of survival and tumour grade in HPV-associated OPSCC. SMuRF employs cross-modality and cross-region window based multi-head self-attention mechanisms to capture interactions between features across tumour habitats and image scales. FINDINGS: Developed and tested on a cohort of 277 patients with OPSCC with matched radiology and pathology images, SMuRF demonstrated strong performance (C-index = 0.81 for DFS prediction and AUC = 0.75 for tumour grade classification) and emerged as an independent prognostic biomarker for DFS (hazard ratio [HR] = 17, 95% confidence interval [CI], 4.9-58, p < 0.0001) and tumour grade (odds ratio [OR] = 3.7, 95% CI, 1.4-10.5, p = 0.01) controlling for other clinical variables (i.e., T-, N-stage, age, smoking, sex and treatment modalities). Importantly, SMuRF outperformed unimodal models derived from radiology or pathology alone. INTERPRETATION: Our findings underscore the potential of multimodal deep learning in accurately stratifying OPSCC risk, informing tailored treatment strategies and potentially refining existing treatment algorithms. FUNDING: The National Institutes of Health, the U.S. Department of Veterans Affairs and National Institute of Biomedical Imaging and Bioengineering.","journal":"EBioMedicine","year":2025,"id":509420,"datarank":0.5456379239589579,"base_score":3.6375861597263857,"endowment":3.6375861597263857,"self_citation_contribution":0.5456379239589579,"citation_network_contribution":0.0,"self_endowment_contribution":0.5456379239589579,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":37,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.9557,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2025-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":1186968,"name":"Amaury Leroy","orcid":"0000-0001-5726-4232","position":1,"is_corresponding":false},{"id":88680,"name":"Kailin Yang","orcid":"0000-0001-5968-6738","position":2,"is_corresponding":false},{"id":1363420,"name":"Tanmoy Dam","orcid":"0000-0003-3022-0971","position":3,"is_corresponding":false},{"id":264729,"name":"Xiangxue Wang","orcid":"0000-0003-3341-9871","position":4,"is_corresponding":false},{"id":1364250,"name":"Himanshu Maurya","orcid":null,"position":5,"is_corresponding":false},{"id":1038192,"name":"Tilak Pathak","orcid":"0000-0002-9738-4172","position":6,"is_corresponding":false},{"id":699423,"name":"Jonathan Lee","orcid":"0000-0002-5955-5145","position":7,"is_corresponding":false},{"id":397746,"name":"Sarah J. Stock","orcid":"0000-0003-4308-856X","position":8,"is_corresponding":false},{"id":336625,"name":"Xiao Li","orcid":"0000-0001-7866-3160","position":9,"is_corresponding":false},{"id":264731,"name":"Pingfu Fu","orcid":"0000-0002-2334-5218","position":10,"is_corresponding":false},{"id":297606,"name":"Cheng Lu","orcid":"0000-0002-7651-3924","position":11,"is_corresponding":false},{"id":1363421,"name":"Paula Toro","orcid":"0000-0003-1660-1137","position":12,"is_corresponding":false},{"id":567928,"name":"Deborah J. Chute","orcid":null,"position":13,"is_corresponding":false},{"id":495301,"name":"Shlomo A. Koyfman","orcid":"0000-0002-2275-5346","position":14,"is_corresponding":false},{"id":241868,"name":"Nabil F. Saba","orcid":"0000-0003-4972-1477","position":15,"is_corresponding":false},{"id":241861,"name":"Mihir R. Patel","orcid":"0000-0001-6477-9728","position":16,"is_corresponding":false},{"id":237305,"name":"Anant Madabhushi","orcid":"0000-0002-5741-0399","position":17,"is_corresponding":false},{"id":1363419,"name":"Bolin Song","orcid":"0000-0001-9541-1731","position":0,"is_corresponding":true}],"reference_count":92,"raw_metadata":{"citation_network_status":"fetched"},"created_at":"2026-07-19T02:47:24.513904Z","pmid":"40121941","pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}