{"doi":"10.1101/2024.06.03.597245","title":"FusOn-pLM: A Fusion Oncoprotein-Specific Language Model via Focused Probabilistic Masking","abstract":"<jats:title>Abstract</jats:title>\n                <jats:p>\n                  Fusion oncoproteins, a class of chimeric proteins arising from chromosomal translocations, drive and sustain various cancers, particularly those impacting children. Unfortunately, due to their intrinsically disordered nature, large size, and lack of well-defined, druggable pockets, they have been historically challenging to target therapeutically: neither small molecule-based methods nor structure-based approaches for binder design are strong options for this class of molecules. Recently, protein language models (pLMs) have demonstrated success at representing protein sequences with information-rich embeddings, enabling downstream design applications from sequence alone. However, no current pLM has been trained on fusion oncoprotein sequences and thus may not produce optimal representations for these proteins. In this work, we introduce\n                  <jats:bold>FusOn-pLM</jats:bold>\n                  , a novel pLM that fine-tunes the state-of-the-art ESM-2 model on fusion oncoprotein sequences. We specifically introduce a novel masked language modeling (MLM) strategy, employing a binding-site probability predictor to focus masking on key amino acid residues, thereby generating more optimal fusion oncoprotein-aware embeddings. Our model improves performance on both fusion oncoprotein-specific benchmarks and disorder prediction tasks in comparison to baseline ESM-2 representations, as well as manually-constructed biophysical embeddings, motivating downstream usage of FusOn-pLM embeddings for therapeutic design tasks targeting these fusions. We have made our model publicly available to the community at\n                  <jats:ext-link xmlns:xlink=\"http://www.w3.org/1999/xlink\" ext-link-type=\"uri\" xlink:href=\"https://huggingface.co/ChatterjeeLab/FusOn-pLM\">https://huggingface.co/ChatterjeeLab/FusOn-pLM</jats:ext-link>\n                  .\n                </jats:p>","journal":null,"year":null,"id":652730,"datarank":0.31191623125197543,"base_score":2.0794415416798357,"endowment":2.0794415416798357,"self_citation_contribution":0.31191623125197543,"citation_network_contribution":0.0,"self_endowment_contribution":0.31191623125197543,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":7,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":null,"is_data_producer":false,"deposit_databanks":null,"is_oa":false,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":null,"fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":1160951,"name":"Shrey Goel","orcid":null,"position":1,"is_corresponding":false},{"id":1160950,"name":"Kseniia Kholina","orcid":null,"position":2,"is_corresponding":false},{"id":1364060,"name":"Rishab Pulugurta","orcid":null,"position":3,"is_corresponding":false},{"id":1160952,"name":"Pranay Vure","orcid":null,"position":4,"is_corresponding":false},{"id":263226,"name":"Pranam Chatterjee","orcid":"0000-0003-3957-8478","position":5,"is_corresponding":false},{"id":1160946,"name":"Sophia Vincoff","orcid":null,"position":0,"is_corresponding":false}],"reference_count":0,"raw_metadata":{"has_enrichment":true,"resolved":true,"title":"FusOn-pLM: A Fusion Oncoprotein-Specific Language Model via Focused Probabilistic Masking","abstract":"<jats:title>Abstract</jats:title>\n                <jats:p>\n                  Fusion oncoproteins, a class of chimeric proteins arising from chromosomal translocations, drive and sustain various cancers, particularly those impacting children. Unfortunately, due to their intrinsically disordered nature, large size, and lack of well-defined, druggable pockets, they have been historically challenging to target therapeutically: neither small molecule-based methods nor structure-based approaches for binder design are strong options for this class of molecules. Recently, protein language models (pLMs) have demonstrated success at representing protein sequences with information-rich embeddings, enabling downstream design applications from sequence alone. However, no current pLM has been trained on fusion oncoprotein sequences and thus may not produce optimal representations for these proteins. In this work, we introduce\n                  <jats:bold>FusOn-pLM</jats:bold>\n                  , a novel pLM that fine-tunes the state-of-the-art ESM-2 model on fusion oncoprotein sequences. We specifically introduce a novel masked language modeling (MLM) strategy, employing a binding-site probability predictor to focus masking on key amino acid residues, thereby generating more optimal fusion oncoprotein-aware embeddings. Our model improves performance on both fusion oncoprotein-specific benchmarks and disorder prediction tasks in comparison to baseline ESM-2 representations, as well as manually-constructed biophysical embeddings, motivating downstream usage of FusOn-pLM embeddings for therapeutic design tasks targeting these fusions. We have made our model publicly available to the community at\n                  <jats:ext-link xmlns:xlink=\"http://www.w3.org/1999/xlink\" ext-link-type=\"uri\" xlink:href=\"https://huggingface.co/ChatterjeeLab/FusOn-pLM\">https://huggingface.co/ChatterjeeLab/FusOn-pLM</jats:ext-link>\n                  .\n                </jats:p>","is_dataset_classified":null,"base_score":2.0794415416798357,"endowment":2.0794415416798357,"datacite_reuse_total":0,"file_count":0,"downloads":0,"views":0,"has_version_chain":false,"is_dataset":false,"is_oa":false,"pmid":"38895377","pmcid":null,"openalex_id":"https://openalex.org/W4399332219","authors":[],"funders":[{"funder_name":"National Institutes of Health","grant_id":"1R21CA278468-01A1","title":"Programmable peptide-guided protein degradation"},{"funder_name":"NCI NIH HHS","grant_id":"R21 CA278468","title":null}],"total_grants":2,"fwci":null,"citation_percentile":null,"influential_citations":0,"citation_trend":[{"year":2023,"count":1},{"year":2024,"count":3},{"year":2025,"count":3}],"oa_status":"green","license":"cc-by-nc-nd","oa_locations":[{"url":"https://www.biorxiv.org/content/biorxiv/early/2024/06/04/2024.06.03.597245.full.pdf","host_type":"repository"},{"url":"https://www.biorxiv.org/content/biorxiv/early/2024/06/04/2024.06.03.597245.full.pdf","host_type":"repository"},{"url":"https://syndication.highwire.org/content/doi/10.1101/2024.06.03.597245","host_type":"publisher"},{"url":"https://doi.org/10.1101/2024.06.03.597245","host_type":"repository"},{"url":"https://pubmed.ncbi.nlm.nih.gov/38895377","host_type":"repository"},{"url":"https://www.ncbi.nlm.nih.gov/pmc/articles/11185609","host_type":"repository"},{"url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC11185609/pdf/nihpp-2024.06.03.597245v1.pdf","host_type":"repository"},{"url":"http://dx.doi.org/10.1101/2024.06.03.597245","host_type":""}],"fields_of_study":["Topic Modeling","Natural Language Processing Techniques","Machine Learning in Bioinformatics","0206 medical engineering","02 engineering and technology"],"mesh_terms":[],"keywords":["Masking (illustration)","Probabilistic logic","Fusion","Computer science","Natural language processing","Speech recognition","Artificial intelligence","Linguistics","Art","Philosophy","Literature","Article"],"sdg_mappings":[],"linked_datasets":[],"clinical_trials":[],"software_tools":[],"database_accessions":[],"source":"live","citation_network_status":"fetched"},"created_at":"2026-08-10T16:44:52.979849Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}