{"doi":"10.1101/2025.03.27.645853","title":"Scalable high-performance single cell data analysis with BPCells","abstract":"<jats:title>Abstract</jats:title>\n                <jats:p>The growth of single-cell datasets to multi-million cell atlases has uncovered major scalability problems for single-cell analysis software. Here, we present BPCells, a package for high-performance single-cell analysis of RNA-seq and ATAC-seq datasets. BPCells uses disk-backed streaming compute algorithms to reduce memory requirements by nearly 70-fold compared to in-memory workflows with little to no loss of execution speed. BPCells also introduces high-performance compressed formats based on bitpacking compression for ATAC-seq fragment files and single-cell sparse matrices. These novel compression algorithms help to accelerate disk-backed analysis by reducing data transfer from disk, while providing the lowest computational overhead of all compression algorithms tested. Using BPCells, we perform normalization and PCA of a 44 million cell dataset on a laptop, demonstrating that BPCells makes working with the largest contemporary single-cell datasets feasible on modest hardware, while leaving headroom on servers for future datasets an order of magnitude larger.</jats:p>","journal":null,"year":null,"id":599972,"datarank":0.41588830833596724,"base_score":2.772588722239781,"endowment":2.772588722239781,"self_citation_contribution":0.41588830833596724,"citation_network_contribution":0.0,"self_endowment_contribution":0.41588830833596724,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":15,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":null,"is_data_producer":false,"deposit_databanks":null,"is_oa":false,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":null,"fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":15493,"name":"William J. Greenleaf","orcid":"0000-0003-1409-3095","position":1,"is_corresponding":false},{"id":621773,"name":"Benjamin Parks","orcid":"0000-0002-0261-7472","position":0,"is_corresponding":false}],"reference_count":0,"raw_metadata":{"has_enrichment":true,"resolved":true,"title":"Scalable high-performance single cell data analysis with BPCells","abstract":"<jats:title>Abstract</jats:title>\n                <jats:p>The growth of single-cell datasets to multi-million cell atlases has uncovered major scalability problems for single-cell analysis software. Here, we present BPCells, a package for high-performance single-cell analysis of RNA-seq and ATAC-seq datasets. BPCells uses disk-backed streaming compute algorithms to reduce memory requirements by nearly 70-fold compared to in-memory workflows with little to no loss of execution speed. BPCells also introduces high-performance compressed formats based on bitpacking compression for ATAC-seq fragment files and single-cell sparse matrices. These novel compression algorithms help to accelerate disk-backed analysis by reducing data transfer from disk, while providing the lowest computational overhead of all compression algorithms tested. Using BPCells, we perform normalization and PCA of a 44 million cell dataset on a laptop, demonstrating that BPCells makes working with the largest contemporary single-cell datasets feasible on modest hardware, while leaving headroom on servers for future datasets an order of magnitude larger.</jats:p>","is_dataset_classified":null,"base_score":2.639057329615259,"endowment":2.639057329615259,"datacite_reuse_total":0,"file_count":0,"downloads":0,"views":0,"has_version_chain":false,"is_dataset":false,"is_oa":false,"pmid":"40236161","pmcid":null,"openalex_id":"https://openalex.org/W4409055145","authors":[],"funders":[],"total_grants":0,"fwci":null,"citation_percentile":null,"influential_citations":0,"citation_trend":[{"year":2025,"count":6},{"year":2026,"count":7}],"oa_status":"closed","license":"http://creativecommons.org/licenses/by-nc/4.0/","oa_locations":[{"url":"https://syndication.highwire.org/content/doi/10.1101/2025.03.27.645853","host_type":"publisher"},{"url":"https://doi.org/10.1101/2025.03.27.645853","host_type":"repository"},{"url":"https://pubmed.ncbi.nlm.nih.gov/40236161","host_type":"repository"},{"url":"https://www.ncbi.nlm.nih.gov/pmc/articles/11996304","host_type":"repository"}],"fields_of_study":["Single-cell and spatial transcriptomics","Advanced Data Storage Technologies","Scientific Computing and Data Management"],"mesh_terms":[],"keywords":["Computer science","Laptop","Scalability","Normalization (sociology)","Workflow","Server","Overhead (engineering)","Data compression","Parallel computing","Operating system","Algorithm","Database"],"sdg_mappings":[],"linked_datasets":[],"clinical_trials":[],"software_tools":[],"database_accessions":[],"source":"live","citation_network_status":"fetched"},"created_at":"2026-07-29T11:52:05.030675Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}