{"doi":"10.1088/2632-2153/abf984","title":"A free-energy principle for representation learning","abstract":"<jats:title>Abstract</jats:title>\n               <jats:p>This paper employs a formal connection of machine learning with thermodynamics to characterize the quality of learned representations for transfer learning. We discuss how information-theoretic functionals such as rate, distortion and classification loss of a model lie on a convex, so-called, equilibrium surface. We prescribe dynamical processes to traverse this surface under specific constraints; in particular we develop an iso-classification process that trades off rate and distortion to keep the classification loss unchanged. We demonstrate how this process can be used for transferring representations from a source task to a target task while keeping the classification loss constant. Experimental validation of the theoretical results is provided on image-classification datasets.</jats:p>","journal":"Machine Learning: Science and Technology","year":2021,"id":669808,"datarank":0.23318254458333207,"base_score":1.3862943611198906,"endowment":1.3862943611198906,"self_citation_contribution":0.20794415416798362,"citation_network_contribution":0.02523839041534846,"self_endowment_contribution":0.20794415416798362,"citer_contribution":0.02523839041534846,"corpus_percentile":null,"corpus_rank":null,"citation_count":3,"citer_count":2,"citers_with_citation_signal":1,"citers_with_endowment":1,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":null,"is_data_producer":false,"deposit_databanks":null,"is_oa":false,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":null,"fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":799956,"name":"Pratik Chaudhari","orcid":"0000-0003-4590-1956","position":1,"is_corresponding":false},{"id":1749381,"name":"Yansong Gao","orcid":null,"position":0,"is_corresponding":false}],"reference_count":0,"raw_metadata":{"has_enrichment":true,"resolved":true,"title":"A free-energy principle for representation learning","abstract":"<jats:title>Abstract</jats:title>\n               <jats:p>This paper employs a formal connection of machine learning with thermodynamics to characterize the quality of learned representations for transfer learning. We discuss how information-theoretic functionals such as rate, distortion and classification loss of a model lie on a convex, so-called, equilibrium surface. We prescribe dynamical processes to traverse this surface under specific constraints; in particular we develop an iso-classification process that trades off rate and distortion to keep the classification loss unchanged. We demonstrate how this process can be used for transferring representations from a source task to a target task while keeping the classification loss constant. Experimental validation of the theoretical results is provided on image-classification datasets.</jats:p>","is_dataset_classified":null,"base_score":1.3862943611198906,"endowment":1.3862943611198906,"datacite_reuse_total":0,"file_count":0,"downloads":0,"views":0,"has_version_chain":false,"is_dataset":false,"is_oa":false,"pmid":"19965766","pmcid":null,"openalex_id":"https://openalex.org/W3034198815","authors":[],"funders":[],"total_grants":0,"fwci":0.1163,"citation_percentile":0.37410113,"influential_citations":0,"citation_trend":[{"year":2020,"count":2},{"year":2021,"count":1}],"oa_status":"gold","license":"cc-by","oa_locations":[{"url":"https://iopscience.iop.org/article/10.1088/2632-2153/abf984/pdf","host_type":"journal"},{"url":"https://iopscience.iop.org/article/10.1088/2632-2153/abf984/pdf","host_type":"publisher"},{"url":"https://iopscience.iop.org/article/10.1088/2632-2153/abf984","host_type":"publisher"},{"url":"https://doi.org/10.1088/2632-2153/abf984","host_type":"journal"},{"url":"http://proceedings.mlr.press/v119/gao20a/gao20a.pdf","host_type":"conference"}],"fields_of_study":["Model Reduction and Neural Networks","Neural Networks and Applications","Gaussian Processes and Bayesian Inference"],"mesh_terms":[],"keywords":["Traverse","Distortion (music)","Computer science","Artificial intelligence","Representation (politics)","Process (computing)","Rate–distortion theory","Quality (philosophy)","Regular polygon","Contextual image classification","Pattern recognition (psychology)","Surface (topology)","Energy (signal processing)","Machine learning","Transfer of learning","Image (mathematics)","Mathematics","Statistics","Physics"],"sdg_mappings":[{"sdg_number":0,"sdg_label":"Affordable and clean energy"}],"linked_datasets":[],"clinical_trials":[],"software_tools":[],"database_accessions":[],"source":"live","citation_network_status":"fetched"},"created_at":"2026-08-15T04:49:53.744954Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}