{"doi":"10.1162/neco.2006.18.7.1527","title":"A Fast Learning Algorithm for Deep Belief Nets","abstract":"We show how to use \"complementary priors\" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.","journal":"Neural Computation","year":2006,"id":1430,"datarank":21.192180119681165,"base_score":9.699778913860525,"endowment":9.699778913860525,"self_citation_contribution":1.454966837079079,"citation_network_contribution":19.737213282602085,"self_endowment_contribution":1.454966837079079,"citer_contribution":19.737213282602085,"corpus_percentile":97.5,"corpus_rank":527,"citation_count":16313,"citer_count":185,"citers_with_citation_signal":185,"citers_with_endowment":185,"datacite_reuse_total":0,"is_dataset":false,"is_oa":false,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2006-07-01","authors":[{"id":17134,"name":"Simon Osindero","orcid":null,"position":1,"is_corresponding":false},{"id":17135,"name":"Yee-Whye Teh","orcid":null,"position":2,"is_corresponding":false},{"id":17136,"name":"Yee‐Whye Teh","orcid":null,"position":3,"is_corresponding":false},{"id":16946,"name":"Geoffrey E. Hinton","orcid":null,"position":0,"is_corresponding":true}],"reference_count":13,"raw_metadata":{"citation_network_status":"fetched"},"created_at":"2026-03-01T18:20:47.508186Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}