{"doi":"10.1093/aje/kwae380","title":"A primer on neural networks","abstract":"Neural networks are a class of machine-learning algorithms inspired by the structure of the nervous system.1 An important result underlying the utility of neural networks is the universal approximation theorem, which states that there exists a sequence of neural network that can closely approximate any smooth function. Practically, this result means neural networks provide a way to more flexibility model data relative to tools more familiar to epidemiologists (eg, generalized linear models [GLMs]). Despite their popularity, how neural networks operate can be mysterious. Here, we review the basics of neural networks. Consider predicting high-density lipoprotein cholesterol (ie, |$Y$|⁠) given the following covariates: age (years; |${X}_1$|⁠), gender (male, female; |${X}_2$|⁠), body weight (kilograms; |${X}_3$|⁠), and height (centimeters; |${X}_4$|⁠). Data from the National Health and Nutrition Examination Survey 2017-2018 are used and restricted to complete cases 18 years or older (⁠|$n=5092$|⁠). Three components define a neural network: structure, activation functions, and loss function. A neural network is a graph consisting of nodes (or neurons) connected by directed edges. Nodes are nested within layers, with layers being divided into an input layer, hidden layer(s), and an output layer. The input layer consists of the predictors (ie, |${X}_1,{X}_2,{X}_3,{X}_4$|⁠). Hidden layers are intermediates between the input and output. The structure of the hidden layers (eg, number of nodes, number of hidden layers) is specified by the analyst. Finally, the output layer returns the predicted outcomes. Here, the output layer is a single node. Neural networks are often referred to by their number of hidden layers (eg, a 3-layer neural network has 2 hidden layers, with the output being the third layer). Each edge in a network is associated with a “weight.” Inputs flow through the layers (Figure 1). Inputs from nodes that directly point to a node are combined using a weighted sum, using the edge’s associated weight. To this weighted sum, another value called “bias” is added. This value is then transformed via an activation function (discussed next). For intuition, each node can be thought as a GLM, where the inputs from the previous layer are the independent variables, bias is the intercept, weights are the coefficients, and the activation function is the link function. The transformed value becomes the input to the next layer. Therefore, one can view a neural network as a multistage regression model, where regression models feed into other regression models. Because weight and bias have a multitude of meanings in epidemiology, these are referred to as parameters hereafter. Schematic diagram of information flow through a neural network. The variable |$\\alpha, \\beta, \\mathrm{and}\\ \\gamma$| are parameters of the neural network; and |$a(z)$| represents the activation function. Note that the intercept or bias term is not traditionally depicted as a node or with arrows in neural networks. Therefore, the number of parameters in a neural network is the |$\\left({N}_I+1\\right)+{\\sum}_{h\\in H}\\left({N}_h+1\\right)$|⁠, where |${N}_I$| is the number of nodes in the input layer, |$H$| is the set of different hidden layers, and |${N}_h$| is the number of nodes in the |$h$| hidden layer. Activation functions are akin to the link function in a GLM. Some activation functions familiar to epidemiologists include the identity and logistic (ie, sigmoid) functions. However, there is a variety of activation functions without a GLM equivalent, such as the rectified linear unit (Appendix S1). Selection of an activation function for the output layer shares a similar motivation to the GLM link functions (ie, using a logistic function for a binary outcome), but the motivation for hidden layers differs. Hidden layers use nonlinear activation functions. By using nonlinear activation functions, a neural network can be used to model nonlinear relationships between predic","journal":"American Journal of Epidemiology","year":2024,"id":504274,"datarank":0.0,"base_score":0.0,"endowment":0.0,"self_citation_contribution":0.0,"citation_network_contribution":0.0,"self_endowment_contribution":0.0,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":0,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.9568,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2024-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":369131,"name":"Ashley I. Naimi","orcid":"0000-0002-1510-8175","position":1,"is_corresponding":false},{"id":289327,"name":"Paul N. Zivich","orcid":"0000-0002-9932-1095","position":0,"is_corresponding":true}],"reference_count":4,"raw_metadata":null,"created_at":"2026-07-19T02:10:35.597333Z","pmid":"39358996","pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}