Artificial Neural Network (ANN)

An artificial neural network (ANN) is a machine learning model built from many simple computing units, called neurons or nodes, arranged in layers and joined by weighted connections. Each unit takes numbers in, combines them, and passes a number on. By adjusting the weights during training, the network learns a mapping from inputs to outputs without that mapping being written by hand.

The design was loosely inspired by nerve cells in the brain, though modern networks are mathematical objects and not biological models. Networks with many layers are the subject of deep learning and underlie most current artificial intelligence. In finance they are used for fraud scoring, credit assessment, text analysis and price or volatility forecasting.

A basic network has an input layer, one or more hidden layers and an output layer. The input layer holds the features, for example the last twenty daily returns of a stock. Each hidden unit computes a weighted sum of the values it receives and applies an activation function, a simple nonlinear rule. The output layer produces the prediction: a number for a regression task or a set of probabilities when the network acts as a classifier.

The nonlinear activation is essential. Without it, any stack of layers would reduce to a single linear regression. With it, a network with enough hidden units can approximate a very wide class of functions.

The output of a single neuron is:

• $x_i$ are the inputs to the neuron.
• $w_i$ are the weights, one for each input.
• $b$ is the bias, a constant added to the sum.
• $f$ is the activation function. A common choice is the rectified linear unit (ReLU), which returns the sum if it is positive and zero otherwise.

Suppose a neuron has two inputs, 0.5 and −1.0, with weights 0.8 and 0.2 and a bias of 0.1. The weighted sum is (0.8 × 0.5) + (0.2 × −1.0) + 0.1 = 0.4 − 0.2 + 0.1 = 0.3. With ReLU the output is 0.3. Had the sum been negative, the output would have been zero.

Training starts with random weights. The network makes predictions on training examples, and a loss function measures how far they are from the correct answers. An algorithm called backpropagation works out how much each weight contributed to the error, and gradient descent moves every weight a small step in the direction that reduces it. The cycle repeats over many passes through the data.

The first artificial neuron was described by McCulloch and Pitts in 1943, and Frank Rosenblatt's perceptron followed in 1958. Backpropagation was brought to wide attention by Rumelhart, Hinton and Williams in 1986. Training large networks became practical in the 2010s, when the graphics processing unit made the arithmetic fast enough.

• Feedforward networks. Data moves one way, from input to output. This is the standard form for tables of features.
• Convolutional networks. Filters slide across the input to find local patterns. They were built for images and are also applied to order book snapshots.
• Recurrent networks. Connections loop back so the network keeps a memory of earlier inputs. The long short-term memory network is the best-known form and is used on sequences such as price series.
• Transformers. An attention mechanism lets the network weigh all parts of a sequence at once. The large language model is built on this design.

Card networks and banks run neural networks for fraud detection, where there are millions of labeled examples and patterns are stable enough to learn. Text models built on neural networks drive natural language processing of news and filings. Trading firms use networks for time series forecasting of returns, volume and volatility, for pricing complex derivatives quickly, and within reinforcement learning systems that plan order execution.

Neural networks need large amounts of data, and financial history is limited: a decade of daily prices is only about 2,500 observations per asset. With thousands or millions of weights, a network can memorize noise, so overfitting is the central danger. Defenses include holding data back, cross-validation that respects time order, penalties on large weights, and stopping training early.

Networks are also hard to interpret. It is difficult to say why a given output was produced, which is a problem where decisions must be explained to customers or regulators. For small tabular data sets, simpler models such as a decision tree ensemble often perform as well and cost less to run and to audit.