Deep Learning

Deep learning is a branch of machine learning based on neural networks that have many layers of processing between input and output. The word deep refers to the number of layers. Each layer transforms the output of the one before it, so that the network builds up from simple patterns to complex ones: from pixels to edges to objects in an image, or from characters to words to meaning in text.

The approach is behind most of the advances in artificial intelligence since about 2012, including speech recognition, machine translation, image generation and conversational systems. In finance it is used most successfully where data is plentiful and unstructured, such as text and transaction records, and with more mixed results for predicting market prices.

A deep network is an artificial neural network with several hidden layers. Each layer computes:

• $h^{(l)}$ is the output of layer $l$, and $h^{(l-1)}$ is the output of the layer below it. The first input is the raw data.
• $W^{(l)}$ is the table of weights for layer $l$ and $b^{(l)}$ is its set of biases.
• $f$ is a nonlinear activation function applied to each element.

The weights and biases are the parameters learned in training. A layer that takes 100 inputs and has 50 units contains 100 × 50 = 5,000 weights and 50 biases, or 5,050 parameters. Modern networks stack many such layers and may hold millions or billions of parameters.

The key difference from earlier methods is that features are learned, not designed. In traditional modeling an analyst decides which inputs matter, such as a moving average or a valuation ratio. A deep network is given raw or lightly processed data and finds useful representations itself. This requires large data sets and heavy computation, normally on a graphics processing unit.

The underlying ideas are decades old, but deep networks were long considered too hard to train. Interest revived in the mid-2000s with work by Geoffrey Hinton and others. In 2012 a deep convolutional network known as AlexNet won the ImageNet image recognition contest by a wide margin, which turned the field toward deep learning. The transformer architecture, introduced by researchers at Google in 2017, became the basis of the large language model and of products such as ChatGPT.

Yoshua Bengio, Geoffrey Hinton and Yann LeCun received the 2018 Turing Award for their work on deep learning, and Hinton shared the 2024 Nobel Prize in Physics with John Hopfield for foundational work on neural networks.

• Convolutional networks. These scan the input with small filters and are suited to images and other grid-like data.
• Recurrent networks. These process sequences one element at a time while keeping a memory. The long short-term memory network is the main example.
• Transformers. These use attention to relate every part of a sequence to every other part and now dominate natural language processing.
• Autoencoders and generative models. These learn compact representations of data or produce new examples, a form of unsupervised learning.

Text is the clearest application. Deep language models read news, company filings, earnings call transcripts and social media, and convert them into scores for sentiment analysis or into summaries for analysts. Banks and card networks use deep networks for fraud detection, where billions of labeled transactions are available.

In trading, deep networks are applied to several tasks. Models trained on snapshots of the order book attempt to predict price moves over the next seconds. Sequence models are used for time series forecasting of volatility and volume. Networks are trained to approximate slow derivative pricing models so that prices and risk figures can be produced in real time, and combined with reinforcement learning to design hedging and order execution policies that account for trading costs. Funds also apply image models to alternative data such as satellite photographs.

Deep learning works best with very large data sets and stable patterns. Market return data offers neither. A single asset produces only a few thousand daily observations per decade, the predictable part of returns is tiny compared with the noise, and relationships change as traders adapt. A network with millions of parameters can fit such data perfectly and predict nothing, the problem of overfitting. Careful validation that respects time order, for example by cross-validation with gaps between training and test periods, is essential.

For tabular financial data of moderate size, simpler methods such as boosted decision tree ensembles frequently match or beat deep networks at much lower cost. Deep models are also difficult to interpret, which complicates model validation and regulatory review, and they are expensive to train and run. Their results can be sensitive to small changes in data or settings, so findings need to be reproduced before they are relied upon.