Long Short-Term Memory (LSTM)

Long short-term memory (LSTM) is a type of recurrent artificial neural network designed to learn from sequences of data in which important information may be separated by long gaps. It was introduced by Sepp Hochreiter and Jürgen Schmidhuber in 1997. An LSTM reads a sequence one step at a time and keeps an internal memory that it can update, preserve or erase, which lets it link an output to inputs seen many steps earlier.

The design became a workhorse of deep learning for speech, text and other sequential data. In finance it is applied to time series such as prices, volumes and order book states, and to text such as news. Its record there is mixed: the architecture is capable, but financial data are noisy and change character over time, and careless testing easily produces results that do not survive live trading.

A basic recurrent network passes a hidden state from each step to the next, so in principle it can remember the past. In practice it learns poorly over long sequences. Training works by sending error signals backwards through every step, and these signals shrink or grow exponentially along the way, a difficulty known as the vanishing or exploding gradient problem. The network ends up sensitive only to the last few inputs. The LSTM adds a separate memory path along which information, and error signals, can travel across many steps largely unchanged.

Each LSTM unit holds a cell state, its long-term memory, and uses three gates to control it. A gate is a small layer that outputs numbers between 0 and 1, acting as a dial between blocking and passing. The forget gate decides how much of the old memory to keep, the input gate decides how much new information to write, and the output gate decides how much of the memory to reveal as the unit's output. The memory update is:

• $c_t$ is the cell state at step $t$ and $c_{t-1}$ is the state from the previous step.
• $f_t$ is the forget gate, with values between 0 and 1.
• $i_t$ is the input gate, with values between 0 and 1.
• $\tilde{c}_t$ is the candidate new content computed from the current input.
• $\odot$ means multiplication element by element.

Suppose one memory element holds 2.0. If the forget gate is 0.9, the input gate is 0.5 and the candidate value is 0.4, the new state is $0.9 \times 2.0 + 0.5 \times 0.4 = 2.0$. The gate values are themselves computed from the current input and the previous output, using weights learned in training. The forget gate was not part of the original 1997 design and was added in later work. A simplified relative, the gated recurrent unit, merges some of the gates.

• Return and direction forecasts. Sequences of past returns, volumes and indicators are fed in to predict the next period's return or its sign, a form of time series forecasting.
• Volatility. LSTMs are used for volatility forecasting, alone or combined with econometric models such as GARCH.
• Order book modelling. At high frequency, sequences of quotes and trades are used to predict short-term price moves.
• Text. Before transformers, LSTMs were the standard model in natural language processing, including sentiment analysis of financial news.

Published comparisons with simpler methods such as ARIMA or linear regression give varied results. Gains, where found, tend to be small relative to the claims made for them, and they depend heavily on how the test was run.

An LSTM has many parameters and financial histories are short, a combination that invites overfitting. A network can memorise the training period and learn nothing that carries forward. Because markets are not stable in the statistical sense of stationarity, a pattern learned in one regime may vanish in the next.

Testing needs particular care. Ordinary random splitting of data into training and test sets lets the model see the future, so time-ordered methods such as walk-forward optimization are used instead. Scaling inputs using statistics from the whole data set, or building features from information not available at the time, introduces look-ahead bias. A common illusion is a chart in which predicted prices track actual prices closely. A model that simply repeats yesterday's price produces the same picture, so forecasts should be judged on returns, not price levels.

LSTMs are also hard to interpret, which complicates risk oversight, and they are slower to train than simpler models. In language tasks they have largely been replaced by the transformer architecture behind the large language model, though they remain in use for numerical sequences of moderate length.