Decision Tree

A decision tree is a predictive model shaped like a flowchart. Starting from a single point, the root, it asks a question about one feature of the input, such as "is the debt-to-income ratio above 40 percent?" The answer sends the case down one of two branches to another question, and so on, until it arrives at an end point, called a leaf, that holds the prediction. Trees are used for classification, where the output is a category, and for regression, where it is a number.

Decision trees are among the most widely used methods in machine learning, both alone and as the building blocks of more powerful models. In finance they are applied to credit decisions, fraud screening and the prediction of market outcomes. Their main attraction is that the reasoning behind any prediction can be read directly from the tree.

A tree is grown from training data by repeated splitting. At each node the algorithm examines every feature and every possible cut-off, and chooses the split that best separates the outcomes. For a classification tree, best means that the two resulting groups are as pure as possible, each dominated by one class. The same procedure is then applied to each group, and the tree grows until a stopping rule is met: a maximum depth, a minimum number of cases in a leaf, or no further gain.

The standard algorithm is known as CART, for classification and regression trees, described by Leo Breiman and colleagues in 1984. A leaf of a classification tree predicts the most common class among the training cases that reached it, and the share of that class serves as a probability. A leaf of a regression tree predicts the average value of its cases.

A common measure of how mixed a node is, called Gini impurity, is:

• $G$ is the impurity of the node. It is zero when all cases belong to one class.
• $p_k$ is the proportion of cases in the node that belong to class $k$.

Suppose a node holds 100 trading days, of which 60 were followed by a rise and 40 by a fall. Its impurity is 1 − (0.6² + 0.4²) = 1 − (0.36 + 0.16) = 0.48. A candidate split on whether volume was above average sends 50 days to each side. The left group has 45 rises and 5 falls, for an impurity of 1 − (0.81 + 0.01) = 0.18. The right group has 15 rises and 35 falls, for 1 − (0.09 + 0.49) = 0.42. The weighted average is (0.5 × 0.18) + (0.5 × 0.42) = 0.30. The split reduces impurity by 0.48 − 0.30 = 0.18, and the algorithm picks the split with the largest such reduction.

Trees handle relationships that are not straight lines and capture interactions between features without being told to look for them. A rule such as "high volume matters only when volatility is low" emerges naturally from two successive splits, where a linear regression would need the interaction specified in advance. Trees need little data preparation: features do not have to be scaled, and outliers in the inputs have limited effect because only the ordering of values matters.

The chief weakness is overfitting. Left to grow, a tree will keep splitting until each leaf contains a handful of cases, fitting the noise in the training data. Such a tree performs well on the data it was built from and poorly on new data. Trees are also unstable: a small change in the training set can produce a quite different tree.

Two approaches limit the problem. One restricts the tree, by capping its depth, requiring a minimum number of cases per leaf, or pruning back branches that add little. The best settings are found by cross-validation. The other combines many trees. In ensemble learning, a random forest averages hundreds of trees, each built on a different random sample of the data and features, while gradient boosting builds trees in sequence, each correcting the errors of those before it. These ensembles are far more accurate than a single tree and are among the strongest methods for tabular data.

Lenders use tree-based models alongside the traditional credit score to predict default. Payment companies use them in fraud detection, where a tree's splits resemble the hand-written rules that analysts already use. In trading, a tree may serve as a classifier that predicts whether the next period's return will be positive, using features such as past returns, volatility, volume and valuation ratios.

Market data makes the overfitting problem severe, since the signal is faint and conditions shift. Financial practitioners therefore favor shallow trees, large ensembles and strict out-of-sample tests, and they check that a model trained on one period still works in backtesting on a later one.

The term also describes a diagram used in decision analysis, in which branches represent choices and uncertain events, and each path is valued by its probability and payoff to find the option with the highest expected value. The price tree of the binomial option pricing model is a third, unrelated use of a tree structure in finance.