Black box trading is an informal name for algorithmic trading, used when the emphasis is on the secrecy or opacity of the rules that decide what to buy and sell. Data goes in and orders come out, and the logic in between is not visible to clients, counterparties or, in some cases, the people who operate the system. The main entry describes how such systems are built, tested and regulated.
The phrase is used in two ways. In the first, the rules are simply confidential: a proprietary trading firm or hedge fund keeps its models secret because they are its main asset. In the second, the rules are hard to explain even for their authors. Strategies built with machine learning may contain thousands of fitted parameters with no simple meaning.
Opacity is a concern for investors and supervisors, since a model that cannot be explained is hard to check. Regulators do not prohibit black box strategies, but they require the firm to understand, test and control what it runs, and a risk management system must be able to stop it.