A kernel, in machine learning, is a function that takes two data points and returns a number measuring how similar they are. Its value equals the result of mapping both points into a higher-dimensional space and comparing them there, but it is computed directly from the original inputs. This shortcut, known as the kernel trick, lets a method that can only draw straight boundaries fit curved ones. The best-known user is the support vector machine.
In trading research, a kernel-based classifier might label the next day as up or down from a set of indicators whose relationship to returns is not linear. The kernel's settings control how flexible the boundary is, and a setting that is too flexible leads to overfitting. The word has other senses. In statistics a kernel is the weighting function used to smooth a histogram into a density estimate, and in computing it is the core of an operating system.