28 October 2019

Curse of Dimensionality

As you increase the number of input features, the combination of inputs can grow exponentially. As the combinations grows, each training sample covers a smaller percentage of possibilities. The result being, as you add features, you need to increase the size of your training set, which may be exponentially. As the number of dimensions goes up, the model must train on significantly more data in order to learn an accurate representation of the input space.