What it is
A nonparametric model does not squeeze what it learns into a fixed number of settings. It keeps the examples (or a summary of them) and lets its complexity grow with the data. The simplest case is nearest neighbours: to classify a new point, find the most similar known examples and let them vote. Kernel methods work on the same idea of similarity between examples.
What it is for
Small or medium datasets where the shape of the answer is not obvious, and cases where you want to be able to say "it decided this because it looks like these known examples". It is also a good sanity check: if nearest neighbours already does well, the problem is mostly about having good features.
How to do it
- Turn each example into a list of numbers (features) and rescale them so that no single feature dominates the distance.
- Choose what "similar" means: usually a distance such as the straight-line one, or a kernel that scores similarity.
- Choose how many neighbours vote (k). Small k follows the data closely and is noisy; large k is smoother but can blur real differences.
- Pick k on a validation set, not on the test set.
- To predict, find the k closest stored examples to the new one and take the majority label or the average value.
- If it is slow on a large dataset, use an index structure for faster neighbour search, or move to a parametric model.