Supervised learning (regression, classification, trees)

What it is

Supervised learning teaches a program from examples that come with the right answer. Each example is an input and a label. If the label is a number (a price, a temperature) the task is regression; if it is a category (spam or not spam, cat or dog) it is classification. Decision trees, linear models and support vector machines are classic ways to do it.

What it is for

Any time you have past examples with known outcomes and want to predict the outcome of new cases: will this customer leave, is this transaction fraud, what will this house sell for. It is the most widely used kind of machine learning, and a good baseline before reaching for something heavier.

How to do it

The process, step by step. Libraries and tools change; these steps stay.

1. Say precisely what you want to predict (the label) and what information you have at prediction time (the features).
2. Collect examples with the correct label and clean them: missing values, duplicates, wrong labels.
3. Split the data into a training set and a test set, and keep the test set locked away until the end.
4. Try a simple model first (for example a decision tree or logistic regression, e.g. with scikit-learn) and measure it on the test set with a metric that matches the goal, such as accuracy or mean error.
5. If it is too simple, try a richer model; if it only does well on training data, it is overfitting: simplify it or add data.
6. Use it on new inputs, and keep checking that real-world data still looks like the data it learned from.