Unsupervised learning (clustering, dimensionality reduction)

What it is

Unsupervised learning looks for structure in data that has no labels. Clustering groups similar examples together (customers with similar habits). Dimensionality reduction squeezes many features into a few that keep most of the information, which is also how you can draw a dataset on a flat plot.

What it is for

Exploring data you do not understand yet, finding groups without deciding them in advance, spotting unusual cases, compressing data, and making features for another model. Labels are expensive: this is the part of machine learning that works without them.

How to do it

The process, step by step. Libraries and tools change; these steps stay.

1. Choose and rescale the features, because distances depend on their scale.
2. To find groups, run a clustering method (for example k-means) for several numbers of groups.
3. Judge the groups with a score such as silhouette and, above all, by looking at a few examples of each: do they mean something to a human?
4. To see the data, reduce it to two dimensions (for example with PCA) and plot it, colouring by anything you do know.
5. Treat the result as a hypothesis, not a fact: different methods and settings give different groups.
6. If the groups are useful, name them and check them against real outcomes or with an expert.