Recurrent networks

What it is

A recurrent neural network (RNN) reads a sequence one item at a time and keeps a hidden state, a kind of memory of what it has seen so far. LSTM is a version designed to keep information over longer stretches.

What it is for

Data where order matters: text, speech, sensor readings, music. Transformers now handle most language tasks, but RNN and LSTM ideas explain why sequences are hard, and they can still be a light choice for small streams of data.

How to do it

The process, step by step. Libraries and tools change; these steps stay.

1. Turn each item into numbers (words become vectors, see Word embeddings).
2. Feed the sequence step by step: at each step the network mixes the new item with its memory.
3. Read the output you need: one answer at the end (for example the sentiment of a review) or one per step (for example a tag for every word).
4. Train with backpropagation through the steps in time.
5. For long sequences prefer an LSTM or a transformer: a plain RNN has trouble remembering the beginning.