Large language models

Level: Intermediate

What it is

A large language model (LLM) is a language model, almost always a transformer, with a very large number of parameters, trained on a huge amount of text. Size brings unexpected skills: with a few examples in the prompt it can do tasks it was never explicitly trained for. Chat assistants are LLMs that have been further trained to follow instructions.

What it is for

Writing and editing, summarising, translating, explaining, answering questions about a text you give it, writing and explaining code, and acting as the reasoning core of agents. It is a tool for drafts and exploration, not a source of truth: it can state false things with confidence.

How to do it

The process, step by step. Libraries and tools change; these steps stay.

Pick how you will reach one: a chat app for exploring, an API for building into a product, or an open model you run yourself when privacy or cost demand it.
Write the prompt as a brief: the role or goal, the context, the exact task, the format you want, and an example if the format matters.
Give it the material instead of trusting its memory: paste the document or connect your sources (see Retrieval-augmented generation).
Iterate: read the answer, say what was wrong, and ask again; split big jobs into steps.
Verify anything important against a reliable source, and never paste secrets or personal data into a service you do not control.
Measure, do not guess: for a real application, collect test cases and re-run them whenever you change the prompt or the model.

Lately

2026-10-13 Gemini 4 Argon is generally available
2026-10-07 Claude Haiku 5.5: a small model for high-volume work

Timeline

2018BERT: pre-trained deep bidirectional transformers for language understanding (Devlin et al.). source
2020GPT-3: a 175-billion-parameter language model that learns tasks from a few examples (Brown et al.). source

Sources

AIMA 4th ed., § 24.5 Verified 2026-11-09
Brown et al. — Language Models are Few-Shot Learners (GPT-3) (2020) Verified 2026-11-09
Devlin et al. — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018) Verified 2026-11-09

Builds on

Transformer architecture
Large pre-trained language models such as BERT and GPT-3 are transformer language models scaled up.
Language models
GPT-3 shows that scaling up language models greatly improves few-shot performance.

Leads to

Fine-tuning & alignment
InstructGPT starts from GPT-3 and fine-tunes it with human feedback so it follows what users actually ask.
Retrieval-augmented generation
RAG pairs a pre-trained language model (its parametric memory) with a searchable external index, to answer knowledge-intensive tasks.
LLM-based agents & tool use
Agents extend a language model so it can act: ReAct interleaves reasoning and actions, Toolformer teaches the model to call tools through APIs.
Chain-of-thought reasoning
Chain-of-thought reasoning emerges in sufficiently large language models when prompted with a few worked examples.
Scaling laws
Scaling laws describe how language-model performance improves with model size, dataset size and training compute.
The limits of AI
Stochastic Parrots asks whether language models can be too big and what the risks are.

Also in

Natural language processing