Artificial Intelligence
Training vs Inference
A machine-learning model has two phases in its life: learning from data (training), and then being used to answer new questions (inference). They differ in cost, timing, and purpose.
| Aspect | Training | Inference |
|---|---|---|
| What happens | The model learns patterns from data | The trained model makes a prediction |
| When | Once (or occasionally, to update) | Every time the model is used |
| Cost | Expensive and slow (lots of compute) | Cheaper and fast per use |
| Data | Needs large training datasets | Works on a single new input |
| Analogy | Studying for an exam | Answering a question on the exam |
When to use training
Training happens when a model is built or updated — the compute-heavy stage where it learns from data. It’s done relatively rarely.
When to use inference
Inference happens every time someone uses the model to get an answer — the fast, per-request stage that runs constantly in a live product.
Frequently asked questions
- Why does training cost so much more than inference?
- Training processes huge datasets many times over to adjust the model, which takes enormous compute. Inference runs the finished model once per input, so each prediction is comparatively cheap — though at massive scale, inference costs add up too.
- Does the model keep learning during inference?
- Usually no. Most deployed models are "frozen" after training and don’t learn from the inputs they see at inference time. Updating the model requires a new round of training.
- Where does this matter for building AI?
- It shapes cost and design: training is a big upfront investment, while inference is the ongoing running cost of a live product. Optimising inference (speed and cost per request) is a major focus when deploying AI at scale.