Data that was made, not collected
Synthetic data is produced by a program or by an AI rather than gathered from reality. Training needs enormous quantities of data, real data is finite, and collecting it carries cost and rights problems — which is why synthetic data keeps gaining ground.
Why it is used
- Scarcity: high-quality text on the web is said to be close to exhausted, making data supply an industry problem
- Privacy: training on data that reproduces statistical properties without using real personal information
- Rare cases: simulating large volumes of situations that seldom occur, such as accidents for self-driving training
- Rights: avoiding disputes over the use of copyrighted material
AI teaching AI
Recent LLM development commonly has a strong model generate large volumes of problems and model answers, which then train a new model. Combined with distillation, it has become a standard way to raise the capability of small models.
Training repeatedly on AI-generated data carries the risk of model collapse, where errors and biases are amplified and quality degrades. Getting the mix of real and synthetic data right is where developer skill shows.
What to take away
If the data is missing, make it — that idea is changing assumptions in AI development. When you see training on synthetic data in the news, it is usually addressing scarcity, privacy, or cost.
Quality control for synthetic data is becoming a specialty in itself, with techniques for generating data and techniques for inspecting it developing together. It is a good marker of how far the competitive front has moved toward data.
Closer to home, computer-generated faces have long been used to train face recognition — synthetic data has been quietly at work inside products for years.