A mesh of small calculations
A neural network is a computing structure that imitates how neurons connect in the brain. Many small units are wired into a mesh, passing signals along until an answer emerges. Deep learning and the LLM inside ChatGPT are both built this way.
The rough mechanics
The basic shape is input layer, hidden layers, output layer. For a model identifying an animal in a photo, the input layer receives the image, the hidden layers read features in stages, and the output layer produces something like a 90% probability of cat.
Each connection carries a number called a weight, and training is the process of nudging those weights to fit large amounts of data. The finished set of weights is what people call the parameters.
Deep networks are deep learning
Training a network with many stacked hidden layers is deep learning. Greater depth captures more complex features, which is what made image recognition and natural text generation possible.
How much do you need to know?
Not the mathematics. It is enough to picture a mesh of calculations modeled on the brain, with connection strengths adjusted by training. When an article mentions a new neural network architecture, read it as a discussion of the blueprint for an AI’s reasoning — the Transformer, which produced LLMs, being the most consequential example.
Parameter count, the total number of connections, is the usual proxy for a model’s size, running into the billions or hundreds of billions for LLMs. That is still short of the connections in a human brain, but scaling the mesh has been the main engine of recent progress.