Inside a Neural Network

In the last lesson we saw that deep learning stacks many layers to learn features on its own. But what is actually inside those layers? Opening the box reveals a surprisingly simple repeated part: the artificial neuron. Once you understand one neuron and how neurons connect, a giant network is just more of the same idea, repeated at scale.

This mental model pays off everywhere. When you later hear about model size, training, or why networks sometimes behave oddly, you will be able to picture signals flowing through weighted connections rather than treating the system as a black box. The parts are few and the logic is intuitive.

The artificial neuron

An artificial neuron is a tiny decision-making unit. It receives several numbers as input, and each input arrives through a connection that has a weight, a number saying how important that input is. The neuron multiplies each input by its weight, adds the results together into a single running total, and then passes that total through an activation step to decide its output. That is the whole neuron: gather, weigh, add, activate.

Think of a neuron deciding whether to recommend an outing. Sunshine, free time, and cost each feed in, each with a different weight reflecting how much it matters. The neuron sums the weighted evidence and produces a signal.

Weights and activation

Weights are the adjustable knobs of a network. A large positive weight means an input strongly pushes the neuron toward firing; a negative weight pushes against it. The activation function is what happens after the sum: it decides how strongly, if at all, the neuron passes its signal onward. A common intuition is a threshold, quiet until the evidence crosses a line, then active. Activation adds a gentle nonlinearity, which is what lets stacked layers capture rich, non-obvious patterns rather than just straight-line relationships.

Layers: input, hidden, and output

Neurons are organized into layers. The input layer simply holds the raw data, for example the brightness of each pixel in an image. One or more hidden layers sit in the middle; these are where the network builds up its learned features, each layer combining the signals from the one before. The output layer produces the final answer, such as the probability that an image shows a cat. Deep learning gets its name from having many hidden layers between input and output.

Picture a small network that decides whether a photo is a smiley face. The input layer receives the pixel values. The first hidden layer has neurons that light up for short edges and curves. Their signals flow to a second hidden layer whose neurons combine edges into parts, one neuron responding to a curved mouth, another to two dots that could be eyes.

Those part-detecting neurons feed the output neuron. If the mouth-curve neuron and both eye neurons fire strongly, their weighted signals push the output neuron past its activation threshold, and it reports face. Notice the direction: data entered at the input, moved forward layer by layer, and emerged as a decision. That single left-to-right journey is called a forward pass.

It is tempting to imagine each neuron holds a clear, nameable concept, one neuron for eyes, one for smiles. In real networks it rarely works so cleanly. A single learned feature is often spread across many neurons, and one neuron may contribute to several different features at once. So while our smiley-face example is a helpful picture of forward flow, do not assume you could open a real network and find one tidy neuron labeled mouth. The structure is orderly; the meaning of individual neurons usually is not.

The artificial neuron has a real origin. In 1958 the psychologist Frank Rosenblatt, working at the Cornell Aeronautical Laboratory, introduced the perceptron, an early model of a single artificial neuron that could learn to classify simple patterns by adjusting its weights. He later built the Mark I Perceptron, a physical machine that learned to distinguish basic shapes from images.

The perceptron caused great excitement, though it had real limits: a single perceptron could only separate patterns that fall into simple, straight-line divisions. Progress stalled for years, in part because of those limits. The revival came from an idea our lesson already showed: stack many such units into layers, with hidden layers in between. Modern neural networks are, at heart, enormous arrangements of perceptron-like neurons, exactly the building block Rosenblatt described.

• A neuron is not a mysterious brain cell; it is a simple unit that weighs inputs, adds them, and activates.
• Weights are not fixed; they are the adjustable knobs a network tunes, though how they are tuned is a separate topic.
• Hidden layers are not hidden from you for secrecy; they are simply the layers between input and output.
• Individual neurons usually do not map to single, nameable concepts; features are spread across many neurons.

Draw three columns of circles for an input, a hidden, and an output layer, and connect them with lines. Pick one output neuron and trace backward every connection that feeds it, then describe in one sentence how a strong input signal could travel forward and make that output neuron fire. Label which layer does what.

Think Like a Maester: A vast network is nothing more than one simple neuron, repeated and connected many times over.

1. What three basic operations does an artificial neuron perform on its inputs?
2. What role do weights play in a neuron?
3. Name the three kinds of layers and say briefly what each does.
4. What is a forward pass?
5. Who introduced the perceptron, and in what year?

Inside a neural network is one repeated building block: the artificial neuron, which weighs its inputs, adds them up, and activates to pass a signal onward. Weights set the importance of each input, and activation decides how strongly a neuron fires. Neurons are arranged into an input layer that holds the data, hidden layers that build up learned features, and an output layer that gives the answer. Signals travel forward from input to output in a forward pass. This design traces back to Frank Rosenblatt's 1958 perceptron; today's deep networks simply stack vast numbers of such units into many layers.

• What Is Deep Learning?
• How Neural Networks Learn