How Neural Networks Work

Neural networks, explained through the brain.

ai neural-networks

Our brain has about 80-90 billion neurons. Each one receives signals, decides if they matter, and passes a message forward. A neural network does the same thing with math.

Architecture

Layers

Our brain processes information in stages. Light hits my eyes 👁️. One region detects edges. Another recognizes shapes. Another says “squirrel!” 🐿️

A neural network has layers that work the same way. A network can have many. The inner ones are called “hidden.”

x1x2x3x4input layerh1h2h3h4hidden layery1y2output layer Input, hidden, output. Every neuron talks to the next layer.

Neurons

Inside each layer sit neurons. In our nervous system a neuron is a cell. It collects signals and fires when they are strong enough ⚡. In a network a neuron is a small math function. It adds up inputs and decides how strong to respond.

input1w1input2w2input3w3sum + biasadd it all upactivationhow hard to fireoutputweights Weighted inputs in, one decision out.

Weights

Not every signal matters equally.

We would (hopefully) pay more attention to smoke 💨 than to background music 🎵. Weights control this. They are numbers on every connection. They say “listen to this” or “ignore that.”

Biases

Our brain has baseline tendencies. Maybe I startle easily. Maybe I don’t. Maybe I favor right turns over left when I’m lost. These are Biases. A bias nudges a neuron to fire more, or less, whatever the input.

Activation Function

Each neuron also has an activation function. It is a rule for how strong to respond. Think of our pain threshold. A light tap does nothing. A hard knock makes us pull away. The activation function sets that threshold.

10inputoutputsmall inputs:neuron stays quietlarge inputs:neuron fires Quiet until it is not. The curve sets the threshold.

Forward Pass

A signal moves through every layer, input to output. That is the forward pass. It is the network “thinking,” not learning (yet).

Just signal in, answer out.

Learning

Loss Function

The network starts with random weights. It guesses wrong. The loss function measures how wrong, like the sting of touching a hot stove 🔥. Bigger mistake, bigger sting.

Backpropagation

Now it needs to fix itself. Backpropagation traces the error backward through the network. It works out which weights did the most damage. We do this too: “I should have checked the burner before touching.”

inputlayerlayerlayerpredictionforward pass — thinkingcompare tothe right answerlosslayerlayerlayeradjust weightsbackpropagation — learning Forward to a guess. Backward to a fix.

Gradient

The gradient tells each weight which direction to move and how far. Picture standing on a hill in fog 🌫️. You can’t see the bottom, but you feel the slope under your feet.

Gradient Descent

Gradient descent is the act of stepping downhill ⛰️ — adjusting every weight to reduce the error.

big gradient (steep)small gradient (flat)minimum (goal)lossweight adjustment → Steep at first, then smaller steps toward the bottom.

Learning Rate

The learning rate controls step size. Too big and you overshoot. Too small and you barely move. The right pace finds the bottom.

Repeat this thousands of times. The weights are trained and the network learns 🎓.


Summary

Structural Primitives 🏗️

Neural Network Human Brain
Layers Processing stages: senses - recognition - understanding
Neurons Brain cells that collect signals and fire
Weights How much attention we give each signal
Biases Tendencies: naturally jumpy or calm
Activation Function Our threshold for reacting
Forward Pass Thinking: signal in, answer out

Learning Mechanics 📚

Neural Network Human Brain
Loss Function The sting of being wrong
Backpropagation Tracing the blame — “I should have checked the burner”
Gradient Feeling which way is downhill and how steep
Gradient Descent Actually stepping downhill — adjusting to make fewer mistakes
Learning Rate How big my steps are — overcorrect and I panic, undercorrect and I never learn