Building a neural network from scratch in C++
Writing the data, training, and inference path directly was a practical way to explore how a neural network works internally.
Trace one prediction.
A handwritten-digit neural network implemented from first principles in C++ as a hands-on project for fun.
- 00.98
- 10.00
- 20.05
- 30.00
- 40.00
- 50.03
- 60.00
- 70.00
- 80.00
- 90.06
What this unit learned to notice
Red weights raise the activation and navy weights suppress it. Select another hidden unit to inspect how its input weights influence the prediction.
void neuralnetwork::train(const Matrix<double>& inputs, const Matrix<double>& targets)
{
Matrix<double> hidden_outputs = inputs.dot(wih).sigmoid();
Matrix<double> final_outputs = hidden_outputs.dot(who).sigmoid();
Matrix<double> output_errors = targets - final_outputs;
Matrix<double> hidden_errors = output_errors.dot(who.transpose());
who = who + hidden_outputs.transpose()
.dot(output_errors.mul(final_outputs.sigmoidprime()))
.scale(lr);
wih = wih + inputs.transpose()
.dot(hidden_errors.mul(hidden_outputs.sigmoidprime()))
.scale(lr);
}The whole backward pass: error at the output, error carried back through the output weights, then one gradient step on each weight matrix. No framework, no autograd — sigmoidprime is the derivative written by hand. main.cpp builds it with neuralnetwork n(784, 100, 10, 0.3), and the interaction above runs those same trained weights, normalising its input with the identical (v / 255) × 0.99 + 0.01 transform — so the browser and the console program agree on what a pixel means.
A handwritten-digit classifier implemented from first principles in C++, without a machine-learning framework. It parses and normalizes MNIST data, trains a 784 → 100 → 10 network, queries predictions, and renders digits through a Win32 console interface.
- Chose
- Wrote the data loading, training, and inference path by hand in C++
- Instead of
- A few lines against an existing framework
- Because
- Implementing the forward and backward pass directly was the entire point. The goal was to see the mechanism, not to obtain a classifier.
- C++
- Linear algebra
- Backpropagation
- MNIST
- Win32 console