Gorch
From-scratch autodiff and neural networks in NumPy
An educational automatic-differentiation and neural-network library written entirely in NumPy — no PyTorch, no TensorFlow. Every backward pass is hand-written, readable, and verified against finite differences.
What is it?
Gorch is a from-scratch deep-learning library built for a Neural Control course at K. N. Toosi University of Technology. It shows exactly how reverse-mode automatic differentiation, optimizers, and network layers work under the hood — because nothing is hidden behind a compiled backend.
Each differentiable operation wraps its inputs in a dedicated *Backward node
that implements the local gradient rule by hand. Backpropagation is then just a
walk down that explicit graph:
import numpy as np
import gorch
x = gorch.Tensor(np.array([[0.0], [1.0]]), requires_grad=True)
W = gorch.Tensor(np.random.randn(1, 1), requires_grad=True)
y = (x @ W).tanh()
y.sum().backward()
print(W.grad.value) # shape matches W.value
Hand-written autograd
Dedicated backward nodes per operation, no hidden tape.
14 optimizers
SGD, Adam, RMSprop, Nesterov, PID, Levenberg–Marquardt and more.
Control tools
KalmanFilter, EKFOptimizer, RLS and a hand-rolled jacobian.
Verified grads
Every gradient is checked against finite differences in the test suite.
From the blog
- How reverse-mode autodiff works in Gorch —
what a
*Backwardnode is, and why grads travel through the graph transposed.
Getting started
git clone https://github.com/Armangb1/pygorch.git
cd pygorch
pip install -e .
python examples/xor_mlp.py # MLP learning the XOR function
python examples/neural_control.py # Kalman filtering + EKF/RLS identification
See the Quickstart page for the full API tour, the Examples page for what the demos actually print, and the Architecture page for how the library is put together.