Jay Singh
A blog for ongoing research
latest dispatches
Splitting a Model Across Machines
april 2026
A model too large for one chip gets split along three axes, each technique answering a different bottleneck, until the binding constraint stops being memory and becomes failure itself.
What the Loss Curve Hides
march 2026
A loss curve looks like smooth decay, but the real story of training lives in the plateaus, the sudden jumps, and the circuits quietly assembling out of sight.
Transformer Architecture Internals and Variants
february 2026
The transformer replaced recurrence with direct attention, and every major variant since is just a different masking rule applied to the same skeleton.
The Shape of the Loss Surface
january 2026
Deep learning breaks the rules classical statistics says it should obey, and the explanation hides in the geometry of a loss surface too vast to picture.
The Mathematics of Quantum Mechanics
december 2025
Quantum mechanics is linear algebra dressed up in physics clothing; the physical content lives in a handful of postulates the math itself never delivers.
playlist songs
DAYDREAM
ESPRIT 空想
see all →
Shelf Books
The Remains of the Day
Kazuo Ishiguro
see all →