notes from the pacific coast highwaynotes from the pacific coast highwaynotes from the pacific coast highway

Jay Singh

A blog for ongoing research


latest dispatches

  • Splitting a Model Across Machines

    april 2026

    A model too large for one chip gets split along three axes, each technique answering a different bottleneck, until the binding constraint stops being memory and becomes failure itself.

  • What the Loss Curve Hides

    march 2026

    A loss curve looks like smooth decay, but the real story of training lives in the plateaus, the sudden jumps, and the circuits quietly assembling out of sight.

  • Transformer Architecture Internals and Variants

    february 2026

    The transformer replaced recurrence with direct attention, and every major variant since is just a different masking rule applied to the same skeleton.

  • The Shape of the Loss Surface

    january 2026

    Deep learning breaks the rules classical statistics says it should obey, and the explanation hides in the geometry of a loss surface too vast to picture.

  • The Mathematics of Quantum Mechanics

    december 2025

    Quantum mechanics is linear algebra dressed up in physics clothing; the physical content lives in a handful of postulates the math itself never delivers.

playlist songs

  • DAYDREAM

    ESPRIT 空想

see all →

Shelf Books

  • The Remains of the Day

    Kazuo Ishiguro

see all →