VRAM is Too Expensive So We Use Graduate Students to Train LLMs: The GUON Optimizer

Scaling large language models (LLMs) confronts a severe memory wall crisis, with modern optimizers easily hoarding terabytes of historical states. Facing a dried-up funding account, we propose \textbf{Guon(Graduate Unpaid Optimization Nodes)}—a stateless second-order optimization paradigm driven entirely by carbon-based compute power. On the theory side, we biologically reconstruct the concept of space-time duality : since we cannot afford the VRAM to accumulate momentum across time , we densely pack cheap graduate students to fill the physical space, forcing them to manually perform FP32 gradient accumulation on scratch paper. On the systems side, we design a streaming offload and two-stream ping-pong schedule : the GPU is only responsible for spitting out single-layer gradients, which are immediately output as paper scrolls by an old dot-matrix printer for asynchronous offload. Group A students frantically calculate the Newton-Schulz iteration formula on blackboards to complete matrix orthogonalization , while Group B students sprint down the hallway to prefetch the printed sheets of the next layer. Miraculously, manual calculation errors caused by extreme sleep deprivation and sweat dripping on the paper perfectly serve as controlled perturbations to stabilize the spectral operator. Evaluations show that GUON successfully slashes the static memory requirement of a 70B model by roughly 8x (down to ~140 GB). This study proves that as long as the supervisor's whip cracks fast enough, the computing potential of graduate students can completely push the training-time memory closer to the inference lower bound, with the only system bottlenecks being the reimbursement limit for A4 paper and the medical treatment costs for carpal tunnel syndrome.