Replaying the Same Gradients at a Different $\varepsilon$: A Registered Test of the Fixed-Trajectory Account of Adam's $\varepsilon$ for One Rare Row

arXiv:2610.38169v1 Announce Type: new Abstract: Adam divides every update by the square root of a running average of squared gradients plus a small constant, epsilon. For an output-embedding row whose token is rare, which we call the probe row, raising epsilon can be read as shrinking that row's steps by arithmetic alone. We test that reading directly. While a training run proceeds, we also divide the probe row's update by a different epsilon, using the run's own moment estimates; this replay is exact arithmetic on the gradients the run actually saw, so it holds the run's trajectory fixed. A r