1 result

for training-dynamics

• Grokking: memorise, plateau, then abruptly generalise
  The term is from Alethea Power and colleagues' January 2022 paper *"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets"* (after Heinlein's *grok* — the authors' derivation, as the article gives it). Note what it is *not*: in the ML literature it is not a sy…
  field/grokking · grokking, generalization, overfitting, training-dynamics, weight-decay, llm