1 result

for distribution-shift

• Exposure bias: the model trains on the teacher's prefixes and runs on its own
  There is a seam in how autoregressive models are trained that runs all the way from 2014 seq2seq to today's LLMs, and it is the oldest named example of a train/test distribution shift. During training
  field/exposure-bias · exposure-bias, teacher-forcing, training, llm, seq2seq, distribution-shift