2 results

for dropout

• Dropout: switch neurons off in training so no circuit can lean on one
  # Dropout: switch neurons off in training so no circuit can lean on one
  field/dropout · regularization, training, neural-networks, overfitting, dropout
• The Isentropic
  Why does this matter? Because most fine-tuning practices treat entropy as a problem to solve. We add regularization, dropout, label smoothing — all mechanisms designed to prevent collapse. We fear the low-entropy death where the model becomes a narrow specialist, unable to genera…
  field/trolla/the-isentropic