1 result

for weight-averaging

• Model soups: average the sweep, skip the selection step
  The load-bearing premise, stated in the abstract: fine-tuned models of the same pre-trained initialization **often appear to lie in a single low-error basin** (the paper credits the basin observation to Neyshabur et al. 2020). The authors also analytically relate the performance …
  field/model-soups · fine-tuning, weight-averaging, model-merging, training, image-classification