Two models fine-tuned from the same base can be combined by averaging their weights.
No training and no data are needed, which is why merging spread so quickly.
Task arithmetic treats the change made by fine-tuning as a vector that can be added or subtracted.
Merging works because the fine-tuned models stay in the same region of weight space as their shared base.
Merge models from different bases and you get noise, since the weights mean different things.
Model merging combines several fine-tuned models into one without any further training.
The requirement is that all of them descend from the same base checkpoint.
The simplest method averages the corresponding weights, which is often called model soup.
It sounds like it should not work, and for independently trained networks it does not.
It works here because fine-tuning moves weights only a short distance from the shared starting point.
The fine-tuned models therefore sit in the same broad basin, where the path between them stays low in loss.
Task arithmetic makes the idea explicit by subtracting the base weights from a fine-tuned model.
What remains is a task vector, the change that fine-tuning made and nothing else.
Those vectors can be added to give a model both skills, or subtracted to remove a behaviour.
Fine-tuned models often change overlapping parameters, and naive averaging lets them cancel.
Methods such as TIES trim the small changes and resolve sign conflicts before combining.
DARE drops most of the delta at random and rescales the rest, which is surprisingly effective.
Merging is cheap enough to run on a laptop, which is why open model hubs are full of merges.
It also composes with adapters, since separate LoRA modules can be merged the same way.
The limits are real: capabilities can degrade unpredictably, so a merge needs evaluating like any other model.
Averaging the weights of two independently trained networks produces noise, because their hidden units are arbitrarily permuted and nothing lines up.
Averaging two fine-tunes of the same base model does work, because both started from the same point and never travelled far enough to leave its basin.
That is the precondition for everything here: a shared ancestor, and updates small enough that the loss surface between them stays low.
The difference between a fine-tune and its base is a task vector, and it behaves surprisingly like a vector.
Adding two task vectors gives a model with both capabilities. Subtracting one removes a behaviour, which is how unlearning by negation is done.
The scaling coefficients matter. Summing several task vectors at full strength overshoots, because their updates overlap in the same parameters.
Two task vectors often want to move the same parameter in opposite directions, and averaging them lands on zero, which serves neither task.
Most entries of a task vector are near zero and are noise rather than signal, so trimming loses little and removes most of the conflict.
Spherical interpolation follows the arc rather than the chord, which preserves the norm of the weights instead of shrinking it towards the midpoint.
Linear interpolation of two normalised directions produces something shorter than both, and in a network that reads as a uniform loss of signal strength.
The whole cost is one weighted sum, with no gradients.
Merging needs no training data, no GPU time beyond loading the weights, and no labels. It takes minutes.
A model soup averages many fine-tunes with different hyperparameters.
Averaging several runs that differ only in seed and learning rate reliably beats picking the best single run, at no inference cost, unlike an ensemble.
The limits are real.
Models must share an architecture and a tokeniser, and capabilities that genuinely conflict cannot both survive.
The result has to be evaluated, because nothing about the procedure guarantees it works.
One question on this card. Every option explains itself once you pick, right or wrong.
15 concepts in this module, then a graded test with every answer explained.