Before a model learns anything, its parameters need some value to start from. Weight initialization is the choice of those starting values. It happens once, at the very beginning of pretraining, and everything the model later learns is a series of adjustments to that starting point.
What it is
A neural network is made of parameters, mostly numbers called weights. A freshly built network has no knowledge, but it cannot start with every weight set to the same number. If it did, every part of the network would behave identically and receive identical updates, so the parts would never specialize. Starting from random values breaks that symmetry.
Think of a classroom where every student is given the same answer to memorize. Nobody can learn to differ. Random starts give each student a slightly different place to begin.
How it is done
Random does not mean careless. Common methods draw numbers from a distribution whose spread is scaled to the size of each layer. The aim is to keep the signals flowing forward, and the error signals flowing backward during gradient descent, from shrinking toward zero or growing out of control as they pass through many layers. Well-known recipes exist for this, and deep-learning libraries apply sensible defaults for you.
The random numbers come from a pseudo-random generator controlled by a seed. Using the same seed and the same software setup lets researchers reproduce a starting point exactly. In practice, how the numbers are generated can depend on details of the software and hardware, which is one reason exact reproduction can be harder than it sounds.
Why it matters
A poor initialization can make training slow, unstable or stuck. A good one gives optimization a fair start. For very large models, a failed run is expensive, so teams treat initialization as part of the recipe they test at small scale first.
It is also different from starting with knowledge. Fine-tuning begins from weights that were already trained, so it skips this step. Training from scratch is the case where initialization applies.
Why researchers care about it
Some of a model's weights change very little during training. Those weights can keep traces of how they began, which is why initialization can matter long after the run starts and why it comes up when people study where a published model came from.
How it relates to other terms
Once training finishes, the result is the set of model weights that gets released or kept private. Initialization is where those numbers began. Everything between is gradient descent at work.