Explore semi-supervised clustering with LVQ. Learn how prototype vectors are adjusted to form subclasses within categories using labeled training data.
Learning Vector Quantization (LVQ) is a semi-supervised clustering algorithm that uses labeled prototypes to form subclasses within categories. Unlike k-means, LVQ prototypes have predefined class labels and are adjusted during training to better separate classes.
LVQ learns to separate classes by moving prototypes toward samples of the same class and away from samples of different classes.
LVQ iteratively adjusts prototype vectors based on labeled training samples:
Initialize q prototype vectors {p₁, p₂, ..., p_q} with predefined class labels {t₁, t₂, ..., t_q}. Each prototype belongs to a specific class.
Randomly select a labeled sample (xⱼ, yⱼ) from the training set, where yⱼ is the class label.
Calculate distance from xⱼ to all prototypes and find the nearest: p_t = argmin_i ||xⱼ - pᵢ||
If yⱼ = t_t (same class): p_t ← p_t + η(xⱼ - p_t)
If yⱼ != t_t (different class): p_t ← p_t - η(xⱼ - p_t)
where η is the learning rate (0 < η < 1)
Repeat steps 2-4 until convergence (prototypes stabilize) or maximum iterations reached.
The learning rate η controls how much prototypes move toward or away from samples. Common strategies:
Use constant η (e.g., 0.1) throughout training. Simple but may overshoot or converge slowly.
Start with larger η and gradually decrease: η(t) = η₀ × (1 - t/T). Allows fine-tuning near convergence.