Weighted Nearest Neighbour

Weighted nearest neighbour (WNN) is an extension of the basic K-Nearest Neighbour (K-NN) algorithm. The idea of WNN is that not all neighbours are equally important, closer neighbours should have more influence on the prediction than farther ones. It is often used for regression tasks. We assign weights to each neighbour based on its distance to the new example.

How it works

1. Choose a distance metric (e.g. Euclidean, Manhattan)
2. Find the $k$ nearest neighbours to the new data point $X_{new}$
3. Assign a weight to each neighbour, often using: $w_i=\frac{1}{D(X_{new},X_i)}$ Where:

• $D(X_{new},X_i)$ is the distance between the example and neighbour $i$
• $w_i$ is the weight to the neighbour $i$

Closer points → smaller distance → larger weight

4. Prediction for $X_{new}$ using formula $\text{Prediction for }{X}_{new}={\sum }_{i=1}^k{w}_i\cdot y_i$ Equivalently: $\text{Prediction for }{X}_{new}=w_1\cdot y_1+w_2\cdot y_2+\cdots +w_k\cdot y_k$ Where:

• $k$: the number of nearest neighbours
• $y_i$: the target value (e.g. salary, house price) of the $i^{th}$ nearest neighbour
• $w_i$: the weight of the $i^{th}$ neighbour calculated in step 3

Alternative using all training examples

In some cases, you don’t just use the $k$ nearest — you use every point in the dataset, but weight all by: $w_i=\frac{1}{D(X_{new},X_i)}$ Distant points contribute very little (almost zero), because most of them can be far away from $X_{new}$. However, the algorithm will be slower.

---

Back to parent page: Supervised Machine Learning

AI Machine_Learning COMP3308 Unsupervised_Learning Lazy_Learning Classification KNN K-Nearest_Neighbour WNN Weighted_Nearest_Neighbour