Skip to main content

Common mistakes and best practices in K-Means

K-Means is simple to call but easy to misinterpret. Its output is the solution found for a specific squared-Euclidean objective on a specific numeric representation. It is not evidence that the world contains those groups.

1. Assuming every cluster shape is suitable​

K-Means partitions space into convex Voronoi cells around means. It often works well for compact groups with comparable dispersion, but elongated, nested, strongly unequal-density, or non-convex groups can be split incorrectly.

import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_moons

X_moons, _ = make_moons(n_samples=300, noise=0.06, random_state=42)
moon_model = KMeans(n_clusters=2, n_init=20, random_state=42)
moon_labels = moon_model.fit_predict(X_moons)

fig, ax = plt.subplots()
ax.scatter(X_moons[:, 0], X_moons[:, 1], c=moon_labels, cmap="viridis", s=18)
ax.set(title="K-Means imposes a centroid partition on two moons")
fig.tight_layout()
plt.close(fig)

Compare DBSCAN, spectral clustering, or a justified hierarchical linkage when their assumptions better match the data. None is automatically superior on every non-convex dataset.

2. Treating the number of clusters as known truth​

Inertia always decreases as k increases, so its minimum cannot select k. An elbow or silhouette maximum is a diagnostic, not an oracle. Compare plausible values using domain usefulness, stability under resampling and seeds, and more than one internal measure.

from sklearn.datasets import make_blobs
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

X, reference_groups = make_blobs(n_samples=360, centers=4, cluster_std=0.60, random_state=42)
X_scaled = StandardScaler().fit_transform(X)
diagnostics = []
for k in range(2, 7):
    model = KMeans(n_clusters=k, n_init=20, random_state=42)
    labels = model.fit_predict(X_scaled)
    diagnostics.append((k, model.inertia_, silhouette_score(X_scaled, labels)))

assert max(diagnostics, key=lambda row: row[2])[0] == 4

The assertion is a known result for this deliberately simple fixture, not a promise that silhouette recovers a true cluster count generally.

3. Relying on one initialization​

Lloyd's algorithm reaches a local optimum. Use k-means++ and explicit repeated starts, and retain random_state for reproducibility. More starts reduce initialization risk but cannot repair unsuitable geometry.

single = KMeans(n_clusters=4, init="random", n_init=1, random_state=7).fit(X_scaled)
robust = KMeans(n_clusters=4, init="k-means++", n_init=20, random_state=7).fit(X_scaled)
assert robust.inertia_ <= single.inertia_

4. Ignoring scale and feature meaning​

Squared Euclidean distance is sensitive to units. Standardize continuous features when unit variance is a defensible weighting choice; use domain-based weights or another representation when it is not. Scaling is not a ritual, and arbitrary integer codes for categories are not meaningful Euclidean coordinates.

import numpy as np

X_mixed_units = np.array([
    [1.0, 100.0], [1.2, 120.0], [4.0, 105.0], [4.2, 125.0]
])
raw_labels = KMeans(n_clusters=2, n_init=20, random_state=42).fit_predict(X_mixed_units)
scaled_mixed = StandardScaler().fit_transform(X_mixed_units)
scaled_labels = KMeans(n_clusters=2, n_init=20, random_state=42).fit_predict(scaled_mixed)
assert not np.array_equal(raw_labels, scaled_labels)

For deployment, fit the scaler and estimator on a reference/training population, then apply both unchanged to later observations. Fitting preprocessing separately on evaluation data changes the geometry being evaluated.

5. Ignoring outliers and missing values​

Means can be pulled strongly by extreme observations. Investigate data quality and use robust domain rules, robust scaling, trimmed approaches, K-Medoids, or density methods when justified. Do not delete unusual cases merely because K-Means dislikes them.

from sklearn.impute import SimpleImputer

X_with_missing = X_scaled.copy()
X_with_missing[0, 0] = np.nan
X_imputed = SimpleImputer(strategy="median").fit_transform(X_with_missing)
prepared_model = KMeans(n_clusters=4, n_init=20, random_state=42).fit(X_imputed)
assert np.isfinite(prepared_model.cluster_centers_).all()

6. Claiming K-Means accepts arbitrary metrics​

Scikit-learn's KMeans has no metric parameter: its mean update is paired with squared Euclidean distance. Manhattan distance leads to median-based methods; cosine-oriented problems often use normalized vectors and spherical K-Means or another estimator. Changing only a distance call no longer implements the K-Means objective described here.

7. Overreading validation metrics​

Silhouette favors separation under the selected metric and may prefer compact convex groups. Davies–Bouldin and Calinski–Harabasz encode other geometric preferences. If trusted reference labels exist, adjusted Rand index can compare partitions without depending on label numbers, but those labels turn the exercise into external validation.

from sklearn.metrics import adjusted_rand_score, calinski_harabasz_score, davies_bouldin_score

labels = robust.labels_
scores = {
    "silhouette": silhouette_score(X_scaled, labels),
    "davies_bouldin": davies_bouldin_score(X_scaled, labels),
    "calinski_harabasz": calinski_harabasz_score(X_scaled, labels),
    "adjusted_rand": adjusted_rand_score(reference_groups, labels),
}
assert scores["silhouette"] > 0.7
assert scores["davies_bouldin"] >= 0
assert scores["adjusted_rand"] > 0.95

Also inspect membership sizes, centroids in original units, sensitivity to preprocessing, and stability across seeds or bootstrap samples. Use a held-out or later sample when the clusters will drive decisions beyond the exploratory dataset.

Conclusion​

Reliable K-Means work states the representation, squared-Euclidean objective, initialization strategy, number-of-clusters rationale, and validation limitations. The algorithm is strongest when those choices have substantive meaning, not merely when a plot looks separated.