Skip to main content

t-SNE implementation in scikit-learn

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a powerful tool for visualizing high-dimensional data. It reduces the dimensions of a dataset while preserving the relationships between points, making it easier to detect patterns and clusters. In this article, we will implement t-SNE using scikit-learn, a widely-used Python library for machine learning.

1. Introduction​

t-SNE is commonly used for visualizing datasets with a large number of features or high-dimensional data. It is particularly useful in fields such as bioinformatics, natural language processing, and image recognition, where understanding the structure of the data can lead to valuable insights.

In this example, we apply t-SNE to scikit-learn's load_digits dataset. It contains 1,797 8×8 images; it is not the 28×28 MNIST dataset. We use a deterministic subset to keep the example quick.

2. Importing necessary libraries​

We start by importing the necessary libraries, including scikit-learn for the t-SNE implementation and matplotlib for visualization.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.manifold import TSNE
from sklearn.decomposition import PCA

3. Loading and preprocessing the dataset​

We use the 8×8 digits dataset bundled with scikit-learn.

# Load a deterministic subset
digits = load_digits()
X = digits.data[:300] / 16.0
y = digits.target[:300]
X_pca = PCA(n_components=30, random_state=42).fit_transform(X)

Explanation:​

  • load_digits: Loads scikit-learn's 8×8 handwritten-digits dataset.
  • Division by 16: All pixels share the documented 0–16 scale. Unit-variance scaling could overemphasize nearly constant background pixels.
  • PCA: Reduces computation with a deterministic linear preprocessing step.

4. Applying t-SNE​

Next, we apply t-SNE to reduce the dimensions of the dataset from 64 (8x8 pixels) to 2 dimensions for visualization.

# Apply t-SNE
tsne = TSNE(
    n_components=2,
    perplexity=30,
    init='pca',
    learning_rate='auto',
    max_iter=350,
    random_state=42,
)
X_tsne = tsne.fit_transform(X_pca)

Explanation:​

  • TSNE: Initializes the t-SNE model.
  • n_components=2: Specifies that we want to reduce the data to 2 dimensions.
  • random_state=42: Sets a random seed for reproducibility.
  • fit_transform: Fits the model to the data and transforms it into the lower-dimensional space.

5. Visualizing the results​

We can now visualize the 2D representation of the dataset. Each point in the plot corresponds to an image in the dataset, and we will color the points according to their digit label.

# Plot the t-SNE result
plt.figure(figsize=(10, 7))
scatter = plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=y, cmap='viridis', s=50, alpha=0.7)
plt.colorbar(scatter, label='Digit Label')
plt.title('t-SNE Visualization of scikit-learn Digits')
plt.xlabel('t-SNE Dimension 1')
plt.ylabel('t-SNE Dimension 2')
plt.grid(True)
plt.close()

Explanation:​

  • scatter: Creates a scatter plot of the t-SNE results.
  • c=y: Colors the points based on their digit label.
  • cmap='viridis': Specifies the colormap for coloring the points.
  • alpha=0.7: Sets the transparency of the points to make overlapping points more visible.
  • plt.colorbar: Adds a colorbar to the plot for reference.

6. Interpreting the results​

The result is a 2D view of this digits sample. Nearby points indicate a local probability neighborhood under this configuration. Distances between islands, their sizes and densities, and the number of visible groups are not faithful measurements of the original space. Labels are used only for coloring.

7. Conclusion​

t-SNE is an exploratory visualization technique, not a clustering algorithm. Compare multiple seeds and perplexities, validate any hypothesis in the original feature space, and remember that scikit-learn's estimator has no transform method for unseen observations.