Machine Learning/Learning Center/Dimensionality Reduction/Manifold Learning Overview

Manifold Learning Overview

Introduction to nonlinear dimensionality reduction through manifold learning. Understand when data lies on low-dimensional manifolds and how to preserve structure.

Module 6 of 9

Intermediate to Advanced

60-80 min

Core Assumption: Manifold Hypothesis

Manifold learning assumes that high-dimensional data lies on or near a low-dimensional manifold embedded in the high-dimensional space. A manifold is a space that is locally Euclidean (locally flat) but globally curved.

Key Insight

While data exists in d-dimensional space, it may actually lie on a d'-dimensional manifold where $d' << d$ . Manifold learning finds this intrinsic structure.

Example: A 2D Swiss roll embedded in 3D space. The data is 3D but intrinsically 2D (can be "unrolled" to a flat sheet).

Why Manifold Learning?

Linear methods (PCA, MDS) assume data lies on a linear subspace. When data lies on a curved manifold, linear methods fail to capture the structure.

• Linear methods: Project onto flat hyperplane
• Manifold learning: Unfold the curved surface

Local vs Global Structure Preservation

Different manifold learning methods preserve different aspects of structure:

Global Structure

Preserves distances and relationships across the entire dataset.

• Isomap: Preserves geodesic distances
• MDS: Preserves pairwise distances

Good for understanding overall data topology.

Local Structure

Preserves relationships within local neighborhoods.

• LLE: Preserves local linear relationships
• t-SNE: Preserves local neighborhoods

Good for preserving fine-grained local geometry.

Isomap vs LLE Comparison

Two fundamental approaches to manifold learning:

Isomap (Isometric Mapping)

Goal: Preserve global geodesic distances (shortest paths on the manifold).

• Constructs neighbor graph
• Computes shortest paths (geodesic distances)
• Applies MDS to preserve geodesic distances
• Strengths: Preserves global topology
• Weaknesses: Sensitive to noise, requires good neighbor selection

LLE (Locally Linear Embedding)

Goal: Preserve local linear reconstruction relationships.

• Finds local neighborhoods
• Computes reconstruction weights
• Preserves weights in low dimensions
• Strengths: Robust, preserves local geometry
• Weaknesses: May not preserve global structure

When to Use Manifold Learning

Use Manifold Learning When:

• Data lies on curved/nonlinear structure (e.g., Swiss roll)
• Linear methods (PCA) fail to capture structure
• Need to preserve local or global topology
• Visualization of complex high-dimensional data

Use Linear Methods When:

• Data lies on linear subspace
• Need fast computation
• Large datasets (manifold methods are slower)
• Need interpretable features

Next Module