From "20 Types in Decades" to "Two Types Per Week with AI": Breaking the Curse of Dimensionality in Search Space

The Event

An international team led by Finland's Aalto University recently published research in *Nature*, demonstrating a superconductor material discovery method combining machine learning with quantum physics simulation. Within a short timeframe, it identified two new candidate room-temperature superconductors. By contrast, human chemists over the past half-century, through experimentation, theoretical deduction, and intuition, discovered only about 20 relatively stable superconductor materials.

Why Are Humans So Slow?

How vast is the superconductor material space? Considering: - 118 elements in the periodic table - Each position can be occupied or vacant - Millions of possible crystal structures - Applied pressure and temperature conditions

The combinatorial count exceeds $10^{50}$—far beyond what humans could feasibly screen in a lifetime. This is the "curse of dimensionality" that Richard Bellman identified in 1957: the more variables, the sparser the space, and exhaustive search becomes exponentially harder.

Human chemists' strategy relies on "intuition + theory" to narrow the search range: - Based on quantum physics principles, hypothesize which electron structures might be superconducting - Based on similarity to known materials, guess at adjacent spaces for new candidates - Verify one by one through experiments

But this strategy's flaw: intuition typically derives from "the local neighborhood of known materials," overlooking distant regularities that may exist elsewhere in the space.

AI's "Dimensionality Reduction" Technique

AI doesn't search high-dimensional space faster—it *projects* high-dimensional space onto a low-dimensional manifold. The concrete process:

1. Train an autoencoder using the 20 known superconductors - Learn: "Which chemical features are necessary for superconductivity" - Compress hundreds of material dimensions into 5-10 dimensions

2. Conduct efficient search in low-dimensional space - Instead of checking $10^{50}$ candidates individually, find "high-probability superconducting regions" in 10-dimensional space - Like extracting "subway lines" from a three-dimensional city map, then only searching subway stations rather than the entire city

3. Verify candidate materials using quantum simulation (DFT—density functional theory) - Use computer simulation before experiments to filter out obvious failures

4. Experimental validation of the most promising candidates - Success rate far exceeds random or traditional screening

Deeper Pattern: A Paradigm Shift

This case embodies a fundamental transition from "brute-force search" to "structural learning":

| Traditional Method | AI Method | | --- | --- | | Assumption: space is uniformly random | Assumption: space has hidden structure (manifold) | | Strategy: more attempts | Strategy: learn structure, narrow search | | Time complexity: $O(N)$ or $O(N^2)$ | Time complexity: $O(d \times \log N)$ ($d$ = manifold dimension) | | Success rate: depends on intuition | Success rate: depends on training data quality |

Insight: Beyond Superconductors

This pattern transfers to any "high-dimensional combinatorial search" problem: - Drug discovery: molecular space $10^{60}$, AI pre-screens potentially active compounds - Protein folding: vast structural space, AlphaFold learned "folding rules" - Crystal materials: AlphaFlow discovered new battery materials using the same logic - Engineering optimization: aircraft design, semiconductor process parameter tuning

Common thread: not "AI is smarter than humans," but rather "AI can identify structure in high-dimensional space and perform dimensionality reduction search."

The Human Role Hasn't Disappeared

Notably, this success depends on three critical layers: 1. Domain knowledge: quantum physicists designed the AI's scoring function (what makes a "good superconductor") 2. Training data: the 20 known superconductors form the foundation for AI learning 3. Verification layer: discovered candidates still require experimental validation; AI predictions aren't 100% trustworthy

AI accelerates the "middle search layer," not every step in the entire scientific process.