MLFF Geometry Optimization
This task measures how closely machine-learning force-field relaxations reproduce DFT-relaxed crystal structures and retain their symmetry.
Reference data: the WBM test set. The headline metric is normalized structure-matching RMSD (unitless, lower is better).
Leaderboard
Not all models relaxed every structure, but each reported model was evaluated on at least 249k relaxations.
Symmetry-tolerance caveat: RMSD is
symprec-invariant. The leaderboard shows symmetry metrics at bothsymprec=1e-2andsymprec=1e-5, while Aggregate Diagnostics usesymprec=1e-5. Compare symmetry values within a single view.
42 of 43 eligible models · Sorted by RMSD (ascending)
| Symmetry | Hyperparams | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| ↓ | |||||||||||
| TECE-OAM-RRA-1.0 | 0.058 | 81.83% | 4.98% | 12.45% | 59.07% | 18.46% | 21.02% | FIRE | 500 | 0.02 | Frechet |
| GRACE-3L-OAM-L | 0.058 | 82.4% | 3.84% | 13% | 69.29% | 4.27% | 25.94% | FIRE | 500 | 0.01 | Frechet |
| EquFlashV2* | 0.058 | 82.02% | 4.74% | 12.51% | 70.05% | 5.07% | 24.34% | FIRE | 500 | 0.02 | Frechet |
| Prophet-OAME-MBD | 0.059 | 81.48% | 5.59% | 12.22% | 60.71% | 17.84% | 20.5% | FIRE | 500 | 0.02 | Frechet |
| EquiformerV3+DeNS-OAM | 0.059 | 74.24% | 12.9% | 11.37% | 37.11% | 45.54% | 12.99% | FIRE | 500 | 0.02 | Frechet |
| PET-OAM-XL | 0.060 | 80.7% | 6.67% | 11.85% | 13.08% | 86.68% | 0.09% | FIRE | 500 | 0.02 | Frechet |
| MatRIS-10M-OAM | 0.060 | 81.53% | 5.47% | 12.27% | 59.55% | 19.5% | 19.96% | FIRE | 500 | 0.02 | Frechet |
| EquFlash* | 0.060 | 81.98% | 4.68% | 12.64% | 70.19% | 4.89% | 24.4% | FIRE | 500 | 0.02 | Frechet |
| TACE-OAM-L | 0.061 | 81.88% | 4.79% | 12.6% | 69.22% | 6.03% | 24.15% | FIRE | 500 | 0.02 | Frechet |
| eSEN-30M-OAM | 0.061 | 71.03% | 16.16% | 10.85% | 36.36% | 47.39% | 11.88% | FIRE | 500 | 0.02 | Frechet |
| SevenNet-Omni-i12* | 0.062 | 81.87% | 4.69% | 12.72% | 70.56% | 4.39% | 24.54% | FIRE | 800 | 0.02 | Frechet |
| Nequip-OAM-XL | 0.063 | 82.23% | 2.53% | 14.37% | 68.34% | 9.75% | 21.17% | GOQN | 500 | 0.005 | Frechet |
| GRACE-2L-OAM-L | 0.064 | 81.66% | 5.37% | 12.26% | 73.43% | 5.39% | 20.67% | FIRE | 500 | 0.03 | Frechet |
| Nequip-OAM-L | 0.065 | 81.6% | 5.43% | 12.26% | 71.25% | 7.92% | 20.18% | GOQN | 500 | 0.05 | Frechet |
| Allegro-OAM-L | 0.065 | 81.6% | 5.42% | 12.26% | 71.63% | 7.53% | 20.23% | GOQN | 500 | 0.05 | Frechet |
| GRACE-2L-OAM | 0.067 | 81.59% | 5.63% | 12.09% | 73.3% | 3.25% | 23.01% | FIRE | 500 | 0.03 | Frechet |
| DPA-4.0.1-Pro-MPtrj | 0.069 | 66.31% | 20.8% | 10.37% | 38.99% | 41% | 14.22% | FIRE | 500 | 0.02 | Frechet |
| eqV2 M | 0.069 | 77.25% | 11.18% | 10.52% | 14.22% | 85.2% | 0.45% | FIRE | 500 | 0.02 | Frechet |
| DPA-3.1-3M-FT | 0.069 | 81.09% | 5.02% | 13.17% | 72.74% | 5.39% | 21.38% | FIRE | 500 | 0.05 | Exp |
| EquiformerV3+DeNS-MP | 0.070 | 64.13% | 23.01% | 10.22% | 39.19% | 42.23% | 13.08% | FIRE | 500 | 0.02 | Frechet |
| MatRIS-10M-MP | 0.072 | 79.92% | 7.89% | 11.45% | 57.66% | 23.95% | 17.2% | FIRE | 500 | 0.05 | Frechet |
| GRACE-1L-OAM | 0.072 | 81.4% | 5.76% | 12.16% | 73.36% | 3.29% | 22.92% | FIRE | 500 | 0.03 | Frechet |
| MACE-MPA-0 | 0.073 | 81.44% | 5.56% | 12.31% | 73.24% | 3.28% | 23.05% | FIRE | 500 | 0.05 | Frechet |
| MatterSim v1 5M | 0.073 | 81.48% | 5.24% | 12.54% | 68.74% | 7.55% | 23.05% | FIRE | 500 | 0.02 | Exp |
| ORB v3 | 0.075 | 72.8% | 15.9% | 10.1% | 13% | 86.9% | 0.01% | FIRE | 500 | 0.02 | Frechet |
| eSEN-30M-MP | 0.075 | 61.06% | 26.04% | 9.8% | 38.59% | 42.81% | 12.82% | FIRE | 500 | 0.02 | Frechet |
| eqV2 S DeNS | 0.076 | 53.96% | 37.24% | 6.53% | 13.22% | 86.71% | 0.06% | FIRE | 500 | 0.02 | Frechet |
| AlphaNet-v1-OAM* | 0.079 | 74.31% | 14.46% | 10.37% | 14.16% | 85.05% | 0.67% | FIRE | 500 | 0.03 | Frechet |
| HIENet | 0.080 | 80.02% | 7.55% | 11.63% | 45.21% | 34.93% | 15.32% | FIRE | 500 | 0.05 | Frechet |
| Allegro-MP-L | 0.082 | 81.25% | 5.72% | 12.3% | 72.11% | 7.12% | 20.15% | GOQN | 500 | 0.05 | Frechet |
| Eqnorm MPtrj | 0.084 | 81.13% | 5.89% | 12.24% | 69.83% | 6.39% | 23.16% | FIRE | 500 | 0.02 | Frechet |
| SevenNet-l3i5* | 0.085 | 78.8% | 8.95% | 11.34% | 44.12% | 36.67% | 14.51% | FIRE | 500 | 0.05 | Frechet |
| Nequix MP | 0.085 | 81.11% | 5.54% | 12.61% | 69.99% | 6.07% | 23.35% | FIRE | 500 | 0.02 | Frechet |
| Nequip-MP-L | 0.086 | 81.09% | 5.92% | 12.27% | 71.63% | 7.88% | 19.82% | GOQN | 500 | 0.05 | Frechet |
| BAM-MP-core | 0.086 | 81.21% | 6.1% | 12.02% | 72.56% | 4.69% | 22.23% | FIRE | 500 | 0.05 | Frechet |
| Nequix MP PFT | 0.087 | 81.09% | 5.4% | 12.76% | 69.76% | 6.02% | 23.62% | FIRE | 500 | 0.02 | Frechet |
| GRACE-2L-MPtrj | 0.090 | 81.07% | 5.96% | 12.28% | 73.15% | 3.49% | 22.92% | FIRE | 300 | 0.03 | Frechet |
| MACE-MP-0 | 0.091 | 81.12% | 5.99% | 12.2% | 73.85% | 3.35% | 22.39% | FIRE | 500 | 0.05 | Frechet |
| CHGNet | 0.095 | 79.21% | 9.22% | 10.76% | 60.62% | 22.41% | 16.01% | FIRE | 500 | 0.05 | Frechet |
| ORB v2 MPA | 0.097 | 49.26% | 45.47% | 4.53% | 14.33% | 85.31% | 0.34% | FIRE | 500 | 0.02 | Frechet |
| ORB v2 MPtrj | 0.101 | 43.32% | 52.13% | 3.87% | 13.35% | 86.55% | 0.09% | FIRE | 500 | 0.02 | Frechet |
| M3GNet | 0.112 | 80.31% | 6.81% | 12.17% | 74.08% | 6.71% | 18.69% | FIRE | 500 | 0.05 | Exp |
Model Comparison
Use the axis, color, and size controls to compare the models with geometry-optimization metrics directly from the current model YAML data.
Σ= 10-2 vs RMSD
The default view compares structure-matching RMSD (lower is better) with the
fraction of matching spacegroups at symprec=1e-2 (higher is better). Marker
size defaults to model parameters and color to training-set size.
Aggregate Diagnostics
These views use the per-structure symprec=1e-5 analyses. The model picker applies to every view below.
- Prophet-OAME-MBD
- BAM-MP-core
- TECE-OAM-RRA-1.0
- GRACE-3L-OAM-L
- EquFlashV2
Cumulative Distribution of RMSD
Cumulative distribution of RMSD between ML and DFT-relaxed structures.
Difference in Number of Symmetry Operations vs DFT
Difference in number of symmetry operations of ML vs DFT-relaxed structures. Models are sorted by the standard deviation σ of ΔNsym ops = Nsym ops,ML - Nsym ops,DFT.
Sankey Diagrams for ML vs DFT Spacegroups
The Sankey diagrams show corresponding spacegroups of DFT-relaxed and MLFF-relaxed structures at symprec=1e-5. For visual clarity, only the 10 most common pairs of (DFT, MLFF) spacegroups are shown.
EquFlashV2
TECE-OAM-RRA-1.0
Prophet-OAME-MBD
GRACE-3L-OAM-L
BAM-MP-core
Test set: WBM
256,963 crystal structures · 85 elements
Data by Wang, Botti, and Marques
Chemically substituted crystals with DFT relaxations test stability predictions and how closely relaxed geometries reproduce DFT. Discovery uses a unique-prototype subset; geometry optimization evaluates the relaxed structures.
Public initial and relaxed structures, energies, and symmetry references.
Shared by discovery and geometry optimization. These summaries describe the full WBM set.
Methodology
Structure matching, symmetry, and relaxation protocol
RMSD measures the normalized, unitless structure-matching RMSD between ML- and DFT-relaxed ground state structures, as returned by pymatgen‘s StructureMatcher after matching. Optimizer, Steps, fmax, and Filter show the ASE optimizer, maximum relaxation steps, force convergence criterion (eV/Å), and cell filter used during structure relaxation.
Symmetry detection uses moyopy, a Rust successor to spglib. Σ= denotes the fraction of structures with identical ML- and DFT-relaxed spacegroups. Σ↓ / Σ↑ denote the fractions whose numbers of symmetry operations decreased / increased during MLFF relaxation. Symmetry metrics depend on the symprec tolerance, so results are shown at multiple values. See the spglib docs and paper for details.
The WBM DFT references were generated with MPRelaxSet: ISYM=2, ISIF=3, and IBRION=2 (conjugate gradient). Most MLFF relaxations instead use FIRE. Different symmetry constraints and optimizers can reach different minima, so a lower symmetry-match rate can occasionally indicate a valid symmetry-broken structure rather than a model error. See MPRelaxSet.yaml. Thanks to Alex Ganose for highlighting this distinction.