MLFF Phonon Modeling Metrics

This task evaluates how well ML force fields reproduce phonons and lattice thermal conductivity at 300 K.

Reference data: PhononDB PBE. The headline κSRME metric compares mode contributions before summation, from 0 (perfect) to 2 (maximum error); lower is better.

Leaderboard

κfailed is the fraction of predictions whose κSRME was censored to 2 because of imaginary modes, broken symmetry, or invalid conductivity data. A valid κSRME of 2 is not a failure; Im(ω) separately reports the imaginary-mode rate.

42 of 42 eligible models · Sorted by κSRME (ascending) · Row numbers follow this filtered order.

Filters:
Submit a model RSS
# ↓
1 Prophet-OAME-MBD 0.065 0.028 −0.001 0% 0% 0.019 62.3M EFSG 2026-09-13 7 Å 16.4M (324M) MPtrj+OMat24+sAlex+ELEMENTA
2 TECE-OAM-RRA-1.0 0.093 0.045 −0.005 0% 0% 0.049 222M EFSG 2026-07-05 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
3 EquFlashV2* 0.094 0.038 0.001 0% 0% 0.055 44.9M EFSG 2026-06-11 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
4 EquiformerV3+DeNS-OAM 0.118 0.05 −0.002 0% 0% 0.065 30.3M EFSG 2026-04-07 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
5 PET-OAM-XL 0.119 0.06 −0.031 0% 0% 0.091 730M EFSG 2026-01-10 n/a 6.6M (113M) MPtrj+OMat24+sAlex
6 GRACE-3L-OAM-L 0.121 0.054 −0.014 0% 0% 0.072 42.1M EFSG 2026-07-02 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
7 Nequip-OAM-XL 0.125 0.054 −0.021 0% 0% 0.089 32.1M EFSG 2025-11-30 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
8 TACE-OAM-L 0.126 0.05 −0.005 0% 0% 0.063 82.9M EFSG 2026-04-09 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
9 EquFlash* 0.158 0.077 −0.024 0% 0% 0.095 28.7M EFSG 2025-06-23 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
10 Nequip-OAM-L 0.166 0.081 −0.059 0% 0% 0.126 9.6M EFSG 2025-09-08 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
11 GRACE-2L-OAM-L 0.169 0.064 −0.015 0% 0% 0.095 26.4M EFSG 2025-09-09 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
12 eSEN-30M-OAM 0.17 0.086 −0.051 1% 0% 0.139 30.2M EFSG 2025-03-17 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
13 SevenNet-Omni-i12* 0.192 0.096 −0.045 0% 0% 0.11 54.9M EFSG 2026-01-12 6 Å n/a COSMOSDataset
14 ORB v3 0.21 0.126 −0.099 0% 0% 0.224 25.5M EFSG 2025-04-05 6 Å 6.47M (133M) MPtrj+Alex+OMat24
15 DPA-4.0.1-Pro-MPtrj 0.211 0.106 −0.034 0% 0% 0.173 22.8M EFSG 2026-06-11 6 Å 146k (1.58M) MPtrj
16 MatRIS-10M-OAM 0.218 0.118 −0.048 0% 0% 0.084 10.4M EFSGM 2025-10-29 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
17 EquiformerV3+DeNS-MP 0.275 0.114 0.028 0% 0% 0.193 30.3M EFSG 2026-04-07 6 Å 146k (1.58M) MPtrj
18 GRACE-2L-OAM 0.294 0.144 −0.093 0% 0% 0.222 12.6M EFSG 2025-02-06 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
19 Nequix MP PFT 0.307 0.169 −0.096 1% 1% 0.198 708k EFSHG 2026-01-08 6 Å 154k (1.59M) MPtrj+MDR-MP PBE ωq
20 Allegro-OAM-L 0.319 0.143 −0.061 1% 1% 0.186 9.7M EFSG 2025-09-08 7 Å 6.6M (113M) MPtrj+OMat24+sAlex
21 eSEN-30M-MP 0.34 0.185 −0.113 1.9% 1% 0.147 30.1M EFSG 2025-03-17 6 Å 146k (1.58M) MPtrj
22 Eqnorm MPtrj 0.408 0.272 −0.216 1% 1% 0.417 1.31M EFSG 2025-05-26 6 Å 146k (1.58M) MPtrj
23 MACE-MPA-0 0.412 0.205 −0.11 0% 0% 0.395 9.06M EFSG 2024-12-09 6 Å 3.37M (12M) MPtrj+sAlex
24 Nequix MP 0.446 0.29 −0.248 0% 0% 0.516 708k EFSG 2025-08-17 6 Å 146k (1.58M) MPtrj
25 Nequip-MP-L 0.466 0.306 −0.188 7.8% 1.9% 0.22 9.6M EFSG 2025-09-08 6 Å 146k (1.58M) MPtrj
26 DPA-3.1-3M-FT 0.469 0.309 −0.249 1% 1% 0.316 3.27M EFSG 2025-06-05 6 Å n/a OpenLAM
27 MatRIS-10M-MP 0.489 0.388 −0.346 2.9% 2.9% 0.159 10.4M EFSGM 2025-10-29 6 Å 146k (1.58M) MPtrj
28 Allegro-MP-L 0.504 0.231 −0.022 1.9% 1.9% 0.376 18.7M EFSG 2025-09-08 6 Å 146k (1.58M) MPtrj
29 GRACE-1L-OAM 0.517 0.282 −0.197 0% 0% 0.553 3.45M EFSG 2025-02-06 6 Å 6.6M (113M) MPtrj+OMat24+sAlex
30 GRACE-2L-MPtrj 0.525 0.319 −0.12 2.9% 2.9% 0.422 15.3M EFSG 2024-11-21 6 Å 146k (1.58M) MPtrj
31 SevenNet-l3i5* 0.55 0.399 −0.372 2.9% 2.9% 0.519 1.17M EFSG 2024-12-10 5 Å 146k (1.58M) MPtrj
32 MatterSim v1 5M 0.575 0.413 −0.38 2.9% 2.9% 0.25 4.55M EFSG 2024-12-16 5 Å n/a MatterSim
33 HIENet 0.642 0.49 −0.44 2.9% 2.9% 0.371 7.51M EFSG 2025-07-01 5 Å 146k (1.58M) MPtrj
34 AlphaNet-v1-OAM* 0.643 0.491 −0.384 3.9% 3.9% 0.239 4.65M EFSG 2025-05-12 5 Å 6.6M (113M) MPtrj+OMat24+sAlex
35 MACE-MP-0 0.682 0.471 −0.284 2.9% 0% 0.871 4.69M EFSG 2023-07-14 6 Å 146k (1.58M) MPtrj
36 BAM-MP-core 0.848 0.719 −0.663 12.6% 6.8% 0.8 17M EFSG 2026-07-16 6 Å 146k (1.58M) MPtrj
37 M3GNet 1.409 1.34 −1.286 6.8% 6.8% 2.1 228k EFSG 2022-09-20 5 Å 62.8k (188k) MPF
38 eqV2 S DeNS 1.676 1.387 0.142 1.9% 1.9% 1.287 31.2M EFSD 2024-10-18 12 Å 146k (1.58M) MPtrj
39 CHGNet 1.717 1.696 −1.682 15.5% 15.5% 1.862 413k EFSGM 2023-03-03 5 Å 146k (1.58M) MPtrj
40 ORB v2 MPtrj 1.726 1.454 0.022 2.9% 2.9% 1.649 25.2M EFSD 2024-10-14 10 Å 146k (1.58M) MPtrj
41 ORB v2 MPA 1.734 1.488 0.43 0% 0% 1.151 25.2M EFSD 2024-10-11 10 Å 3.25M (32.1M) MPtrj+Alex
42 eqV2 M 1.771 1.483 −0.159 5.8% 5.8% 2.325 86.6M EFSD 2024-10-18 12 Å 3.37M (102M) MPtrj+OMat24

Model Comparison: κSRME vs κSRE

This defaults to κSRME versus κSRE: stronger models sit toward the lower left, while separation between the axes reveals cancellation across mode contributions. Use the selectors to compare other properties; clicking a point opens that model below.

  • Params Params · 42 models
Log Scale

Model Inspector

ML vs DFT Lattice Thermal Conductivity

Test set: PhononDB PBE

103 crystal structures · 34 elements

Data by Togo

Materials Project crystals with PBE reference calculations test harmonic phonons and anharmonic lattice thermal conductivity at 300 K.

Public structures and thermal-conductivity references without non-analytical corrections.

Structures DFT references
H 6
He
Li 5
Be 8
B 6
C 2
N 8
O 9
F 5
Ne
Na 5
Mg 3
Al 8
Si 2
P 8
S 9
Cl 7
Ar
K 5
Ca 4
Sc
Ti
V
Cr
Mn
Fe
Co
Ni
Cu 8
Zn 8
Ga 8
Ge
As 8
Se 9
Br 7
Kr
Rb 5
Sr 1
Y
Zr
Nb
Mo
Tc
Ru
Rh
Pd
Ag 4
Cd 7
In 8
Sn
Sb 6
Te 11
I 8
Xe
Cs 1
Ba 4
La
Ce
Pr
Nd
Pm
Sm
Eu
Gd
Tb
Dy
Ho
Er
Tm
Yb
Lu
Hf
Ta
W
Re
Os
Ir
Pt
Au
Hg
Tl
Pb 3
Bi
Po
At
Rn
Fr
Ra
Ac
Th
Pa
U
Np
Pu
Am
Cm
Bk
Cf
Es
Fm
Md
No
Lr
Rf
Db
Sg
Bh
Hs
Mt
Ds
Rg
Cn
Nh
Fl
Mc
Lv
Ts
Og
Each element is counted once per structure.
Direction-averaged reference conductivity, with logarithmic bins to show the range across materials.

Methodology

The PhononDB PBE test set contains 103 Materials Project crystals. After one simultaneous cell-and-site relaxation, finite-displacement forces yield second- and third-order force constants and each material's conductivity.

κSRME compares mode contributions before summation, while κSRE compares only the final scalar conductivity, where cancellation can hide errors. Both range from 0 (perfect) to 2 (maximum error), with lower values better. κSRD retains the sign of the scalar error to show systematic under- or overprediction.

κSRME follows the method of Póta et al.. See the implementations for phonon and conductivity prediction and metric evaluation.