Visualization
| Optimizer |
Rastrigin |
Rosenbrock |
| ADOPT |
 |
 |
| APOLLO |
 |
 |
| ASGD |
 |
 |
| AccSGD |
 |
 |
| AdEMAMix |
 |
 |
| AdaBelief |
 |
 |
| AdaBound |
 |
 |
| AdaDelta |
 |
 |
| AdaFactor |
 |
 |
| AdaGC |
 |
 |
| AdaHessian |
 |
 |
| AdaMax |
 |
 |
| AdaMod |
 |
 |
| AdaNorm |
 |
 |
| AdaPNM |
 |
 |
| AdaShift |
 |
 |
| AdaSmooth |
 |
 |
| AdaTAM |
 |
 |
| Adai |
 |
 |
| Adalite |
 |
 |
| Adam |
 |
 |
| AdamG |
 |
 |
| AdamMini |
 |
 |
| AdamP |
 |
 |
| AdamS |
 |
 |
| AdamW |
 |
 |
| Adan |
 |
 |
| AggMo |
 |
 |
| Aida |
 |
 |
| AliG |
 |
 |
| Amos |
 |
 |
| ApolloDQN |
 |
 |
| AvaGrad |
 |
 |
| BSAM |
 |
 |
| BCOS |
 |
 |
| CAME |
 |
 |
| DAdaptAdaGrad |
 |
 |
| DAdaptAdam |
 |
 |
| DAdaptAdan |
 |
 |
| DAdaptLion |
 |
 |
| DAdaptSGD |
 |
 |
| DiffGrad |
 |
 |
| EXAdam |
 |
 |
| EmoFact |
 |
 |
| EmoLynx |
 |
 |
| EmoNavi |
 |
 |
| FAdam |
 |
 |
| FOCUS |
 |
 |
| FTRL |
 |
 |
| Fira |
 |
 |
| Fromage |
 |
 |
| GaLore |
 |
 |
| Grams |
 |
 |
| Gravity |
 |
 |
| GrokFastAdamW |
 |
 |
| Kate |
 |
 |
| Kron |
 |
 |
| LARS |
 |
 |
| LaProp |
 |
 |
| Lamb |
 |
 |
| Lion |
 |
 |
| MADGRAD |
 |
 |
| MARS |
 |
 |
| MSVAG |
 |
 |
| Nero |
 |
 |
| NovoGrad |
 |
 |
| PAdam |
 |
 |
| PID |
 |
 |
| PNM |
 |
 |
| Prodigy |
 |
 |
| QHAdam |
 |
 |
| QHM |
 |
 |
| RACS |
 |
 |
| RAdam |
 |
 |
| Ranger |
 |
 |
| Ranger21 |
 |
 |
| Ranger25 |
 |
 |
| SCION |
 |
 |
| SCIONLight |
 |
 |
| SGD |
 |
 |
| SGDP |
 |
 |
| SGDSaI |
 |
 |
| SGDW |
 |
 |
| SM3 |
 |
 |
| SOAP |
 |
 |
| SPAM |
 |
 |
| SRMM |
 |
 |
| SWATS |
 |
 |
| ScalableShampoo |
 |
 |
| ScheduleFreeAdamW |
 |
 |
| ScheduleFreeRAdam |
 |
 |
| ScheduleFreeSGD |
 |
 |
| Shampoo |
 |
 |
| SignSGD |
 |
 |
| SimplifiedAdEMAMix |
 |
 |
| SophiaH |
 |
 |
| StableAdamW |
 |
 |
| StableSPAM |
 |
 |
| TAM |
 |
 |
| Tiger |
 |
 |
| VSGD |
 |
 |
| Yogi |
 |
 |