pytorch-optimizer¶
Choose from more than 100 optimizers, plus learning rate schedulers and loss functions for PyTorch.
Use an optimizer class or select an optimizer by name.
Use create_optimizer() to configure weight decay, gradient centralization, and Lookahead.
from torch import nn
from pytorch_optimizer import create_optimizer
model = nn.Linear(10, 1)
optimizer = create_optimizer(model, 'adamp', lr=1e-3)
Start training¶
Follow the getting started guide to install the package and run a training step. The guide also shows how to find component names. Read the supported algorithms for descriptions and links to papers.
API reference¶
| Reference | Contents |
|---|---|
| Optimizers | Optimizer classes, loaders, and parameter groups |
| Learning rate schedulers | Warmup, cosine, polynomial, and other schedules |
| Loss functions | Classification and segmentation objectives |
| Utilities | Component discovery, gradient operations, and CPU offloading |
| Base optimizer | Shared validation and update helpers for optimizer authors |
Compare and troubleshoot¶
- Compare execution time and memory use in the optimizer benchmarks.
- Compare optimizer paths in the visualizations.
- Check the FAQ for Hessian computation and memory issues.
- Read the changelog for release notes.
Contribute¶
Follow the contributing guide to add an optimizer, revise documentation, or report a bug. Check the license notes before you use code with additional terms.