pytorch-optimizer¶
Use more than 100 optimizers, learning rate schedulers, and loss functions with PyTorch.
Start with a named optimizer, or use create_optimizer() to combine weight decay, gradient centralization, and Lookahead.
from torch import nn
from pytorch_optimizer import create_optimizer
model = nn.Linear(10, 1)
optimizer = create_optimizer(model, 'adamp', lr=1e-3)
Start training¶
Follow the getting started guide for installation, a training step, and component discovery. For algorithm descriptions and links to papers, browse the supported algorithms.
API reference¶
| Reference | Contents |
|---|---|
| Optimizers | Optimizer classes, loaders, and parameter groups |
| Learning rate schedulers | Warmup, cosine, polynomial, and other schedules |
| Loss functions | Classification and segmentation objectives |
| Utilities | Component discovery, gradient operations, and CPU offloading |
| Base optimizer | Shared validation and update helpers for optimizer authors |
Compare and troubleshoot¶
- Compare runtime and memory in the foreach benchmarks.
- Inspect optimizer paths in the visualizations.
- Check the FAQ for Hessian computation and memory issues.
- Read the changelog for release notes.
Contribute¶
See the contributing guide to add an optimizer, improve the docs, or report a bug. Check the license notes before using code with additional terms.