Keras 3 API documentation
Models API Layers API Callbacks API Ops API Optimizers SGD RMSprop Adam AdamW Adadelta Adagrad Adamax Adafactor Nadam Ftrl Lion Lamb Loss Scale Optimizer MultiOptimizer OptimizerMap Learning rate schedules API Muon Metrics Losses Data loading Tree API Built-in small datasets Keras Applications Mixed precision Multi-device distribution RNG API Quantizers Scope Rematerialization Utilities
Keras 2 API documentation
► Keras 3 API documentation / Optimizers / MultiOptimizer

MultiOptimizer

[source]

MultiOptimizer class

keras.optimizers.MultiOptimizer(optimizer_map, loss_scale_factor=None, name=None)

An optimizer wrapper that delegates variables to different optimizers.

Initialize the object with an OptimizerMap instance or a callable function that returns an optimizer for a given variable.

Example

model.compile( optimizer=MultiOptimizer( OptimizerMap(default_optimizer=optimizers.SGD(), {"encoder/.*": optimizers.Adam()}) ), loss="binary_crossentropy", )

Or using a callable

def optimizer_selector(variable): if "encoder" in variable.path: return optimizers.Adam() else: return optimizers.SGD()

model.compile( optimizer=MultiOptimizer(optimizer_selector), loss="binary_crossentropy", )

To access the attributes of the sub-optimizers, iterate over the optimizers using .optimizers:

For example:

optimizer = MultiOptimizer(OptimizerMap( default_optimizer=optimizers.Adam() )) optimizer['.encoder'] = optimizers.SGD()

for optim in optimizer.optimizers: print(optim.learning_rate) print(optim.iterations) print(optim.loss_scale_factor) ...

The MultiOptimizer class instances will not expose learning_rate attribute and will raise an error if accessed. This is because the learning rate might be different for different sub-optimizers.

Note: Optimizer-specific callbacks are not supported yet.

MultiOptimizer
MultiOptimizer class
Or using a callable