Optimizers
Usage with compile() & fit()
An optimizer is one of the two arguments required for compiling a Keras model:
import keras_core as keras
from keras_core import layers
model = keras.Sequential()
model.add(layers.Dense(64, kernel_initializer="uniform", input_shape=(10,)))
model.add(layers.Activation("softmax"))
opt = keras.optimizers.Adam(learning_rate=0.01)
model.compile(loss="categorical_crossentropy", optimizer=opt)
You can either instantiate an optimizer before passing it to model.compile() , as in the above example,
or you can pass it by its string identifier. In the latter case, the default parameters for the optimizer will be used.
# pass optimizer by name: default parameters will be used
model.compile(loss="categorical_crossentropy", optimizer='adam')
Learning rate decay / scheduling
You can use a learning rate schedule to modulate how the learning rate of your optimizer changes over time:
import keras_core as keras
lr_schedule = keras.optimizers.schedules.ExponentialDecay(
initial_learning_rate=1e-2,
decay_steps=10000,
decay_rate=0.9)
optimizer = keras.optimizers.SGD(learning_rate=lr_schedule)
Check out the learning rate schedule API documentation for a list of available schedules.
Available optimizers
Core Optimizer API
These methods and attributes are common to all Keras optimizers.
apply_gradients method
Optimizer.apply_gradients(grads_and_vars)
variables property
keras_core.optimizers.Optimizer.variables