ยป Keras API reference / KerasTuner / Oracles / Hyperband Oracle

Hyperband Oracle

HyperbandOracle class

keras_tuner.oracles.HyperbandOracle(
    objective,
    max_epochs,
    factor=3,
    hyperband_iterations=1,
    seed=None,
    hyperparameters=None,
    allow_new_entries=True,
    tune_new_entries=True,
)

Oracle class for Hyperband.

Note that to use this Oracle with your own subclassed Tuner, your Tuner class must be able to handle in Tuner.run_trial three special hyperparameters that will be set by this Tuner:

  • "tuner/trial_id": String, optionally set. The trial_id of the Trial to load from when starting this trial.
  • "tuner/initial_epoch": Int, always set. The initial epoch the Trial should be started from.
  • "tuner/epochs": Int, always set. The cumulative number of epochs this Trial should be trained.

These hyperparameters will be set during the "successive halving" portion of the Hyperband algorithm.

Examples

def run_trial(self, trial, *args, **kwargs):
    hp = trial.hyperparameters
    if "tuner/trial_id" in hp:
        past_trial = self.oracle.get_trial(hp['tuner/trial_id'])
        model = self.load_model(past_trial)
    else:
        model = self.hypermodel.build(hp)

    initial_epoch = hp['tuner/initial_epoch']
    last_epoch = hp['tuner/epochs']

    for epoch in range(initial_epoch, last_epoch):
        self.on_epoch_begin(...)
        for step in range(...):
            # Run model training step here.
        self.on_epoch_end(...)

Arguments

  • objective: A string or keras_tuner.Objective instance. If a string, the direction of the optimization (min or max) will be inferred.
  • max_epochs: Integer, the maximum number of epochs to train one model. It is recommended to set this to a value slightly higher than the expected epochs to convergence for your largest Model, and to use early stopping during training (for example, via tf.keras.callbacks.EarlyStopping).
  • factor: Integer, the reduction factor for the number of epochs and number of models for each bracket. Defaults to 3.
  • hyperband_iterations: Integer, at least 1, the number of times to iterate over the full Hyperband algorithm. One iteration will run approximately max_epochs * (math.log(max_epochs, factor) ** 2) cumulative epochs across all trials. It is recommended to set this to as high a value as is within your resource budget. Defaults to 1.
  • seed: Optional integer, the random seed.
  • hyperparameters: Optional HyperParameters instance. Can be used to override (or register in advance) hyperparameters in the search space.
  • tune_new_entries: Boolean, whether hyperparameter entries that are requested by the hypermodel but that were not specified in hyperparameters should be added to the search space, or not. If not, then the default value for these parameters will be used. Defaults to True.
  • allow_new_entries: Boolean, whether the hypermodel is allowed to request hyperparameter entries not listed in hyperparameters. Defaults to True.