StochasticDepth layer

[source]

StochasticDepth class

keras_cv.layers.StochasticDepth(rate=0.5, **kwargs)

Implements the Stochastic Depth layer. It randomly drops residual branches in residual architectures. It is used as a drop-in replacement for addition operation. Note that this layer DOES NOT drop a residual block across individual samples but across the entire batch.

Reference

Arguments

  • rate: float, the probability of the residual branch being dropped.

Example

StochasticDepth can be used in a residual network as follows:

# (...)
input = tf.ones((1, 3, 3, 1), dtype=tf.float32)
residual = keras.layers.Conv2D(1, 1)(input)
output = keras_cv.layers.StochasticDepth()([input, residual])
# (...)

At train time, StochasticDepth returns: $$ x[0] + b_l * x[1], $$ where $b_l$ is a random Bernoulli variable with probability $P(b_l = 1) = rate$. At test time, StochasticDepth rescales the activations of the residual branch based on the drop rate ($rate$): $$ x[0] + (1 - rate) * x[1] $$