StochasticDepth layer
StochasticDepth class
keras_cv.layers.StochasticDepth(rate=0.5, **kwargs)
Implements the Stochastic Depth layer. It randomly drops residual branches in residual architectures. It is used as a drop-in replacement for addition operation. Note that this layer DOES NOT drop a residual block across individual samples but across the entire batch.
Reference
- Deep Networks with Stochastic Depth - [Docstring taken from stochastic_depth.py
Arguments
- rate: float, the probability of the residual branch being dropped.
Example
StochasticDepth can be used in a residual network as follows:
# (...)
input = tf.ones((1, 3, 3, 1), dtype=tf.float32)
residual = keras.layers.Conv2D(1, 1)(input)
output = keras_cv.layers.StochasticDepth()([input, residual])
# (...)
At train time, StochasticDepth returns: $$ x[0] + b_l * x[1], $$ where $b_l$ is a random Bernoulli variable with probability $P(b_l = 1) = rate$. At test time, StochasticDepth rescales the activations of the residual branch based on the drop rate ($rate$): $$ x[0] + (1 - rate) * x[1] $$