Layer weight regularizers

[source]

L1 class

tf_keras.regularizers.L1(l1=0.01, **kwargs)

A regularizer that applies a L1 regularization penalty.

The L1 regularization penalty is computed as: loss = l1 * reduce_sum(abs(x))

L1 may be passed to a layer as a string identifier:

>>> dense = tf.keras.layers.Dense(3, kernel_regularizer='l1')

In this case, the default value used is l1=0.01.

Arguments

• l1: Float; L1 regularization factor.

[source]

L2 class

tf_keras.regularizers.L2(l2=0.01, **kwargs)

A regularizer that applies a L2 regularization penalty.

The L2 regularization penalty is computed as: loss = l2 * reduce_sum(square(x))

L2 may be passed to a layer as a string identifier:

>>> dense = tf.keras.layers.Dense(3, kernel_regularizer='l2')

In this case, the default value used is l2=0.01.

Arguments

• l2: Float; L2 regularization factor.

[source]

L1L2 class

tf_keras.regularizers.L1L2(l1=0.0, l2=0.0)

A regularizer that applies both L1 and L2 regularization penalties.

The L1 regularization penalty is computed as: loss = l1 * reduce_sum(abs(x))

The L2 regularization penalty is computed as loss = l2 * reduce_sum(square(x))

L1L2 may be passed to a layer as a string identifier:

>>> dense = tf.keras.layers.Dense(3, kernel_regularizer='l1_l2')

In this case, the default values used are l1=0.01 and l2=0.01.

Arguments

• l1: Float; L1 regularization factor.
• l2: Float; L2 regularization factor.

[source]

OrthogonalRegularizer class

tf_keras.regularizers.OrthogonalRegularizer(factor=0.01, mode="rows")

Regularizer that encourages input vectors to be orthogonal to each other.

It can be applied to either the rows of a matrix (mode="rows") or its columns (mode="columns"). When applied to a Dense kernel of shape (input_dim, units), rows mode will seek to make the feature vectors (i.e. the basis of the output space) orthogonal to each other.

Arguments

• factor: Float. The regularization factor. The regularization penalty will be proportional to factor times the mean of the dot products between the L2-normalized rows (if mode="rows", or columns if mode="columns") of the inputs, excluding the product of each row/column with itself. Defaults to 0.01.
• mode: String, one of {"rows", "columns"}. Defaults to "rows". In rows mode, the regularization effect seeks to make the rows of the input orthogonal to each other. In columns mode, it seeks to make the columns of the input orthogonal to each other.

Example

>>> regularizer = tf.keras.regularizers.OrthogonalRegularizer(factor=0.01)
>>> layer = tf.keras.layers.Dense(units=4, kernel_regularizer=regularizer)