Keras backends

What is a "backend"?

Keras is a model-level library, providing high-level building blocks for developing deep learning models. It does not handle itself low-level operations such as tensor products, convolutions and so on. Instead, it relies on a specialized, well-optimized tensor manipulation library to do so, serving as the "backend engine" of Keras. Rather than picking one single tensor library and making the implementation of Keras tied to that library, Keras handles the problem in a modular way, and several different backend engines can be plugged seamlessly into Keras.

At this time, Keras has two backend implementations available: the TensorFlow backend and the Theano backend.

  • TensorFlow is an open-source symbolic tensor manipulation framework developed by Google, Inc.
  • Theano is an open-source symbolic tensor manipulation framework developed by LISA/MILA Lab at Université de Montréal.

In the future, we are likely to add more backend options. If you are interested in developing a new backend, get in touch!


Switching from one backend to another

If you have run Keras at least once, you will find the Keras configuration file at:

~/.keras/keras.json

If it isn't there, you can create it.

The default configuration file looks like this:

{
    "image_dim_ordering": "tf",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

Simply change the field backend to either "theano" or "tensorflow", and Keras will use the new configuration next time you run any Keras code.

You can also define the environment variable KERAS_BACKEND and this will override what is defined in your config file :

KERAS_BACKEND=tensorflow python -c "from keras import backend"
Using TensorFlow backend.

keras.json details

{
    "image_dim_ordering": "tf",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

You can change these settings by editing ~/.keras/keras.json.

  • image_dim_ordering: string, either "tf" or "th". It specifies which dimension ordering convention Keras will follow. (keras.backend.image_dim_ordering() returns it.)
  • For 2D data (e.g. image), "tf" assumes (rows, cols, channels) while "th" assumes (channels, rows, cols).
  • For 3D data, "tf" assumes (conv_dim1, conv_dim2, conv_dim3, channels) while "th" assumes (channels, conv_dim1, conv_dim2, conv_dim3).
  • epsilon: float, a numeric fuzzing constant used to avoid dividing by zero in some operations.
  • floatx: string, "float16", "float32", or "float64". Default float precision.
  • backend: string, "tensorflow" or "theano".

Using the abstract Keras backend to write new code

If you want the Keras modules you write to be compatible with both Theano and TensorFlow, you have to write them via the abstract Keras backend API. Here's an intro.

You can import the backend module via:

from keras import backend as K

The code below instantiates an input placeholder. It's equivalent to tf.placeholder() or T.matrix(), T.tensor3(), etc.

input = K.placeholder(shape=(2, 4, 5))
# also works:
input = K.placeholder(shape=(None, 4, 5))
# also works:
input = K.placeholder(ndim=3)

The code below instantiates a shared variable. It's equivalent to tf.variable() or theano.shared().

val = np.random.random((3, 4, 5))
var = K.variable(value=val)

# all-zeros variable:
var = K.zeros(shape=(3, 4, 5))
# all-ones:
var = K.ones(shape=(3, 4, 5))

Most tensor operations you will need can be done as you would in TensorFlow or Theano:

a = b + c * K.abs(d)
c = K.dot(a, K.transpose(b))
a = K.sum(b, axis=2)
a = K.softmax(b)
a = concatenate([b, c], axis=-1)
# etc...

Backend functions

epsilon

epsilon()

Returns the value of the fuzz factor used in numeric expressions.

Returns

A float.

Example

>>> keras.backend.epsilon()
1e-08

set_epsilon

set_epsilon(e)

Sets the value of the fuzz factor used in numeric expressions.

Arguments

  • e: float. New value of epsilon.

Example

>>> from keras import backend as K
>>> K.epsilon()
1e-08
>>> K.set_epsilon(1e-05)
>>> K.epsilon()
1e-05

floatx

floatx()

Returns the default float type, as a string (e.g. 'float16', 'float32', 'float64').

Returns

String, the current default float type.

Example

>>> keras.backend.floatx()
'float32'

set_floatx

set_floatx(floatx)

Sets the default float type.

Arguments

  • String: 'float16', 'float32', or 'float64'.

Example

>>> from keras import backend as K
>>> K.floatx()
'float32'
>>> K.set_floatx('float16')
>>> K.floatx()
'float16'

cast_to_floatx

cast_to_floatx(x)

Cast a Numpy array to the default Keras float type.

Arguments

  • x: Numpy array.

Returns

The same Numpy array, cast to its new type.

Example

>>> from keras import backend as K
>>> K.floatx()
'float32'
>>> arr = numpy.array([1.0, 2.0], dtype='float64')
>>> arr.dtype
dtype('float64')
>>> new_arr = K.cast_to_floatx(arr)
>>> new_arr
array([ 1.,  2.], dtype=float32)
>>> new_arr.dtype
dtype('float32')

image_dim_ordering

image_dim_ordering()

Returns the default image dimension ordering convention ('th' or 'tf').

Returns

A string, either 'th' or 'tf'

Example

>>> keras.backend.image_dim_ordering()
'th'

set_image_dim_ordering

set_image_dim_ordering(dim_ordering)

Sets the value of the image dimension ordering convention ('th' or 'tf').

Arguments

  • dim_ordering: string. 'th' or 'tf'.

Example

>>> from keras import backend as K
>>> K.image_dim_ordering()
'th'
>>> K.set_image_dim_ordering('tf')
>>> K.image_dim_ordering()
'tf'

get_uid

get_uid(prefix='')

Provides a unique UID given a string prefix.

Arguments

  • prefix: string.

Returns

An integer.

Example

>>> keras.backend.get_uid('dense')
>>> 1
>>> keras.backend.get_uid('dense')
>>> 2

is_keras_tensor

is_keras_tensor(x)

Returns whether x is a Keras tensor.

Arguments

  • x: a potential tensor.

Returns

A boolean: whether the argument is a Keras tensor.

Examples

>>> from keras import backend as K
>>> np_var = numpy.array([1, 2])
>>> K.is_keras_tensor(np_var)
False
>>> keras_var = K.variable(np_var)
>>> K.is_keras_tensor(keras_var)  # A variable is not a Tensor.
False
>>> keras_placeholder = K.placeholder(shape=(2, 4, 5))
>>> K.is_keras_tensor(keras_placeholder)  # A placeholder is a Tensor.
True

clear_session

clear_session()

Destroys the current TF graph and creates a new one.

Useful to avoid clutter from old models / layers.


manual_variable_initialization

manual_variable_initialization(value)

Sets the manual variable initialization flag.

This boolean flag determines whether variables should be initialized as they are instantiated (default), or if the user should handle the initialization (e.g. via tf.initialize_all_variables()).

Arguments

  • value: Python boolean.

learning_phase

learning_phase()

Returns the learning phase flag.

The learning phase flag is a bool tensor (0 = test, 1 = train) to be passed as input to any Keras function that uses a different behavior at train time and test time.


set_learning_phase

set_learning_phase(value)

Sets the learning phase to a fixed value, either 0 or 1 (integers).


is_sparse

is_sparse(tensor)

Returns whether a tensor is a sparse tensor.

Arguments

  • tensor: A tensor instance.

Returns

A boolean.

Example

>>> from keras import backend as K
>>> a = K.placeholder((2, 2), sparse=False)
>>> print(K.is_sparse(a))
False
>>> b = K.placeholder((2, 2), sparse=True)
>>> print(K.is_sparse(b))
True

to_dense

to_dense(tensor)

Converts a sparse tensor into a dense tensor and returns it.

Arguments

  • tensor: A tensor instance (potentially sparse).

Returns

A dense tensor.

Examples

>>> from keras import backend as K
>>> b = K.placeholder((2, 2), sparse=True)
>>> print(K.is_sparse(b))
True
>>> c = K.to_dense(b)
>>> print(K.is_sparse(c))
False

variable

variable(value, dtype=None, name=None)

Instantiates a variable and returns it.

Arguments

  • value: Numpy array, initial value of the tensor.
  • dtype: Tensor type.
  • name: Optional name string for the tensor.

Returns

A variable instance (with Keras metadata included).

Examples

>>> from keras import backend as K
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val, dtype='float64', name='example_var')
>>> K.dtype(kvar)
'float64'
>>> print(kvar)
example_var
>>> kvar.eval()
array([[ 1.,  2.],
   [ 3.,  4.]])

placeholder

placeholder(shape=None, ndim=None, dtype=None, sparse=False, name=None)

Instantiates a placeholder tensor and returns it.

Arguments

  • shape: Shape of the placeholder (integer tuple, may include None entries).
  • ndim: Number of axes of the tensor. At least one of {shape, ndim} must be specified. If both are specified, shape is used.
  • dtype: Placeholder type.
  • name: Optional name string for the placeholder.

Returns

Tensor instance (with Keras metadata included).

Examples

>>> from keras import backend as K
>>> input_ph = K.placeholder(shape=(2, 4, 5))
>>> input_ph._keras_shape
(2, 4, 5)
>>> input_ph
<tf.Tensor 'Placeholder_4:0' shape=(2, 4, 5) dtype=float32>

shape

shape(x)

Returns the symbolic shape of a tensor or variable.

Arguments

  • x: A tensor or variable.

Returns

A symbolic shape (which is itself a tensor).

Examples

__TensorFlow example__

>>> from keras import backend as K
>>> tf_session = K.get_session()
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> input = keras.backend.placeholder(shape=(2, 4, 5))
>>> K.shape(kvar)
<tf.Tensor 'Shape_8:0' shape=(2,) dtype=int32>
>>> K.shape(input)
<tf.Tensor 'Shape_9:0' shape=(3,) dtype=int32>
__To get integer shape (Instead, you can use K.int_shape(x))__

>>> K.shape(kvar).eval(session=tf_session)
array([2, 2], dtype=int32)
>>> K.shape(input).eval(session=tf_session)
array([2, 4, 5], dtype=int32)

int_shape

int_shape(x)

Returns the shape of a Keras tensor or a Keras variable as a tuple of integers or None entries.

Arguments

  • x: Tensor or variable.

Returns

A tuple of integers (or None entries).

Examples

>>> from keras import backend as K
>>> input = K.placeholder(shape=(2, 4, 5))
>>> K.int_shape(input)
(2, 4, 5)
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.int_shape(kvar)
(2, 2)

ndim

ndim(x)

Returns the number of axes in a tensor, as an integer.

Arguments

  • x: Tensor or variable.

Returns

Integer (scalar), number of axes.

Examples

>>> from keras import backend as K
>>> input = K.placeholder(shape=(2, 4, 5))
>>> val = np.array([[1, 2], [3, 4]])
>>> kvar = K.variable(value=val)
>>> K.ndim(input)
3
>>> K.ndim(kvar)
2

dtype

dtype(x)

Returns the dtype of a Keras tensor or variable, as a string.

Arguments

  • x: Tensor or variable.

Returns

String, dtype of x.

Examples

>>> from keras import backend as K
>>> K.dtype(K.placeholder(shape=(2,4,5)))
'float32'
>>> K.dtype(K.placeholder(shape=(2,4,5), dtype='float32'))
'float32'
>>> K.dtype(K.placeholder(shape=(2,4,5), dtype='float64'))
'float64'
__Keras variable__

>>> kvar = K.variable(np.array([[1, 2], [3, 4]]))
>>> K.dtype(kvar)
'float32_ref'
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]), dtype='float32')
>>> K.dtype(kvar)
'float32_ref'

eval

eval(x)

Evaluates the value of a variable. Returns a Numpy array.

Arguments

  • x: A variable.

Returns

A Numpy array.

Examples

>>> from keras import backend as K
>>> kvar = K.variable(np.array([[1, 2], [3, 4]]), dtype='float32')
>>> K.eval(kvar)
array([[ 1.,  2.],
   [ 3.,  4.]], dtype=float32)

zeros

zeros(shape, dtype=None, name=None)

Instantiates an all-zeros variable and returns it.

Arguments

  • shape: Tuple of integers, shape of returned Keras variable
  • dtype: String, data type of returned Keras variable
  • name: String, name of returned Keras variable

Returns

A variable (including Keras metadata), filled with 0.0.

Example

>>> from keras import backend as K
>>> kvar = K.zeros((3,4))
>>> K.eval(kvar)
array([[ 0.,  0.,  0.,  0.],
   [ 0.,  0.,  0.,  0.],
   [ 0.,  0.,  0.,  0.]], dtype=float32)

ones

ones(shape, dtype=None, name=None)

Instantiates an all-ones tensor variable and returns it.

Arguments

  • shape: Tuple of integers, shape of returned Keras variable.
  • dtype: String, data type of returned Keras variable.
  • name: String, name of returned Keras variable.

Returns

A Keras variable, filled with 1.0.

Example

>>> from keras import backend as K
>>> kvar = K.ones((3,4))
>>> K.eval(kvar)
array([[ 1.,  1.,  1.,  1.],
   [ 1.,  1.,  1.,  1.],
   [ 1.,  1.,  1.,  1.]], dtype=float32)

eye

eye(size, dtype=None, name=None)

Instantiate an identity matrix and returns it.

Arguments

  • size: Integer, number of rows/columns.
  • dtype: String, data type of returned Keras variable.
  • name: String, name of returned Keras variable.

Returns

A Keras variable, an identity matrix.

Example

>>> from keras import backend as K
>>> kvar = K.eye(3)
>>> K.eval(kvar)
array([[ 1.,  0.,  0.],
   [ 0.,  1.,  0.],
   [ 0.,  0.,  1.]], dtype=float32)

zeros_like

zeros_like(x, name=None)

Instantiates an all-zeros Keras variable of the same shape as another Keras variable or tensor and returns it.

Arguments

  • x: Keras variable or Keras tensor.

Returns

A Keras variable, filled with 0.0.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.random.random((2,3)))
>>> kvar_zeros = K.zeros_like(kvar)
>>> K.eval(kvar_zeros)
array([[ 0.,  0.,  0.],
   [ 0.,  0.,  0.]], dtype=float32)

ones_like

ones_like(x, name=None)

Instantiates an all-ones Keras variable of the same shape as another Keras variable or tensor and returns it.

Arguments

  • x: Keras variable or tensor.

Returns

A Keras variable, filled with 1.0.

Example

>>> from keras import backend as K
>>> kvar = K.variable(np.random.random((2,3)))
>>> kvar_ones = K.ones_like(kvar)
>>> K.eval(kvar_ones)
array([[ 1.,  1.,  1.],
   [ 1.,  1.,  1.]], dtype=float32)

random_uniform_variable

random_uniform_variable(shape, low, high, dtype=None, name=None, seed=None)

Instantiates an Keras variable filled with samples drawn from a uniform distribution and returns it.

Arguments

  • shape: Tuple of integers, shape of returned Keras variable.
  • low: Float, lower boundary of the output inteval.
  • high: Float, upper boundary of the output interval.
  • dtype: String, dtype of returned Keras variable.
  • name: String, name of returned Keras variable.
  • seed: Integer, random seed.

Returns

A Keras variable, filled with drawn samples.

Example

__TensorFlow example__

>>> kvar = K.random_uniform_variable((2,3), 0, 1)
>>> kvar
<tensorflow.python.ops.variables.Variable object at 0x10ab40b10>
>>> K.eval(kvar)
array([[ 0.10940075,  0.10047495,  0.476143  ],
   [ 0.66137183,  0.00869417,  0.89220798]], dtype=float32)

random_normal_variable

random_normal_variable(shape, mean, scale, dtype=None, name=None, seed=None)

Instantiates an Keras variable filled with samples drawn from a normal distribution and returns it.

Arguments

  • shape: Tuple of integers, shape of returned Keras variable.
  • mean: Float, mean of the normal distribution.
  • scale: Float, standard deviation of the normal distribution.
  • dtype: String, dtype of returned Keras variable.
  • name: String, name of returned Keras variable.
  • seed: Integer, random seed.

Returns

A Keras variable, filled with drawn samples.

Example

__TensorFlow example__

>>> kvar = K.random_normal_variable((2,3), 0, 1)
>>> kvar
<tensorflow.python.ops.variables.Variable object at 0x10ab12dd0>
>>> K.eval(kvar)
array([[ 1.19591331,  0.68685907, -0.63814116],
   [ 0.92629528,  0.28055015,  1.70484698]], dtype=float32)

count_params

count_params(x)

Returns the number of scalars in a Keras variable.

Arguments

  • x: Keras variable.

Returns

Integer, the number of scalars in x.

Example

>>> kvar = K.zeros((2,3))
>>> K.count_params(kvar)
6
>>> K.eval(kvar)
array([[ 0.,  0.,  0.],
   [ 0.,  0.,  0.]], dtype=float32)

cast

cast(x, dtype)

Casts a tensor to a different dtype and returns it.

You can cast a Keras variable but it still returns a Keras tensor.

Arguments

  • x: Keras tensor (or variable).
  • dtype: String, either ('float16', 'float32', or 'float64').

Returns

Keras tensor with dtype dtype.

Example

>>> from keras import backend as K
>>> input = K.placeholder((2, 3), dtype='float32')
>>> input
<tf.Tensor 'Placeholder_2:0' shape=(2, 3) dtype=float32>
__It doesn't work in-place as below.__

>>> K.cast(input, dtype='float16')
<tf.Tensor 'Cast_1:0' shape=(2, 3) dtype=float16>
>>> input
<tf.Tensor 'Placeholder_2:0' shape=(2, 3) dtype=float32>
__you need to assign it.__

>>> input = K.cast(input, dtype='float16')
>>> input
<tf.Tensor 'Cast_2:0' shape=(2, 3) dtype=float16>```

----

### dot


```python
dot(x, y)

Multiplies 2 tensors (and/or variables) and returns a tensor. When attempting to multiply a ND tensor with a ND tensor, it reproduces the Theano behavior. (e.g. (2, 3).(4, 3, 5) = (2, 4, 5))

Arguments

  • x: Tensor or variable.
  • y: Tensor or variable.

Returns

A tensor, dot product of x and y.

Examples

__dot product between tensors__

>>> x = K.placeholder(shape=(2, 3))
>>> y = K.placeholder(shape=(3, 4))
>>> xy = K.dot(x, y)
>>> xy
<tf.Tensor 'MatMul_9:0' shape=(2, 4) dtype=float32>
__dot product between tensors__

>>> x = K.placeholder(shape=(32, 28, 3))
>>> y = K.placeholder(shape=(3, 4))
>>> xy = K.dot(x, y)
>>> xy
<tf.Tensor 'MatMul_9:0' shape=(32, 28, 4) dtype=float32>
__Theano-like behavior example__

>>> x = K.random_uniform_variable(shape=(2, 3), low=0, high=1)
>>> y = K.ones((4, 3, 5))
>>> xy = K.dot(x, y)
>>> K.int_shape(xy)
(2, 4, 5)

batch_dot

batch_dot(x, y, axes=None)

Batchwise dot product.

batch_dot is used to compute dot product of x and y when x and y are data in batch, i.e. in a shape of (batch_size, :). batch_dot results in a tensor or variable with less dimensions than the input. If the number of dimensions is reduced to 1, we use expand_dims to make sure that ndim is at least 2.

Arguments

x, y: Keras tensors or variables with ndim >= 2 (With TensorFlow backend, batch_dot() only supports ndim >= 3) - axes: list of (or single) int with target dimensions. The lengths of axes[0] and axes[1] should be the same.

Returns

A tensor with shape equal to the concatenation of x's shape (less the dimension that was summed over) and y's shape (less the batch dimension and the dimension that was summed over). If the final rank is 1, we reshape it to (batch_size, 1).

Examples

Assume x = [[1, 2], [3, 4]] and y = [[5, 6], [7, 8]] batch_dot(x, y, axes=1) = [[17, 53]] which is the main diagonal of x.dot(y.T), although we never have to calculate the off-diagonal elements.

Shape inference: Let x's shape be (100, 20) and y's shape be (100, 30, 20). If axes is (1, 2), to find the output shape of resultant tensor, loop through each dimension in x's shape and y's shape:

  • x.shape[0] : 100 : append to output shape
  • x.shape[1] : 20 : do not append to output shape, dimension 1 of x has been summed over. (dot_axes[0] = 1)
  • y.shape[0] : 100 : do not append to output shape, always ignore first dimension of y
  • y.shape[1] : 30 : append to output shape
  • y.shape[2] : 20 : do not append to output shape, dimension 2 of y has been summed over. (dot_axes[1] = 2) output_shape = (100, 30)
>>> x_batch = K.ones(shape=(32, 20, 1))
>>> y_batch = K.ones(shape=(32, 30, 20))
>>> xy_batch_dot = K.batch_dot(x_batch, y_batch, axes=[1, 2])
>>> K.int_shape(xy_batch_dot)
(32, 1, 30)

transpose

transpose(x)

Transposes a tensor and returns it.

Arguments

  • x: Tensor or variable.

Returns

A tensor.

Examples

>>> var = K.variable([[1, 2, 3], [4, 5, 6]])
>>> K.eval(var)
array([[ 1.,  2.,  3.],
   [ 4.,  5.,  6.]], dtype=float32)
>>> var_transposed = K.transpose(var)
>>> K.eval(var_transposed)
array([[ 1.,  4.],
   [ 2.,  5.],
   [ 3.,  6.]], dtype=float32)
>>> input = K.placeholder((2, 3))
>>> input
<tf.Tensor 'Placeholder_11:0' shape=(2, 3) dtype=float32>
>>> input_transposed = K.transpose(input)
>>> input_transposed
<tf.Tensor 'transpose_4:0' shape=(3, 2) dtype=float32>


gather

gather(reference, indices)

Retrieves the elements of indices indices in the tensor reference.

Arguments

  • reference: A tensor.
  • indices: An integer tensor of indices.

Returns

A tensor of same type as reference.


max

max(x, axis=None, keepdims=False)

Maximum value in a tensor.


min

min(x, axis=None, keepdims=False)

Minimum value in a tensor.


sum

sum(x, axis=None, keepdims=False)

Sum of the values in a tensor, alongside the specified axis.


prod

prod(x, axis=None, keepdims=False)

Multiplies the values in a tensor, alongside the specified axis.


var

var(x, axis=None, keepdims=False)

Variance of a tensor, alongside the specified axis.


std

std(x, axis=None, keepdims=False)

Standard deviation of a tensor, alongside the specified axis.


mean

mean(x, axis=None, keepdims=False)

Mean of a tensor, alongside the specified axis.


any

any(x, axis=None, keepdims=False)

Bitwise reduction (logical OR).

Returns an uint8 tensor (0s and 1s).


all

all(x, axis=None, keepdims=False)

Bitwise reduction (logical AND).

Returns an uint8 tensor


argmax

argmax(x, axis=-1)

Returns the index of the maximum value along a tensor axis.


argmin

argmin(x, axis=-1)

Returns the index of the minimum value along a tensor axis.


square

square(x)

Element-wise square.


abs

abs(x)

Element-wise absolute value.


sqrt

sqrt(x)

Element-wise square root.


exp

exp(x)

Element-wise exponential.


log

log(x)

Element-wise log.


round

round(x)

Element-wise rounding to the closest integer.


sign

sign(x)

Element-wise sign.


pow

pow(x, a)

Element-wise exponentiation.


clip

clip(x, min_value, max_value)

Element-wise value clipping.


equal

equal(x, y)

Element-wise equality between two tensors. Returns a bool tensor.


not_equal

not_equal(x, y)

Element-wise inequality between two tensors. Returns a bool tensor.


greater

greater(x, y)

Element-wise truth value of (x > y). Returns a bool tensor.


greater_equal

greater_equal(x, y)

Element-wise truth value of (x >= y). Returns a bool tensor.


lesser

lesser(x, y)

Element-wise truth value of (x < y). Returns a bool tensor.


lesser_equal

lesser_equal(x, y)

Element-wise truth value of (x <= y). Returns a bool tensor.


maximum

maximum(x, y)

Element-wise maximum of two tensors.


minimum

minimum(x, y)

Element-wise minimum of two tensors.


sin

sin(x)

Computes sin of x element-wise.


cos

cos(x)

Computes cos of x element-wise.


normalize_batch_in_training

normalize_batch_in_training(x, gamma, beta, reduction_axes, epsilon=0.001)

Computes mean and std for batch then apply batch_normalization on batch.


batch_normalization

batch_normalization(x, mean, var, beta, gamma, epsilon=0.001)

Applies batch normalization on x given mean, var, beta and gamma:

output = (x - mean) / (sqrt(var) + epsilon) * gamma + beta


concatenate

concatenate(tensors, axis=-1)

Concatenates a list of tensors alongside the specified axis.


reshape

reshape(x, shape)

Reshapes a tensor to the specified shape.


permute_dimensions

permute_dimensions(x, pattern)

Permutes axes in a tensor.

Arguments

  • pattern: should be a tuple of dimension indices, e.g. (0, 2, 1).

resize_images

resize_images(X, height_factor, width_factor, dim_ordering)

Resizes the images contained in a 4D tensor of shape - [batch, channels, height, width] (for 'th' dim_ordering) - [batch, height, width, channels] (for 'tf' dim_ordering) by a factor of (height_factor, width_factor). Both factors should be positive integers.


resize_volumes

resize_volumes(X, depth_factor, height_factor, width_factor, dim_ordering)

Resizes the volume contained in a 5D tensor of shape - [batch, channels, depth, height, width] (for 'th' dim_ordering) - [batch, depth, height, width, channels] (for 'tf' dim_ordering) by a factor of (depth_factor, height_factor, width_factor). All three factors should be positive integers.


repeat_elements

repeat_elements(x, rep, axis)

Repeats the elements of a tensor along an axis, like np.repeat

If x has shape (s1, s2, s3) and axis=1, the output will have shape (s1, s2 * rep, s3)


repeat

repeat(x, n)

Repeats a 2D tensor:

if x has shape (samples, dim) and n=2, the output will have shape (samples, 2, dim)


arange

arange(start, stop=None, step=1, dtype='int32')

Creates a 1-D tensor containing a sequence of integers.

The function arguments use the same convention as Theano's arange: if only one argument is provided, it is in fact the "stop" argument.

The default type of the returned tensor is 'int32' to match TensorFlow's default.


batch_flatten

batch_flatten(x)

Turn a n-D tensor into a 2D tensor where the first dimension is conserved.


expand_dims

expand_dims(x, dim=-1)

Adds a 1-sized dimension at index "dim".


squeeze

squeeze(x, axis)

Removes a 1-dimension from the tensor at index "axis".


temporal_padding

temporal_padding(x, padding=1)

Pads the middle dimension of a 3D tensor with "padding" zeros left and right.


asymmetric_temporal_padding

asymmetric_temporal_padding(x, left_pad=1, right_pad=1)

Pad the middle dimension of a 3D tensor with "left_pad" zeros left and "right_pad" right.


spatial_2d_padding

spatial_2d_padding(x, padding=(1, 1), dim_ordering='default')

Pads the 2nd and 3rd dimensions of a 4D tensor with "padding[0]" and "padding[1]" (resp.) zeros left and right.


asymmetric_spatial_2d_padding

asymmetric_spatial_2d_padding(x, top_pad=1, bottom_pad=1, left_pad=1, right_pad=1, dim_ordering='default')

Pad the rows and columns of a 4D tensor with "top_pad", "bottom_pad", "left_pad", "right_pad" (resp.) zeros rows on top, bottom; cols on left, right.


spatial_3d_padding

spatial_3d_padding(x, padding=(1, 1, 1), dim_ordering='default')

Pads 5D tensor with zeros for the depth, height, width dimension with "padding[0]", "padding[1]" and "padding[2]" (resp.) zeros left and right

For 'tf' dim_ordering, the 2nd, 3rd and 4th dimension will be padded. For 'th' dim_ordering, the 3rd, 4th and 5th dimension will be padded.


one_hot

one_hot(indices, nb_classes)

Input: nD integer tensor of shape (batch_size, dim1, dim2, ... dim(n-1)) - Output: (n + 1)D one hot representation of the input with shape (batch_size, dim1, dim2, ... dim(n-1), nb_classes)


reverse

reverse(x, axes)

Reverse a tensor along the the specified axes


get_value

get_value(x)

Returns the value of a tensor variable, as a Numpy array.


batch_get_value

batch_get_value(xs)

Returns the value of more than one tensor variable, as a list of Numpy arrays.


set_value

set_value(x, value)

Sets the value of a tensor variable, from a Numpy array.


batch_set_value

batch_set_value(tuples)

Sets the values of many tensor variables at once.

Arguments

  • tuples: a list of tuples (tensor, value). value should be a Numpy array.

print_tensor(x, message='')

Print the message and the tensor when evaluated and return the same tensor.


function

function(inputs, outputs, updates=[])

Instantiates a Keras function.

Arguments

  • inputs: list of placeholder/variable tensors.
  • outputs: list of output tensors.
  • updates: list of update tuples (old_tensor, new_tensor).

gradients

gradients(loss, variables)

Returns the gradients of variables (list of tensor variables) with regard to loss.


stop_gradient

stop_gradient(variables)

Returns variables but with zero gradient with respect to every other variables.


rnn

rnn(step_function, inputs, initial_states, go_backwards=False, mask=None, constants=None, unroll=False, input_length=None)

Iterates over the time dimension of a tensor.

Arguments

  • inputs: tensor of temporal data of shape (samples, time, ...) (at least 3D).
  • step_function:
  • Parameters:
    • input: tensor with shape (samples, ...) (no time dimension), representing input for the batch of samples at a certain time step.
    • states: list of tensors.
  • Returns:
    • output: tensor with shape (samples, output_dim) (no time dimension).
    • new_states: list of tensors, same length and shapes as 'states'. The first state in the list must be the output tensor at the previous timestep.
  • initial_states: tensor with shape (samples, output_dim) (no time dimension), containing the initial values for the states used in the step function.
  • go_backwards: boolean. If True, do the iteration over the time dimension in reverse order.
  • mask: binary tensor with shape (samples, time, 1), with a zero for every element that is masked.
  • constants: a list of constant values passed at each step.
  • unroll: with TensorFlow the RNN is always unrolled, but with Theano you can use this boolean flag to unroll the RNN.
  • input_length: not relevant in the TensorFlow implementation. Must be specified if using unrolling with Theano.

Returns

A tuple (last_output, outputs, new_states).

  • last_output: the latest output of the rnn, of shape (samples, ...)
  • outputs: tensor with shape (samples, time, ...) where each entry outputs[s, t] is the output of the step function at time t for sample s.
  • new_states: list of tensors, latest states returned by the step function, of shape (samples, ...).

switch

switch(condition, then_expression, else_expression)

Switches between two operations depending on a scalar value (int or bool). Note that both then_expression and else_expression should be symbolic tensors of the same shape.

Arguments

  • condition: scalar tensor.
  • then_expression: TensorFlow operation.
  • else_expression: TensorFlow operation.

in_train_phase

in_train_phase(x, alt)

Selects x in train phase, and alt otherwise. Note that alt should have the same shape as x.


in_test_phase

in_test_phase(x, alt)

Selects x in test phase, and alt otherwise. Note that alt should have the same shape as x.


relu

relu(x, alpha=0.0, max_value=None)

Rectified linear unit

Arguments

  • alpha: slope of negative section.
  • max_value: saturation threshold.

elu

elu(x, alpha=1.0)

Exponential linear unit.

Arguments

  • x: Tensor to compute the activation function for.
  • alpha: scalar

softmax

softmax(x)

Softmax of a tensor.


softplus

softplus(x)

Softplus of a tensor.


softsign

softsign(x)

Softsign of a tensor.


categorical_crossentropy

categorical_crossentropy(output, target, from_logits=False)

Categorical crossentropy between an output tensor and a target tensor, where the target is a tensor of the same shape as the output.


sparse_categorical_crossentropy

sparse_categorical_crossentropy(output, target, from_logits=False)

Categorical crossentropy between an output tensor and a target tensor, where the target is an integer tensor.


binary_crossentropy

binary_crossentropy(output, target, from_logits=False)

Binary crossentropy between an output tensor and a target tensor.


sigmoid

sigmoid(x)

Element-wise sigmoid.


hard_sigmoid

hard_sigmoid(x)

Segment-wise linear approximation of sigmoid. Faster than sigmoid.


tanh

tanh(x)

Element-wise tanh.


dropout

dropout(x, level, noise_shape=None, seed=None)

Sets entries in x to zero at random, while scaling the entire tensor.

Arguments

  • x: tensor
  • level: fraction of the entries in the tensor that will be set to 0.
  • noise_shape: shape for randomly generated keep/drop flags, must be broadcastable to the shape of x
  • seed: random seed to ensure determinism.

l2_normalize

l2_normalize(x, axis)

Normalizes a tensor wrt the L2 norm alongside the specified axis.


in_top_k

in_top_k(predictions, targets, k)

Returns whether the targets are in the top k predictions

Arguments

  • predictions: A tensor of shape batch_size x classess and type float32.
  • targets: A tensor of shape batch_size and type int32 or int64.
  • k: An int, number of top elements to consider.

Returns

A tensor of shape batch_size and type bool. output_i is True if targets_i is within top-k values of predictions_i


conv1d

conv1d(x, kernel, stride=1, border_mode='valid', image_shape=None, filter_shape=None)

1D convolution.

Arguments

  • kernel: kernel tensor.
  • strides: stride integer.
  • border_mode: string, "same" or "valid".

conv2d

conv2d(x, kernel, strides=(1, 1), border_mode='valid', dim_ordering='default', image_shape=None, filter_shape=None, filter_dilation=(1, 1))

2D convolution.

Arguments

  • kernel: kernel tensor.
  • strides: strides tuple.
  • border_mode: string, "same" or "valid".
  • dim_ordering: "tf" or "th". Whether to use Theano or TensorFlow dimension ordering for inputs/kernels/ouputs.

deconv2d

deconv2d(x, kernel, output_shape, strides=(1, 1), border_mode='valid', dim_ordering='default', image_shape=None, filter_shape=None)

2D deconvolution (i.e. transposed convolution).

Arguments

  • x: input tensor.
  • kernel: kernel tensor.
  • output_shape: 1D int tensor for the output shape.
  • strides: strides tuple.
  • border_mode: string, "same" or "valid".
  • dim_ordering: "tf" or "th". Whether to use Theano or TensorFlow dimension ordering for inputs/kernels/ouputs.

conv3d

conv3d(x, kernel, strides=(1, 1, 1), border_mode='valid', dim_ordering='default', volume_shape=None, filter_shape=None)

3D convolution.

Arguments

  • kernel: kernel tensor.
  • strides: strides tuple.
  • border_mode: string, "same" or "valid".
  • dim_ordering: "tf" or "th". Whether to use Theano or TensorFlow dimension ordering for inputs/kernels/ouputs.

pool2d

pool2d(x, pool_size, strides=(1, 1), border_mode='valid', dim_ordering='default', pool_mode='max')

2D Pooling.

Arguments

  • pool_size: tuple of 2 integers.
  • strides: tuple of 2 integers.
  • border_mode: one of "valid", "same".
  • dim_ordering: one of "th", "tf".
  • pool_mode: one of "max", "avg".

pool3d

pool3d(x, pool_size, strides=(1, 1, 1), border_mode='valid', dim_ordering='default', pool_mode='max')

3D Pooling.

Arguments

  • pool_size: tuple of 3 integers.
  • strides: tuple of 3 integers.
  • border_mode: one of "valid", "same".
  • dim_ordering: one of "th", "tf".
  • pool_mode: one of "max", "avg".

ctc_batch_cost

ctc_batch_cost(y_true, y_pred, input_length, label_length)

Runs CTC loss algorithm on each batch element.

Arguments

  • y_true: tensor (samples, max_string_length) containing the truth labels
  • y_pred: tensor (samples, time_steps, num_categories) containing the prediction, or output of the softmax
  • input_length: tensor (samples,1) containing the sequence length for each batch item in y_pred
  • label_length: tensor (samples,1) containing the sequence length for each batch item in y_true

Returns

Tensor with shape (samples,1) containing the CTC loss of each element


ctc_decode

ctc_decode(y_pred, input_length, greedy=True, beam_width=100, top_paths=1)

Decodes the output of a softmax using either greedy (also known as best path) or a constrained dictionary search.

Arguments

  • y_pred: tensor (samples, time_steps, num_categories) containing the prediction, or output of the softmax
  • input_length: tensor (samples,) containing the sequence length for each batch item in y_pred
  • greedy: perform much faster best-path search if true. This does not use a dictionary
  • beam_width: if greedy is false: a beam search decoder will be used with a beam of this width
  • top_paths: if greedy is false: how many of the most probable paths will be returned

Returns

  • Tuple:
  • List: if greedy is true, returns a list of one element that contains the decoded sequence. If false, returns the top_paths most probable decoded sequences. Important: blank labels are returned as -1 Tensor (top_paths,) that contains the log probability of each decoded sequence

map_fn

map_fn(fn, elems, name=None)

Map the function fn over the elements elems and return the outputs.

Arguments

  • fn: Callable that will be called upon each element in elems
  • elems: tensor
  • name: A string name for the map node in the graph

Returns

Tensor with first dimension equal to the elems and second depending on fn


foldl

foldl(fn, elems, initializer=None, name=None)

Reduce elems using fn to combine them from left to right.

Arguments

  • fn: Callable that will be called upon each element in elems and an accumulator, for instance lambda acc, x: acc + x
  • elems: tensor
  • initializer: The first value used (elems[0] in case of None)
  • name: A string name for the foldl node in the graph

Returns

Same type and shape as initializer


foldr

foldr(fn, elems, initializer=None, name=None)

Reduce elems using fn to combine them from right to left.

Arguments

  • fn: Callable that will be called upon each element in elems and an accumulator, for instance lambda acc, x: acc + x
  • elems: tensor
  • initializer: The first value used (elems[-1] in case of None)
  • name: A string name for the foldr node in the graph

Returns

Same type and shape as initializer


backend

backend()

Publicly accessible method for determining the current backend.