tf.keras.preprocessing.timeseries_dataset_from_array( data, targets, sequence_length, sequence_stride=1, sampling_rate=1, batch_size=128, shuffle=False, seed=None, start_index=None, end_index=None, )
Creates a dataset of sliding windows over a timeseries provided as array.
This function takes in a sequence of data-points gathered at equal intervals, along with time series parameters such as length of the sequences/windows, spacing between two sequence/windows, etc., to produce batches of timeseries inputs and targets.
targets[i]should be the target corresponding to the window that starts at index
i(see example 2 below). Pass None if you don't have target data (in this case the dataset will only yield the input data).
s, output samples would start at index
data[i + s],
data[i + 2 * s], etc.
data[i], data[i + r], ... data[i + sequence_length]are used for create a sample sequence.
start_indexwill not be used in the output sequences. This is useful to reserve part of the data for test or validation.
end_indexwill not be used in the output sequences. This is useful to reserve part of the data for test or validation.
A tf.data.Dataset instance. If
targets was passed, the dataset yields
(batch_of_sequences, batch_of_targets). If not, the dataset yields
[0, 1, ... 99].
sequence_length=10, sampling_rate=2, sequence_stride=3,
shuffle=False, the dataset will yield batches of sequences
composed of the following indices:
First sequence: [0 2 4 6 8 10 12 14 16 18] Second sequence: [3 5 7 9 11 13 15 17 19 21] Third sequence: [6 8 10 12 14 16 18 20 22 24] ... Last sequence: [78 80 82 84 86 88 90 92 94 96]
In this case the last 3 data points are discarded since no full sequence can be generated to include them (the next sequence would have started at index 81, and thus its last step would have gone over 99).
Example 2: Temporal regression.
Consider an array
data of scalar values, of shape
To generate a dataset that uses the past 10
timesteps to predict the next timestep, you would use:
input_data = data[:-10] targets = data[10:] dataset = tf.keras.preprocessing.timeseries_dataset_from_array( input_data, targets, sequence_length=10) for batch in dataset: inputs, targets = batch assert np.array_equal(inputs, data[:10]) # First sequence: steps [0-9] assert np.array_equal(targets, data) # Corresponding target: step 10 break
Example 3: Temporal regression for many-to-many architectures.
Consider two arrays of scalar values
both of shape
(100,). The resulting dataset should consist samples with
20 timestamps each. The samples should not overlap.
To generate a dataset that uses the current timestamp
to predict the corresponding target timestep, you would use:
X = np.arange(100) Y = X*2 sample_length = 20 input_dataset = tf.keras.preprocessing.timeseries_dataset_from_array( X, None, sequence_length=sample_length, sequence_stride=sample_length) target_dataset = tf.keras.preprocessing.timeseries_dataset_from_array( Y, None, sequence_length=sample_length, sequence_stride=sample_length) for batch in zip(input_dataset, target_dataset): inputs, targets = batch assert np.array_equal(inputs, X[:sample_length]) # second sample equals output timestamps 20-40 assert np.array_equal(targets, Y[sample_length:2*sample_length]) break
tf.keras.preprocessing.sequence.pad_sequences( sequences, maxlen=None, dtype="int32", padding="pre", truncating="pre", value=0.0 )
Pads sequences to the same length.
This function transforms a list (of length
of sequences (lists of integers)
into a 2D Numpy array of shape
num_timesteps is either the
maxlen argument if provided,
or the length of the longest sequence in the list.
Sequences that are shorter than
are padded with
value until they are
Sequences longer than
num_timesteps are truncated
so that they fit the desired length.
The position where padding or truncation happens is determined by
Pre-padding or removing values from the beginning of the sequence is the
>>> sequence = [, [2, 3], [4, 5, 6]] >>> tf.keras.preprocessing.sequence.pad_sequences(sequence) array([[0, 0, 1], [0, 2, 3], [4, 5, 6]], dtype=int32)
>>> tf.keras.preprocessing.sequence.pad_sequences(sequence, value=-1) array([[-1, -1, 1], [-1, 2, 3], [ 4, 5, 6]], dtype=int32)
>>> tf.keras.preprocessing.sequence.pad_sequences(sequence, padding='post') array([[1, 0, 0], [2, 3, 0], [4, 5, 6]], dtype=int32)
>>> tf.keras.preprocessing.sequence.pad_sequences(sequence, maxlen=2) array([[0, 1], [2, 3], [5, 6]], dtype=int32)
maxlen, either at the beginning or at the end of the sequences.
Numpy array with shape
padding, or in case of invalid shape for a