Time Series Forecasting: Kick-Start Your Project With My New Book
Time Series Forecasting: Kick-Start Your Project With My New Book
time series
forecasting.
There are many types of CNN models that can be used for each specific type of time series
forecasting problem.
In this tutorial, you will discover how to develop a suite of CNN models for a range of
standard time series forecasting problems.
The objective of this tutorial is to provide standalone examples of each model on each type
of time series problem as a template that you can copy and adapt for your specific time
series forecasting problem.
Kick-start your project with my new book Deep Learning for Time Series Forecasting,
including step-by-step tutorials and the Python source code files for all examples.
Let’s get started.
How to Develop Convolutional Neural Network Models for Time Series Forecasting
Tutorial Overview
In this tutorial, we will explore how to develop a suite of different types of CNN models for
time series forecasting.
The models are demonstrated on small contrived time series problems intended to give the
flavor of the type of time series problem being addressed. The chosen configuration of the
models is arbitrary and not optimized for each problem; that was not the goal.
Univariate time series are datasets comprised of a single series of observations with a
temporal ordering and a model is required to learn from the series of past observations to
predict the next value in the sequence.
1. Data Preparation
2. CNN Model
Data Preparation
Before a univariate series can be modeled, it must be prepared.
The CNN model will learn a function that maps a sequence of past observations as input to
an output observation. As such, the sequence of observations must be transformed into
multiple examples from which the model can learn.
We can divide the sequence into multiple input/output patterns called samples, where three
time steps are used as input and one time step is used as output for the one-step prediction
that is being learned.
1X, y
210, 20, 30 40
320, 30, 40 50
430, 40, 50 60
5...
3 X, y = list(), list()
4 for i in range(len(sequence)):
6 end_ix = i + n_steps
9 break
12 X.append(seq_x)
13 y.append(seq_y)
3
6 X, y = list(), list()
7 for i in range(len(sequence)):
9 end_ix = i + n_steps
12 break
15 X.append(seq_x)
16 y.append(seq_y)
18
22 n_steps = 3
24 X, y = split_sequence(raw_seq, n_steps)
26 for i in range(len(X)):
27 print(X[i], y[i])
Running the example splits the univariate series into six samples where each sample has
three input time steps and one output time step.
1[10 20 30] 40
2[20 30 40] 50
3[30 40 50] 60
4[40 50 60] 70
5[50 60 70] 80
6[60 70 80] 90
Now that we know how to prepare a univariate series for modeling, let’s look at developing
a CNN model that can learn the mapping of inputs to outputs.
Click to sign-up and also get a free PDF Ebook version of the course.
CNN Model
A one-dimensional CNN is a CNN model that has a convolutional hidden layer that operates
over a 1D sequence. This is followed by perhaps a second convolutional layer in some
cases, such as very long input sequences, and then a pooling layer whose job it is to distill
the output of the convolutional layer to the most salient elements.
The convolutional and pooling layers are followed by a dense fully connected layer that
interprets the features extracted by the convolutional part of the model. A flatten layer is
used between the convolutional layers and the dense layer to reduce the feature maps to a
single one-dimensional vector.
We can define a 1D CNN Model for univariate time series forecasting as follows.
1# define model
2model = Sequential()
4model.add(MaxPooling1D(pool_size=2))
5model.add(Flatten())
6model.add(Dense(50, activation='relu'))
7model.add(Dense(1))
8model.compile(optimizer='adam', loss='mse')
Key in the definition is the shape of the input; that is what the model expects as input for
each sample in terms of the number of time steps and the number of features.
We are working with a univariate series, so the number of features is one, for one variable.
The number of time steps as input is the number we chose when preparing our dataset as
an argument to the split_sequence() function.
The input shape for each sample is specified in the input_shape argument on the definition
of the first hidden layer.
We almost always have multiple samples, therefore, the model will expect the input
component of training data to have the dimensions or shape:
2n_features = 1
The CNN does not actually view the data as having time steps, instead, it is treated as a
sequence over which convolutional read operations can be performed, like a one-
dimensional image.
In this example, we define a convolutional layer with 64 filter maps and a kernel size of 2.
This is followed by a max pooling layer and a dense layer to interpret the input feature. An
output layer is specified that predicts a single numerical value.
The model is fit using the efficient Adam version of stochastic gradient descent and
optimized using the mean squared error, or ‘mse‘, loss function.
Once the model is defined, we can fit it on the training dataset.
1# fit model
We can predict the next value in the sequence by providing the input:
1[100]
The model expects the input shape to be three-dimensional with [samples, timesteps,
features], therefore, we must reshape the single input sample before making the prediction.
1# demonstrate prediction
We can tie all of this together and demonstrate how to develop a 1D CNN model for
univariate time series forecasting and make a single prediction.
8
11 X, y = list(), list()
12 for i in range(len(sequence)):
14 end_ix = i + n_steps
17 break
20 X.append(seq_x)
21 y.append(seq_y)
23
25 raw_seq = [10, 20, 30, 40, 50, 60, 70, 80, 90]
27 n_steps = 3
29 X, y = split_sequence(raw_seq, n_steps)
31 n_features = 1
33 # define model
34 model = Sequential()
36 model.add(MaxPooling1D(pool_size=2))
37 model.add(Flatten())
38 model.add(Dense(50, activation='relu'))
39 model.add(Dense(1))
40 model.compile(optimizer='adam', loss='mse')
41 # fit model
47 print(yhat)
Running the example prepares the data, fits the model, and makes a prediction.
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
We can see that the model predicts the next value in the sequence.
1[[101.67965]]
There are two main models that we may require with multivariate time series data; they are:
The input time series are parallel because each series has observations at the same time
steps.
We can demonstrate this with a simple example of two parallel input time series where the
output series is the simple addition of the input series.
2in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
3in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
4out_seq = array([in_seq1[i]+in_seq2[i] for i in range(len(in_seq1))])
We can reshape these three arrays of data as a single dataset where each row is a time
step and each column is a separate time series.
5 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
6 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
14 print(dataset)
Running the example prints the dataset with one row per time step and one column for each
of the two input and one output parallel time series.
1[[ 10 15 25]
2 [ 20 25 45]
3 [ 30 35 65]
4 [ 40 45 85]
5 [ 50 55 105]
6 [ 60 65 125]
7 [ 70 75 145]
8 [ 80 85 165]
9 [ 90 95 185]]
As with the univariate time series, we must structure these data into samples with input and
output samples.
A 1D CNN model needs sufficient context to learn a mapping from an input sequence to an
output value. CNNs can support parallel input time series as separate channels, like red,
green, and blue components of an image. Therefore, we need to split the data into samples
maintaining the order of observations across the two input sequences.
If we chose three input time steps, then the first sample would look as follows:
Input:
110, 15
220, 25
330, 35
Output:
165
That is, the first three time steps of each parallel series are provided as input to the model
and the model associates this with the value in the output series at the third time step, in
this case, 65.
We can see that, in transforming the time series into input/output samples to train the
model, that we will have to discard some values from the output time series where we do
not have values in the input time series at prior time steps. In turn, the choice of the size of
the number of input time steps will have an important effect on how much of the training
data is used.
4 X, y = list(), list()
5 for i in range(len(sequences)):
6 # find the end of this pattern
7 end_ix = i + n_steps
8 # check if we are beyond the dataset
9 if end_ix > len(sequences):
10 break
11 # gather input and output parts of the pattern
12 seq_x, seq_y = sequences[i:end_ix, :-1], sequences[end_ix-1, -1]
13 X.append(seq_x)
14 y.append(seq_y)
We can test this function on our dataset using three time steps for each input time series as
input.
4
7 X, y = list(), list()
8 for i in range(len(sequences)):
10 end_ix = i + n_steps
13 break
17 y.append(seq_y)
19
21 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
22 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
31 n_steps = 3
33 X, y = split_sequences(dataset, n_steps)
34 print(X.shape, y.shape)
36 for i in range(len(X)):
37 print(X[i], y[i])
This is the exact three-dimensional structure expected by a 1D CNN as input. The data is
ready to use without further reshaping.
We can then see that the input and output for each sample is printed, showing the three
time steps for each of the two input series and the associated output for each sample.
1 (7, 3, 2) (7,)
2
3 [[10 15]
4 [20 25]
5 [30 35]] 65
6 [[20 25]
7 [30 35]
8 [40 45]] 85
9 [[30 35]
10 [40 45]
12 [[40 45]
13 [50 55]
15 [[50 55]
16 [60 65]
18 [[60 65]
19 [70 75]
21 [[70 75]
22 [80 85]
We are now ready to fit a 1D CNN model on this data, specifying the expected number of
time steps and features to expect for each input sample, in this case three and two
respectively.
1# define model
2model = Sequential()
4model.add(MaxPooling1D(pool_size=2))
5model.add(Flatten())
6model.add(Dense(50, activation='relu'))
7model.add(Dense(1))
8model.compile(optimizer='adam', loss='mse')
When making a prediction, the model expects three time steps for two input time series.
We can predict the next value in the output series providing the input values of:
180, 85
290, 95
3100, 105
The shape of the one sample with three time steps and two variables must be [1, 3, 2].
We would expect the next value in the sequence to be 100 + 105 or 205.
1# demonstrate prediction
9
12 X, y = list(), list()
13 for i in range(len(sequences)):
15 end_ix = i + n_steps
21 X.append(seq_x)
22 y.append(seq_y)
24
26 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
27 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
36 n_steps = 3
38 X, y = split_sequences(dataset, n_steps)
40 n_features = X.shape[2]
41 # define model
42 model = Sequential()
44 model.add(MaxPooling1D(pool_size=2))
45 model.add(Flatten())
46 model.add(Dense(50, activation='relu'))
47 model.add(Dense(1))
48 model.compile(optimizer='adam', loss='mse')
49 # fit model
55 print(yhat)
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
Running the example prepares the data, fits the model, and makes a prediction.
1[[206.0161]]
Each input series can be handled by a separate CNN and the output of each of these
submodels can be combined before a prediction is made for the output sequence.
We can refer to this as a multi-headed CNN model. It may offer more flexibility or better
performance depending on the specifics of the problem that is being modeled. For example,
it allows you to configure each sub-model differently for each input series, such as the
number of filter maps and the kernel size.
This type of model can be defined in Keras using the Keras functional API.
First, we can define the first input model as a 1D CNN with an input layer that expects
vectors with n_steps and 1 feature.
1# first input model
4cnn1 = MaxPooling1D(pool_size=2)(cnn1)
5cnn1 = Flatten()(cnn1)
4cnn2 = MaxPooling1D(pool_size=2)(cnn2)
5cnn2 = Flatten()(cnn2)
Now that both input submodels have been defined, we can merge the output from each
model into one long vector which can be interpreted before making a prediction for the
output sequence.
4output = Dense(1)(dense)
The image below provides a schematic for how this model looks, including the shape of the
inputs and outputs of each layer.
In order to achieve this, we can split the 3D input data into two separate arrays of input
data; that is from one array with the shape [7, 3, 2] to two 3D arrays with [7, 3, 1]
2n_features = 1
1# fit model
Similarly, we must prepare the data for a single sample as two separate two-dimensional
arrays when making a single one-step prediction.
We can tie all of this together; the complete example is listed below.
11
14 X, y = list(), list()
15 for i in range(len(sequences)):
17 end_ix = i + n_steps
20 break
23 X.append(seq_x)
24 y.append(seq_y)
26
28 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
29 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
38 n_steps = 3
40 X, y = split_sequences(dataset, n_steps)
42 n_features = 1
49 cnn1 = MaxPooling1D(pool_size=2)(cnn1)
50 cnn1 = Flatten()(cnn1)
54 cnn2 = MaxPooling1D(pool_size=2)(cnn2)
55 cnn2 = Flatten()(cnn2)
59 output = Dense(1)(dense)
61 model.compile(optimizer='adam', loss='mse')
62 # fit model
64 # demonstrate prediction
69 print(yhat)
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
Running the example prepares the data, fits the model, and makes a prediction.
1[[205.871]]
1[[ 10 15 25]
2 [ 20 25 45]
3 [ 30 35 65]
4 [ 40 45 85]
5 [ 50 55 105]
6 [ 60 65 125]
7 [ 70 75 145]
8 [ 80 85 165]
9 [ 90 95 185]]
We may want to predict the value for each of the three time series for the next time step.
Again, the data must be split into input/output samples in order to train a model.
Input:
110, 15, 25
220, 25, 45
330, 35, 65
Output:
140, 45, 85
The split_sequences() function below will split multiple parallel time series with rows for time
steps and one series per column into the required input/output shape.
1 # split a multivariate sequence into samples
3 X, y = list(), list()
4 for i in range(len(sequences)):
6 end_ix = i + n_steps
9 break
12 X.append(seq_x)
13 y.append(seq_y)
We can demonstrate this on the contrived problem; the complete example is listed below.
4
7 X, y = list(), list()
8 for i in range(len(sequences)):
10 end_ix = i + n_steps
13 break
16 X.append(seq_x)
17 y.append(seq_y)
19
21 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
22 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
31 n_steps = 3
33 X, y = split_sequences(dataset, n_steps)
34 print(X.shape, y.shape)
36 for i in range(len(X)):
37 print(X[i], y[i])
Running the example first prints the shape of the prepared X and y components.
The shape of X is three-dimensional, including the number of samples (6), the number of
time steps chosen per sample (3), and the number of parallel time series or features (3).
The shape of y is two-dimensional as we might expect for the number of samples (6) and
the number of time variables per sample to be predicted (3).
The data is ready to use in a 1D CNN model that expects three-dimensional input and two-
dimensional output shapes for the X and y components of each sample.
Then, each of the samples is printed showing the input and output components of each
sample.
1 (6, 3, 3) (6, 3)
2
3 [[10 15 25]
4 [20 25 45]
6 [[20 25 45]
7 [30 35 65]
10 [ 40 45 85]
12 [[ 40 45 85]
13 [ 50 55 105]
15 [[ 50 55 105]
16 [ 60 65 125]
18 [[ 60 65 125]
19 [ 70 75 145]
In this model, the number of time steps and parallel series (features) are specified for the
input layer via the input_shape argument.
The number of parallel series is also used in the specification of the number of values to
predict by the model in the output layer; again, this is three.
1# define model
2model = Sequential()
4model.add(MaxPooling1D(pool_size=2))
5model.add(Flatten())
6model.add(Dense(50, activation='relu'))
7model.add(Dense(n_features))
8model.compile(optimizer='adam', loss='mse')
We can predict the next value in each of the three parallel series by providing an input of
three time steps for each series.
The shape of the input for making a single prediction must be 1 sample, 3 time steps, and 3
features, or [1, 3, 3].
1# demonstrate prediction
We can tie all of this together and demonstrate a 1D CNN for multivariate output time series
forecasting below.
9
12 X, y = list(), list()
13 for i in range(len(sequences)):
15 end_ix = i + n_steps
18 break
21 X.append(seq_x)
22 y.append(seq_y)
24
25 # define input sequence
26 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
27 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
36 n_steps = 3
38 X, y = split_sequences(dataset, n_steps)
40 n_features = X.shape[2]
41 # define model
42 model = Sequential()
44 model.add(MaxPooling1D(pool_size=2))
45 model.add(Flatten())
46 model.add(Dense(50, activation='relu'))
47 model.add(Dense(n_features))
48 model.compile(optimizer='adam', loss='mse')
49 # fit model
51 # demonstrate prediction
55 print(yhat)
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
Running the example prepares the data, fits the model and makes a prediction.
As with multiple input series, there is another more elaborate way to model the problem.
We can refer to this as a multi-output CNN model. It may offer more flexibility or better
performance depending on the specifics of the problem that is being modeled.
This type of model can be defined in Keras using the Keras functional API.
First, we can define the first input model as a 1D CNN model.
1# define model
4cnn = MaxPooling1D(pool_size=2)(cnn)
5cnn = Flatten()(cnn)
We can then define one output layer for each of the three series that we wish to forecast,
where each output submodel will forecast a single time step.
1# define output 1
2output1 = Dense(1)(cnn)
3# define output 2
4output2 = Dense(1)(cnn)
5# define output 3
6output3 = Dense(1)(cnn)
We can then tie the input and output layers together into a single model.
1# tie together
3model.compile(optimizer='adam', loss='mse')
To make the model architecture clear, the schematic below clearly shows the three
separate output layers of the model and the input and output shapes of each layer.
When training the model, it will require three separate output arrays per sample. We can
achieve this by converting the output training data that has the shape [7, 3] to three arrays
with the shape [7, 1].
1# separate output
1# fit model
10
13 X, y = list(), list()
14 for i in range(len(sequences)):
16 end_ix = i + n_steps
19 break
22 X.append(seq_x)
23 y.append(seq_y)
25
27 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
28 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
37 n_steps = 3
39 X, y = split_sequences(dataset, n_steps)
41 n_features = X.shape[2]
42 # separate output
46 # define model
49 cnn = MaxPooling1D(pool_size=2)(cnn)
50 cnn = Flatten()(cnn)
52 # define output 1
53 output1 = Dense(1)(cnn)
54 # define output 2
55 output2 = Dense(1)(cnn)
56 # define output 3
57 output3 = Dense(1)(cnn)
58 # tie together
60 model.compile(optimizer='adam', loss='mse')
61 # fit model
63 # demonstrate prediction
67 print(yhat)
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
Running the example prepares the data, fits the model, and makes a prediction.
1[array([[100.96118]], dtype=float32),
2 array([[105.502686]], dtype=float32),
3 array([[205.98045]], dtype=float32)]
Nevertheless, there are subtle and important differences in the way the training data is
prepared. In this section, we will demonstrate the case of developing a multi-step forecast
model using a vector model.
Before we look at the specifics of the model, let’s first look at the preparation of data for
multi-step forecasting.
Data Preparation
As with one-step forecasting, a time series used for multi-step time series forecasting must
be split into samples with input and output components.
Both the input and output components will be comprised of multiple time steps and may or
may not have the same number of steps.
We could use the last three time steps as input and forecast the next two time steps.
Input:
1[40, 50]
3 X, y = list(), list()
4 for i in range(len(sequence)):
6 end_ix = i + n_steps_in
10 break
13 X.append(seq_x)
14 y.append(seq_y)
3
6 X, y = list(), list()
7 for i in range(len(sequence)):
9 end_ix = i + n_steps_in
10 out_end_ix = end_ix + n_steps_out
13 break
16 X.append(seq_x)
17 y.append(seq_y)
19
21 raw_seq = [10, 20, 30, 40, 50, 60, 70, 80, 90]
23 n_steps_in, n_steps_out = 3, 2
27 for i in range(len(X)):
28 print(X[i], y[i])
Running the example splits the univariate series into input and output time steps and prints
the input and output components of each.
Now that we know how to prepare data for multi-step forecasting, let’s look at a 1D CNN
model that can learn this mapping.
As with the 1D CNN models for univariate data in a prior section, the prepared samples
must first be reshaped. The CNN expects data to have a three-dimensional structure of
[samples, timesteps, features], and in this case, we only have one feature so the reshape is
straightforward.
1# reshape from [samples, timesteps] into [samples, timesteps, features]
2n_features = 1
2model = Sequential()
4model.add(MaxPooling1D(pool_size=2))
5model.add(Flatten())
6model.add(Dense(50, activation='relu'))
7model.add(Dense(n_steps_out))
8model.compile(optimizer='adam', loss='mse')
The model can make a prediction for a single sample. We can predict the next two steps
beyond the end of the dataset by providing the input:
1[100, 110]
As expected by the model, the shape of the single sample of input data when making the
prediction must be [1, 3, 1] for the 1 sample, 3 time steps of the input, and the single
feature.
1# demonstrate prediction
Tying all of this together, the 1D CNN for multi-step forecasting with a univariate time series
is listed below.
8
11 X, y = list(), list()
12 for i in range(len(sequence)):
14 end_ix = i + n_steps_in
18 break
21 X.append(seq_x)
22 y.append(seq_y)
24
26 raw_seq = [10, 20, 30, 40, 50, 60, 70, 80, 90]
28 n_steps_in, n_steps_out = 3, 2
32 n_features = 1
34 # define model
35 model = Sequential()
37 model.add(MaxPooling1D(pool_size=2))
38 model.add(Flatten())
39 model.add(Dense(50, activation='relu'))
40 model.add(Dense(n_steps_out))
41 model.compile(optimizer='adam', loss='mse')
42 # fit model
44 # demonstrate prediction
48 print(yhat)
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
Running the example forecasts and prints the next two time steps in the sequence.
1[[102.86651 115.08979]]
It is possible to mix and match the different types of 1D CNN models presented so far for
the different problems. This too applies to time series forecasting problems that involve
multivariate and multi-step forecasting, but it may be a little more challenging.
In this section, we will explore short examples of data preparation and modeling for
multivariate multi-step time series forecasting as a template to ease this challenge,
specifically:
For example, consider our multivariate time series from a prior section:
1[[ 10 15 25]
2 [ 20 25 45]
3 [ 30 35 65]
4 [ 40 45 85]
5 [ 50 55 105]
6 [ 60 65 125]
7 [ 70 75 145]
8 [ 80 85 165]
9 [ 90 95 185]]
We may use three prior time steps of each of the two input time series to predict two time
steps of the output time series.
Input:
110, 15
220, 25
330, 35
Output:
165
285
3 X, y = list(), list()
4 for i in range(len(sequences)):
6 end_ix = i + n_steps_in
10 break
13 X.append(seq_x)
14 y.append(seq_y)
We can demonstrate this on our contrived dataset. The complete example is listed below.
4
7 X, y = list(), list()
8 for i in range(len(sequences)):
10 end_ix = i + n_steps_in
14 break
17 X.append(seq_x)
18 y.append(seq_y)
20
22 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
23 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
32 n_steps_in, n_steps_out = 3, 2
35 print(X.shape, y.shape)
37 for i in range(len(X)):
38 print(X[i], y[i])
Running the example first prints the shape of the prepared training data.
We can see that the shape of the input portion of the samples is three-dimensional,
comprised of six samples, with three time steps and two variables for the two input time
series.
The output portion of the samples is two-dimensional for the six samples and the two time
steps for each sample to be predicted.
The prepared samples are then printed to confirm that the data was prepared as we
specified.
1 (6, 3, 2) (6, 2)
2
3 [[10 15]
4 [20 25]
6 [[20 25]
7 [30 35]
9 [[30 35]
10 [40 45]
12 [[40 45]
13 [50 55]
15 [[50 55]
16 [60 65]
18 [[60 65]
19 [70 75]
In this case, we will demonstrate a vector output model. The complete example is listed
below.
9
10 # split a multivariate sequence into samples
12 X, y = list(), list()
13 for i in range(len(sequences)):
15 end_ix = i + n_steps_in
19 break
22 X.append(seq_x)
23 y.append(seq_y)
25
27 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
28 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
37 n_steps_in, n_steps_out = 3, 2
41 n_features = X.shape[2]
42 # define model
43 model = Sequential()
45 model.add(MaxPooling1D(pool_size=2))
46 model.add(Flatten())
47 model.add(Dense(50, activation='relu'))
48 model.add(Dense(n_steps_out))
49 model.compile(optimizer='adam', loss='mse')
50 # fit model
52 # demonstrate prediction
56 print(yhat)
Running the example fits the model and predicts the next two time steps of the output
sequence beyond the dataset.
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
It is a challenging framing of the problem with very little data, and the arbitrarily configured
version of the model gets close.
1[[185.57011 207.77893]]
For example, consider our multivariate time series from a prior section:
1[[ 10 15 25]
2 [ 20 25 45]
3 [ 30 35 65]
4 [ 40 45 85]
5 [ 50 55 105]
6 [ 60 65 125]
7 [ 70 75 145]
8 [ 80 85 165]
9 [ 90 95 185]]
We may use the last three time steps from each of the three time series as input to the
model, and predict the next time steps of each of the three time series as output.
Input:
110, 15, 25
220, 25, 45
330, 35, 65
Output:
140, 45, 85
3 X, y = list(), list()
4 for i in range(len(sequences)):
6 end_ix = i + n_steps_in
10 break
13 X.append(seq_x)
14 y.append(seq_y)
15 return array(X), array(y)
9
12 X, y = list(), list()
13 for i in range(len(sequences)):
15 end_ix = i + n_steps_in
19 break
22 X.append(seq_x)
23 y.append(seq_y)
25
27 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
28 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
37 n_steps_in, n_steps_out = 3, 2
40 print(X.shape, y.shape)
42 for i in range(len(X)):
43 print(X[i], y[i])
Running the example first prints the shape of the prepared training dataset.
We can see that both the input (X) and output (Y) elements of the dataset are three
dimensional for the number of samples, time steps, and variables or parallel time series
respectively.
The input and output elements of each series are then printed side by side so that we can
confirm that the data was prepared as we expected.
1 (5, 3, 3) (5, 2, 3)
2
3 [[10 15 25]
4 [20 25 45]
6 [ 50 55 105]]
7 [[20 25 45]
8 [30 35 65]
10 [ 60 65 125]]
11 [[ 30 35 65]
12 [ 40 45 85]
13 [ 50 55 105]] [[ 60 65 125]
14 [ 70 75 145]]
15 [[ 40 45 85]
16 [ 50 55 105]
18 [ 80 85 165]]
19 [[ 50 55 105]
20 [ 60 65 125]
22 [ 90 95 185]]
We will use a vector-output model in this case. As such, we must flatten the three-
dimensional structure of the output portion of each sample in order to train the model. This
means, instead of predicting two steps for each series, the model is trained on and
expected to predict a vector of six numbers directly.
1# flatten output
3y = y.reshape((y.shape[0], n_output))
9
12 X, y = list(), list()
13 for i in range(len(sequences)):
15 end_ix = i + n_steps_in
19 break
22 X.append(seq_x)
23 y.append(seq_y)
25
27 in_seq1 = array([10, 20, 30, 40, 50, 60, 70, 80, 90])
28 in_seq2 = array([15, 25, 35, 45, 55, 65, 75, 85, 95])
37 n_steps_in, n_steps_out = 3, 2
40 # flatten output
42 y = y.reshape((y.shape[0], n_output))
44 n_features = X.shape[2]
45 # define model
46 model = Sequential()
48 model.add(MaxPooling1D(pool_size=2))
49 model.add(Flatten())
50 model.add(Dense(50, activation='relu'))
51 model.add(Dense(n_output))
52 model.compile(optimizer='adam', loss='mse')
53 # fit model
55 # demonstrate prediction
56 x_input = array([[60, 65, 125], [70, 75, 145], [80, 85, 165]])
59 print(yhat)
Running the example fits the model and predicts the values for each of the three time steps
for the next two time steps beyond the end of the dataset.
We would expect the values for these series and time steps to be as follows:
Note: Your results may vary given the stochastic nature of the algorithm or evaluation
procedure, or differences in numerical precision. Consider running the example a few times
and compare the average outcome.
We can see that the model forecast gets reasonably close to the expected values.
Summary
In this tutorial, you discovered how to develop a suite of CNN models for a range of
standard time series forecasting problems.