Skip to content

================ by Jawad Haider

RNN Exercises - Solutions

For these exercises we’re using data from the Federal Reserve Economic Database (FRED) concerning Electricity and Gas Utilities Production from January 1992 to January 2019 (325 records).

Data source: https://fred.stlouisfed.org/series/IPG2211A2N

In the exercises below you’ll be asked to do the following: * Perform standard imports, load & plot the dataset (code provided) * Prepare data for an LSTM model * Define the LSTM model, loss and optimization functions * Train the model * Evaluate the model on test data * OPTIONAL: Plot the results

IMPORTANT NOTE! Make sure you don’t run the cells directly above the example output shown,
otherwise you will end up writing over the example output!

Perform standard imports, load and plot the dataset

Run the cells below to load the libraries needed for this exercise and the Energy Production dataset, and to plot the data.

# RUN THIS CELL
import torch
import torch.nn as nn
from sklearn.preprocessing import MinMaxScaler

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

from pandas.plotting import register_matplotlib_converters
register_matplotlib_converters()

df = pd.read_csv('../Data/TimeSeriesData/Energy_Production.csv',index_col=0,parse_dates=True)
df.dropna(inplace=True)
print(len(df))
df.head()

325
IPG2211A2N
DATE
1992-01-01 85.5560
1992-02-01 80.4178
1992-03-01 74.7390
1992-04-01 69.8367
1992-05-01 67.3781 
# RUN THIS CELL
plt.figure(figsize=(12,4))
plt.title('Industrial Production Index for Electricity and Gas Utilities')
plt.ylabel('Index 2012=100, Not Seasonally Adjusted')
plt.grid(True)
plt.autoscale(axis='x',tight=True)
plt.plot(df['IPG2211A2N'])
plt.show()

Prepare the data

For the first set of exercises we’ll * divide the data into train and test sets * normalize the training set * prepare windowed seq/label tuples for an LSTM model

1. Divide the data into train and test sets

Working with a window_size of 12, divide the dataset into a sequence of 313 training records (including the window), and a test set of 12 records.

# CODE HERE
y = df['IPG2211A2N'].values.astype(float)

test_size = 
window_size = 

train_set = 
test_set = 

# Run the code below to check your results:
print(f'Train: {len(train_set)}')
print(f'Test:  {len(test_set)}')

# DON'T WRITE HERE
y = df['IPG2211A2N'].values.astype(float)

test_size = 12
window_size = 12

train_set = y[:-test_size]
test_set = y[-test_size:]

print(f'Train: {len(train_set)}')
print(f'Test:  {len(test_set)}')

Train: 313
Test:  12

2. Normalize the training set

Feature scale the training set to fit within the range [-1,1].

# CODE HERE
scaler = MinMaxScaler(feature_range=(-1, 1))


train_norm = 

# Run the code below to check your results:
print(f'First item, original: {train_set[0]}')
print(f'First item, scaled:  {train_norm[0]}')

# DON'T WRITE HERE
scaler = MinMaxScaler(feature_range=(-1, 1))

train_norm = scaler.fit_transform(train_set.reshape(-1, 1))

print(f'First item, original: {train_set[0]}')
print(f'First item, scaled: {train_norm[0]}')

First item, original: 85.556
First item, scaled: [-0.4091274]

3. Prepare data for LSTM

Prepare the list of windowed sequence/label tuples to be fed into an LSTM model.

# RUN THIS CELL
train_norm = torch.FloatTensor(train_norm).view(-1)

def input_data(seq,ws):
    out = []
    L = len(seq)
    for i in range(L-ws):
        window = seq[i:i+ws]
        label = seq[i+ws:i+ws+1]
        out.append((window,label))
    return out

# CODE HERE

train_data = 

# Run the code below to check your results:
print(f'Train_data: {len(train_data)}')  # should equal 301

# DON'T WRITE HERE
train_data = input_data(train_norm,window_size)

print(f'Train_data: {len(train_data)}')

Train_data: 301

4. Define the model

Design a model that has a (1,64) LSTM layer and a (64,1) fully-connected linear layer. Be sure to initialize \(h_0\) and \(c_0\), and return only the last predicted value.

# CODE HERE
class LSTMnetwork(nn.Module):

# Run the code below to check your results:
torch.manual_seed(101)
model = LSTMnetwork()
model

# DON'T WRITE HERE
class LSTMnetwork(nn.Module):
    def __init__(self,input_size=1,hidden_size=64,output_size=1):
        super().__init__()
        self.hidden_size = hidden_size

        # Add an LSTM layer:
        self.lstm = nn.LSTM(input_size,hidden_size)

        # Add a fully-connected layer:
        self.linear = nn.Linear(hidden_size,output_size)

        # Initialize h0 and c0:
        self.hidden = (torch.zeros(1,1,self.hidden_size),
                       torch.zeros(1,1,self.hidden_size))

    def forward(self,seq):
        lstm_out, self.hidden = self.lstm(
            seq.view(len(seq),1,-1), self.hidden)
        pred = self.linear(lstm_out.view(len(seq),-1))
        return pred[-1]

torch.manual_seed(101)
model = LSTMnetwork()
model

LSTMnetwork(
  (lstm): LSTM(1, 64)
  (linear): Linear(in_features=64, out_features=1, bias=True)
)

5. Define loss and optimization functions

Define a loss function called “criterion” and an optimizer called “optimizer”.
You can use any functions you want, although we used MSELoss and Adam (learning rate of 0.001) respectively.

# CODE HERE

# DON'T WRITE HERE
criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

6. Train the model

Don’t worry about tracking loss values, displaying results, or validating the test set. Just train the model through 50 epochs. We’ll evaluate the trained model in the next step.
OPTIONAL: print something after each epoch to indicate training progress.

# CODE HERE

# DON'T WRITE HERE
epochs = 50

for i in range(epochs):
    for seq, y_train in train_data:

        # reset the parameters and hidden states
        optimizer.zero_grad()
        model.hidden = (torch.zeros(1,1,model.hidden_size),
                        torch.zeros(1,1,model.hidden_size))

        # apply the model
        y_pred = model(seq)

        # update parameters
        loss = criterion(y_pred, y_train)
        loss.backward()
        optimizer.step()

    # OPTIONAL print statement
    print(f'{i+1} of {epochs} epochs completed')

1 of 50 epochs completed
2 of 50 epochs completed
3 of 50 epochs completed
4 of 50 epochs completed
5 of 50 epochs completed
6 of 50 epochs completed
7 of 50 epochs completed
8 of 50 epochs completed
9 of 50 epochs completed
10 of 50 epochs completed
11 of 50 epochs completed
12 of 50 epochs completed
13 of 50 epochs completed
14 of 50 epochs completed
15 of 50 epochs completed
16 of 50 epochs completed
17 of 50 epochs completed
18 of 50 epochs completed
19 of 50 epochs completed
20 of 50 epochs completed
21 of 50 epochs completed
22 of 50 epochs completed
23 of 50 epochs completed
24 of 50 epochs completed
25 of 50 epochs completed
26 of 50 epochs completed
27 of 50 epochs completed
28 of 50 epochs completed
29 of 50 epochs completed
30 of 50 epochs completed
31 of 50 epochs completed
32 of 50 epochs completed
33 of 50 epochs completed
34 of 50 epochs completed
35 of 50 epochs completed
36 of 50 epochs completed
37 of 50 epochs completed
38 of 50 epochs completed
39 of 50 epochs completed
40 of 50 epochs completed
41 of 50 epochs completed
42 of 50 epochs completed
43 of 50 epochs completed
44 of 50 epochs completed
45 of 50 epochs completed
46 of 50 epochs completed
47 of 50 epochs completed
48 of 50 epochs completed
49 of 50 epochs completed
50 of 50 epochs completed

9. Evaluate the model using the test set

Be sure to re-initialize the hidden parameters \(h_0\) and \(c_0\) before running the model.

# CODE HERE
future = 
preds = 

model.eval()

for i in range(future):

# Run the code below to check your results:
preds[window_size:]

# DON'T WRITE HERE
future = 12
preds = train_norm[-window_size:].tolist()

model.eval()

for i in range(future):
    seq = torch.FloatTensor(preds[-window_size:])
    with torch.no_grad():
        model.hidden = (torch.zeros(1,1,model.hidden_size),
                        torch.zeros(1,1,model.hidden_size))
        preds.append(model(seq).item())

preds[window_size:]

[0.25382155179977417,
 -0.0027704648673534393,
 -0.343053936958313,
 -0.21152164041996002,
 0.23945370316505432,
 0.4895053505897522,
 0.24688751995563507,
 -0.08669154345989227,
 -0.25793153047561646,
 0.022461334243416786,
 0.5438402891159058,
 0.6108715534210205]

10. Inverse transform the predicted values

Rescale the predicted values up to the original test set range.

# CODE HERE
true_predictions = 

# Run the code below to check your results:
true_predictions

# DON'T WRITE HERE
true_predictions = scaler.inverse_transform(np.array(preds[window_size:]).reshape(-1, 1))
true_predictions

array([[105.95129313],
       [ 98.05736803],
       [ 87.58871716],
       [ 91.63524249],
       [105.50927345],
       [113.20198736],
       [105.73797111],
       [ 95.47557801],
       [ 90.20746543],
       [ 98.83361172],
       [114.87357457],
       [116.93575791]])

BONUS EXERCISE: Plot the result

Plot the true_predictions values together with the original data. Remember to create a range of datetime values for the predicted data.

# CODE HERE

# CODE HERE TO DISPLAY THE END OF THE GRAPH

# DON'T WRITE HERE
x = np.arange('2018-02-01', '2019-02-01', dtype='datetime64[M]').astype('datetime64[D]')

plt.figure(figsize=(12,4))
plt.title('Industrial Production Index for Electricity and Gas Utilities')
plt.ylabel('Index 2012=100, Not Seasonally Adjusted')
plt.grid(True)
plt.autoscale(axis='x',tight=True)
plt.plot(df['IPG2211A2N'])
plt.plot(x,true_predictions)
plt.show()


# DON'T WRITE HERE
fig = plt.figure(figsize=(12,4))
plt.title('Industrial Production Index for Electricity and Gas Utilities')
plt.ylabel('Index 2012=100, Not Seasonally Adjusted')
plt.grid(True)
plt.autoscale(axis='x',tight=True)
fig.autofmt_xdate()
plt.plot(df['IPG2211A2N']['2017-01-01':])
plt.plot(x,true_predictions)
plt.show()

Great job!

Logo

Copyright Qalmaqihir
For more information, visit us at www.github.com/qalmaqihir/