================ by Jawad Haider

RNN Exercises
Perform standard imports, load and plot the dataset
Prepare the data
1. Divide the data into train and test sets
2. Normalize the training set
3. Prepare data for LSTM
4. Define the model
5. Define loss and optimization functions
- 6. Train the model
- 9. Evaluate the model using the test set
10. Inverse transform the predicted values
BONUS EXERCISE: Plot the result
Great job!

RNN Exercises¶

For these exercises we’re using data from the Federal Reserve Economic Database (FRED) concerning Electricity and Gas Utilities Production from January 1992 to January 2019 (325 records).

Data source: https://fred.stlouisfed.org/series/IPG2211A2N

In the exercises below you’ll be asked to do the following: * Perform standard imports, load & plot the dataset (code provided) * Prepare data for an LSTM model * Define the LSTM model, loss and optimization functions * Train the model * Evaluate the model on test data * OPTIONAL: Plot the results

IMPORTANT NOTE! Make sure you don’t run the cells directly above the example output shown,
otherwise you will end up writing over the example output!

Perform standard imports, load and plot the dataset¶

Run the cells below to load the libraries needed for this exercise and the Energy Production dataset, and to plot the data.

# RUN THIS CELL
import torch
import torch.nn as nn
from sklearn.preprocessing import MinMaxScaler

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

from pandas.plotting import register_matplotlib_converters
register_matplotlib_converters()

df = pd.read_csv('../Data/TimeSeriesData/Energy_Production.csv',index_col=0,parse_dates=True)
df.dropna(inplace=True)
print(len(df))
df.head()

	IPG2211A2N
DATE
1992-01-01	85.5560
1992-02-01	80.4178
1992-03-01	74.7390
1992-04-01	69.8367
1992-05-01	67.3781

# RUN THIS CELL
plt.figure(figsize=(12,4))
plt.title('Industrial Production Index for Electricity and Gas Utilities')
plt.ylabel('Index 2012=100, Not Seasonally Adjusted')
plt.grid(True)
plt.autoscale(axis='x',tight=True)
plt.plot(df['IPG2211A2N'])
plt.show()

Prepare the data¶

For the first set of exercises we’ll * divide the data into train and test sets * normalize the training set * prepare windowed seq/label tuples for an LSTM model

1. Divide the data into train and test sets¶

Working with a window_size of 12, divide the dataset into a sequence of 313 training records (including the window), and a test set of 12 records.

# CODE HERE
y = df['IPG2211A2N'].values.astype(float)

test_size = 
window_size = 

train_set = 
test_set =

# Run the code below to check your results:
print(f'Train: {len(train_set)}')
print(f'Test:  {len(test_set)}')

# DON'T WRITE HERE

Train: 313
Test:  12

2. Normalize the training set¶

Feature scale the training set to fit within the range [-1,1].

# CODE HERE
scaler = MinMaxScaler(feature_range=(-1, 1))


train_norm =

# Run the code below to check your results:
print(f'First item, original: {train_set[0]}')
print(f'First item, scaled:  {train_norm[0]}')

# DON'T WRITE HERE

First item, original: 85.556
First item, scaled: [-0.4091274]

3. Prepare data for LSTM¶

Prepare the list of windowed sequence/label tuples to be fed into an LSTM model.

# RUN THIS CELL
train_norm = torch.FloatTensor(train_norm).view(-1)

def input_data(seq,ws):
    out = []
    L = len(seq)
    for i in range(L-ws):
        window = seq[i:i+ws]
        label = seq[i+ws:i+ws+1]
        out.append((window,label))
    return out

# CODE HERE

train_data =

# Run the code below to check your results:
print(f'Train_data: {len(train_data)}')  # should equal 301

# DON'T WRITE HERE

Train_data: 301

4. Define the model¶

Design a model that has a (1,64) LSTM layer and a (64,1) fully-connected linear layer. Be sure to initialize \(h_0\) and \(c_0\), and return only the last predicted value.

# CODE HERE
class LSTMnetwork(nn.Module):

# Run the code below to check your results:
torch.manual_seed(101)
model = LSTMnetwork()
model

# DON'T WRITE HERE

LSTMnetwork(
  (lstm): LSTM(1, 64)
  (linear): Linear(in_features=64, out_features=1, bias=True)
)

5. Define loss and optimization functions¶

Define a loss function called “criterion” and an optimizer called “optimizer”.
You can use any functions you want, although we used MSELoss and Adam (learning rate of 0.001) respectively.

# CODE HERE

# DON'T WRITE HERE

6. Train the model¶

Don’t worry about tracking loss values, displaying results, or validating the test set. Just train the model through 50 epochs. We’ll evaluate the trained model in the next step.
OPTIONAL: print something after each epoch to indicate training progress.

# CODE HERE

# DON'T WRITE HERE

1 of 50 epochs completed
2 of 50 epochs completed
3 of 50 epochs completed
4 of 50 epochs completed
5 of 50 epochs completed
6 of 50 epochs completed
7 of 50 epochs completed
8 of 50 epochs completed
9 of 50 epochs completed
10 of 50 epochs completed
11 of 50 epochs completed
12 of 50 epochs completed
13 of 50 epochs completed
14 of 50 epochs completed
15 of 50 epochs completed
16 of 50 epochs completed
17 of 50 epochs completed
18 of 50 epochs completed
19 of 50 epochs completed
20 of 50 epochs completed
21 of 50 epochs completed
22 of 50 epochs completed
23 of 50 epochs completed
24 of 50 epochs completed
25 of 50 epochs completed
26 of 50 epochs completed
27 of 50 epochs completed
28 of 50 epochs completed
29 of 50 epochs completed
30 of 50 epochs completed
31 of 50 epochs completed
32 of 50 epochs completed
33 of 50 epochs completed
34 of 50 epochs completed
35 of 50 epochs completed
36 of 50 epochs completed
37 of 50 epochs completed
38 of 50 epochs completed
39 of 50 epochs completed
40 of 50 epochs completed
41 of 50 epochs completed
42 of 50 epochs completed
43 of 50 epochs completed
44 of 50 epochs completed
45 of 50 epochs completed
46 of 50 epochs completed
47 of 50 epochs completed
48 of 50 epochs completed
49 of 50 epochs completed
50 of 50 epochs completed

9. Evaluate the model using the test set¶

Be sure to re-initialize the hidden parameters \(h_0\) and \(c_0\) before running the model.

# CODE HERE
future = 
preds = 

model.eval()

for i in range(future):

# Run the code below to check your results:
preds[window_size:]

# DON'T WRITE HERE

[0.25382155179977417,
 -0.0027704648673534393,
 -0.343053936958313,
 -0.21152164041996002,
 0.23945370316505432,
 0.4895053505897522,
 0.24688751995563507,
 -0.08669154345989227,
 -0.25793153047561646,
 0.022461334243416786,
 0.5438402891159058,
 0.6108715534210205]

10. Inverse transform the predicted values¶

Rescale the predicted values up to the original test set range.

# CODE HERE
true_predictions =

# Run the code below to check your results:
true_predictions

# DON'T WRITE HERE

array([[105.95129313],
       [ 98.05736803],
       [ 87.58871716],
       [ 91.63524249],
       [105.50927345],
       [113.20198736],
       [105.73797111],
       [ 95.47557801],
       [ 90.20746543],
       [ 98.83361172],
       [114.87357457],
       [116.93575791]])

BONUS EXERCISE: Plot the result¶

Plot the true_predictions values together with the original data. Remember to create a range of datetime values for the predicted data.

# CODE HERE

# CODE HERE TO DISPLAY THE END OF THE GRAPH

# DON'T WRITE HERE

# DON'T WRITE HERE

Great job!¶

Copyright Qalmaqihir For more information, visit us at www.github.com/qalmaqihir/