Pytorch Tutorial #23 | Exploring Recurrent Neural Networks: A Guide to GRU Implementation

In this tutorial, we will be exploring Recurrent Neural Networks (RNNs) with a focus on Gated Recurrent Units (GRUs) using PyTorch. RNNs are a type of neural network that is well-suited for sequential data, such as time series data, text data, and video data. GRUs are a variant of RNNs that have been shown to be very effective in learning long-range dependencies in sequential data.

Before we begin, make sure you have PyTorch installed on your system. You can install PyTorch using pip by running the following command:

pip install torch torchvision

Now let’s get started with implementing a GRU in PyTorch.

Step 1: Importing the necessary libraries

import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np

Step 2: Defining the GRU model

class GRUNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_layers, output_size):
        super(GRUNet, self).__init__()
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.gru = nn.GRU(input_size, hidden_size, num_layers, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size).to(device)
        out, _ = self.gru(x, h0)
        out = self.fc(out[:, -1, :])
        return out

Step 3: Preparing the data

# Create some dummy data
input_size = 1
hidden_size = 64
num_layers = 1
output_size = 1
seq_length = 5
batch_size = 1

x_train = np.array([[0.1], [0.2], [0.3], [0.4], [0.5]]).reshape(1, seq_length, input_size)
y_train = np.array([[0.6]])

x_train = torch.from_numpy(x_train).float()
y_train = torch.from_numpy(y_train).float()

Step 4: Training the model

# Initialize the model, loss function, and optimizer
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = GRUNet(input_size, hidden_size, num_layers, output_size).to(device)
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Training the model
num_epochs = 1000
for epoch in range(num_epochs):
    outputs = model(x_train)
    loss = criterion(outputs, y_train)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (epoch+1) % 100 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')

Step 5: Making predictions with the model

# Predicting a single value
model.eval()
x_test = np.array([[0.6, 0.7, 0.8, 0.9, 1.0]]).reshape(1, seq_length, input_size)
x_test = torch.from_numpy(x_test).float()
predicted = model(x_test).item()

print(f'Predicted value: {predicted}')

In this tutorial, we have learned how to implement a GRU in PyTorch for a simple time series prediction task. You can further experiment with different hyperparameters, datasets, and tasks to get a better understanding of how GRUs work and how to effectively use them in your projects. I hope this tutorial has been helpful in getting you started with RNNs and GRUs in PyTorch. Happy coding!