Example 1: Predicting Sequential Data: An RNN Approach Using PyTorch
To use an RNN to predict the next value in a series of numbers, we will build a basic synthetic dataset. This will assist us in comprehending the fundamentals of RNN operation and PyTorch implementation. Step-by-Step Implementation:
Step 1: Import Libraries
First, we need to import the necessary libraries.
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
Step 2: Create Synthetic Dataset
We will create a simple sine wave dataset. The goal is to predict the next value in the sine wave sequence.
# Generate sine wave data
def generate_data(seq_length, num_samples):
X = []
y = []
for i in range(num_samples):
x = np.linspace(i * 2 * np.pi, (i + 1) * 2 * np.pi, seq_length + 1)
sine_wave = np.sin(x)
X.append(sine_wave[:-1]) # input sequence
y.append(sine_wave[1:]) # target sequence
return np.array(X), np.array(y)
seq_length = 50
num_samples = 1000
X, y = generate_data(seq_length, num_samples)
# Convert to PyTorch tensors
X = torch.tensor(X, dtype=torch.float32)
y = torch.tensor(y, dtype=torch.float32)
print(X.shape, y.shape) # Output: (1000, 50), (1000, 50)
Output:
torch.Size([1000, 50]) torch.Size([1000, 50])
Step 3: Define the RNN Model
Next, we will define the RNN model.
class SimpleRNN(nn.Module):
def __init__(self, input_size, hidden_size, output_size):
super(SimpleRNN, self).__init__()
self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
self.fc = nn.Linear(hidden_size, output_size)
def forward(self, x):
h0 = torch.zeros(1, x.size(0), hidden_size).to(x.device)
out, _ = self.rnn(x, h0)
out = self.fc(out)
return out
input_size = 1
hidden_size = 20
output_size = 1
model = SimpleRNN(input_size, hidden_size, output_size)
Step 4: Train the Model
Now, we will train the model using Mean Squared Error (MSE) loss and the Adam optimizer.
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
# Training loop
num_epochs = 100
for epoch in range(num_epochs):
model.train()
outputs = model(X.unsqueeze(2)) # Add a dimension for input size
loss = criterion(outputs, y.unsqueeze(2))
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch + 1) % 10 == 0:
print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}')
Output:
Epoch [10/100], Loss: 0.3548
Epoch [20/100], Loss: 0.2653
Epoch [30/100], Loss: 0.1757
Epoch [40/100], Loss: 0.0921
Epoch [50/100], Loss: 0.0592
Epoch [60/100], Loss: 0.0421
Epoch [70/100], Loss: 0.0306
Epoch [80/100], Loss: 0.0222
Epoch [90/100], Loss: 0.0151
Epoch [100/100], Loss: 0.0093
Step 5: Visualize the Results
Finally, we will visualize the predictions made by the model.
# Make predictions
model.eval()
with torch.no_grad():
predictions = model(X.unsqueeze(2)).squeeze(2).numpy()
# Plot results
plt.figure(figsize=(10, 6))
plt.plot(y[0].numpy(), label='True')
plt.plot(predictions[0], label='Predicted')
plt.legend()
plt.show()
Output:
Implementing Recurrent Neural Networks in PyTorch
Recurrent Neural Networks (RNNs) are a class of neural networks that are particularly effective for sequential data. Unlike traditional feedforward neural networks, RNNs have connections that form directed cycles, allowing them to maintain a hidden state that can capture information from previous inputs. This makes them suitable for tasks such as time series prediction, natural language processing, and more.In this article, we will explore how to implement RNNs in PyTorch.
Table of Content
- Introduction to Recurrent Neural Networks
- Building an RNN from Scratch in Pytorch
- Setting Up the Environment
- Steps to Build an RNN
- Example 1: Predicting Sequential Data: An RNN Approach Using PyTorch
- Example 2: Sentiment Analysis with RNN: Classifying Movie Reviews Using PyTorch
Contact Us