Neural Networks in PyTorch
Basics of nn.Module and nn.Parameter
The ‘nn.Module’ is a base class in PyTorch for all neural network modules. It includes the trainable parameters and defines the forward method for performing forward-pass computations. Thus, it is responsible for parameter management and submodule management, Serialization, and Loading.
On the other hand, nn.Parameter is the subclass of the torch.Tensor that is responsible for parameter initialization, optimization, and access. The nn.Parameter tensors are defined as attributes within the nn.Module subclass. The nn.Parameter tensors behave like regular tensors but are recognized as model parameters by PyTorch’s Autograd system.
Building Neural Network using PyTorch
Let’s create simple neural network model using the Iris dataset, which is popular dataset for classification tasks. The Iris dataset contains measurements of iris flowers, including sepal length, sepal width, petal length, and petal width, along with their corresponding species (setosa, Versicolor, or Virginia).
Defining Neural Network Architecture
- We are first loading the Iris dataset and split it into training and testing sets.
- After this, we define simple neural network architecture. Here, we are defining the input layer (fc1) that contains the linear transformation (nn.Linear) to map the input features to the hidden layer.
- Then, we have to apply the ReLU activation function (nn.ReLU) to introduce non-linearity in the model.
Python
import torchimport torch.nn as nnimport torch.optim as optimfrom sklearn.datasets import load_irisfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import StandardScaler# Load the Iris datasetiris = load_iris()X, y = iris.data, iris.target# Split the dataset into training and testing setsX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)# Standardize the featuresscaler = StandardScaler()X_train = scaler.fit_transform(X_train)X_test = scaler.transform(X_test)# Define the neural network architectureclass SimpleNN(nn.Module): def __init__(self, input_size, hidden_size, output_size): super(SimpleNN, self).__init__() self.fc1 = nn.Linear(input_size, hidden_size) # Input layer self.relu = nn.ReLU() # Activation function self.fc2 = nn.Linear(hidden_size, output_size) # Output layer def forward(self, x): x = self.fc1(x) x = self.relu(x) x = self.fc2(x) return x |
Training a Neural Network on a Simple Dataset
After defining the architecture, we have to train the model. In the following code snippet, we set a random seed for reproducibility, and define the input, hidden, and output sizes of the neural network architecture. After this, we instantiate the neural network model, define the loss function (CrossEntropyLoss) and optimizer (Adam), convert the training data to PyTorch tensors, and train the model for a fixed number of epochs.
During training, we perform forward pass computations to obtain predicted outputs and calculate the loss between predicted and actual labels. Also, we have to update model parameters using the optimizer.
Python
# Set random seed for reproducibilitytorch.manual_seed(42)# Define the input size, hidden size, and output size of the neural networkinput_size = X.shape[1]hidden_size = 10output_size = len(iris.target_names)# Instantiate the neural networkmodel = SimpleNN(input_size, hidden_size, output_size)# Define the loss function and optimizercriterion = nn.CrossEntropyLoss()optimizer = optim.Adam(model.parameters(), lr=0.01)# Convert datto PyTorch tensorsX_train_tensor = torch.FloatTensor(X_train)y_train_tensor = torch.LongTensor(y_train)# Train the modelnum_epochs = 100for epoch in range(num_epochs): # Forward pass outputs = model(X_train_tensor) loss = criterion(outputs, y_train_tensor) # Backward pass and optimization optimizer.zero_grad() loss.backward() optimizer.step() # Print the loss every 10 epochs if (epoch+1) % 10 == 0: print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}') |
Evaluating the Trained Model
Now, our Model has been trained. We will evaluate the model on the test dataset. For this, we have to convert the test dataset from NumPy Arrays into the PyTorch Sensors using the torch.FloatTensor() and torch.LongTensor(). Then, we pass the test input X_test_tensor through the trained model to obtain the output. At last, these are compared with the actual predicted labels y_test_tensor to calculate the accuracy.
Python
# Evaluate the modelwith torch.no_grad(): X_test_tensor = torch.FloatTensor(X_test) y_test_tensor = torch.LongTensor(y_test) outputs = model(X_test_tensor) _, predicted = torch.max(outputs, 1) accuracy = (predicted == y_test_tensor).sum().item() / len(y_test_tensor) print(f'Accuracy on the test set: {accuracy:.2f}') |
Output:
Epoch [10/100], Loss: 0.7783
Epoch [20/100], Loss: 0.5399
Epoch [30/100], Loss: 0.3921
Epoch [40/100], Loss: 0.2934
Epoch [50/100], Loss: 0.2166
Epoch [60/100], Loss: 0.1639
Epoch [70/100], Loss: 0.1284
Epoch [80/100], Loss: 0.1050
Epoch [90/100], Loss: 0.0902
Epoch [100/100], Loss: 0.0800
Start learning PyTorch for Beginners
Machine Learning helps us to extract meaningful insights from the data. But now, it is capable of mimicking the human brain. This is done using neural networks, which contain the various interconnected layers of nodes containing the data. This data is passed to forward layers. Subsequently, the model learns from the data and predicts output for the new data.
PyTorch helps us to create and train these neural networks that act like our brains and learn from the data.
Table of Content
- What is Pytorch?
- Why use PyTorch?
- How to install Pytorch ?
- PyTorch Basics
- Autograd: Automatic Differentiation in PyTorch
- Neural Networks in PyTorch
- Working with Data in PyTorch
- Intermediate Topics in PyTorch
- Validation and Testing
- Frequently Asked Questions
Contact Us