Learn PyTorch with an ex Amazon SDE: KFold Validation for a CNN on the Fashion MNIST dataset

If you’re interested in deep learning and want to learn how to use PyTorch to build and train a convolutional neural network (CNN), then you’re in the right place. In this article, we’ll explore KFold validation for a CNN using the Fashion MNIST dataset, and we’ll be guided by a former Amazon software development engineer (SDE).

What is PyTorch?

PyTorch is an open source machine learning library based on the Torch library. It is primarily developed by Facebook’s AI Research lab. PyTorch provides a wide range of tools and libraries to build deep learning models and is widely used in the AI and machine learning community.

What is KFold Validation?

KFold validation is a technique used to assess the performance of a machine learning model. It involves splitting the dataset into K subsets (or folds), training the model on K-1 of the subsets, and evaluating it on the remaining subset. This process is repeated K times, with each subset used exactly once as the validation data. The results are then averaged to produce a single performance metric.

The Fashion MNIST dataset

The Fashion MNIST dataset is a collection of 28×28 grayscale images of 10 different types of clothing items, such as shirts, dresses, and shoes. It is a popular benchmark for image classification tasks and is often used to test the performance of machine learning models.

Training a CNN with PyTorch

Now, let’s dive into the code. We’ll create a simple CNN using PyTorch to classify the images in the Fashion MNIST dataset. Our model will consist of convolutional and max pooling layers, followed by fully connected layers and a softmax output layer. We’ll use the KFold validation technique to evaluate the performance of our model.

Below is an example of how the code might look like:

import torch
import torchvision
import torchvision.transforms as transforms
import torch.nn as nn
import torch.optim as optim
from sklearn.model_selection import KFold

# Load the Fashion MNIST dataset
transform = transforms.Compose([
    transforms.ToTensor(), 
    transforms.Normalize((0.5,), (0.5,))
])
trainset = torchvision.datasets.FashionMNIST(root='./data', train=True,
                                        download=True, transform=transform)
testset = torchvision.datasets.FashionMNIST(root='./data', train=False,
                                       download=True, transform=transform)

# Define the CNN model
class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 4 * 4, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 4 * 4)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

# Split the dataset into K folds
kfold = KFold(n_splits=5, shuffle=True)

for train_index, test_index in kfold.split(trainset):
    # Split the data into training and validation sets
    trainset_fold = torch.utils.data.Subset(trainset, train_index)
    validset_fold = torch.utils.data.Subset(trainset, test_index)
    
    # Create dataloaders
    trainloader = torch.utils.data.DataLoader(trainset_fold, batch_size=64,shuffle=True)
    validloader = torch.utils.data.DataLoader(validset_fold, batch_size=64,shuffle=False)
    
    # Train the model
    model = CNN()
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)

    for epoch in range(10):  # loop over the dataset multiple times
        running_loss = 0.0
        for i, data in enumerate(trainloader, 0):
            inputs, labels = data
            optimizer.zero_grad()  # zero the parameter gradients
            outputs = model(inputs)  # forward pass
            loss = criterion(outputs, labels)  # compute the loss
            loss.backward()  # backward pass
            optimizer.step()  # update weights
            running_loss += loss.item()
        print(f"Epoch {epoch+1}, Loss: {running_loss / len(trainloader)}")
    
    # Validate the model
    correct = 0
    total = 0
    with torch.no_grad():
        for data in validloader:
            images, labels = data
            outputs = model(images)
            _, predicted = torch.max(outputs.data, 1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()

    print(f"Accuracy: {100 * correct / total}%")

print('Finished Training')

With this code, we create a CNN model and use KFold validation to train and evaluate it on the Fashion MNIST dataset. We can then use the trained model to make predictions on new images and assess its performance.

Conclusion

Learning PyTorch and implementing KFold validation for a CNN on the Fashion MNIST dataset is a great way to gain practical experience with deep learning. By following the guidance of an ex Amazon SDE, you can develop a deep understanding of these concepts and techniques, and apply them to real-world problems. So, go ahead and dive into the world of PyTorch and start building your own deep learning models!