Which model to choose - pytorch CNN

[asadiv]

this is my dataset : https://www.kaggle.com/datasets/ishaqedu/rice-seed-image-dataset-from-pakistan

and this is my model : https://colab.research.google.com/drive/1GM66BXaoGDR9VQ1Syv8_40K9hLcF0Mno?usp=sharing

guide me on wether i need to change my model or not. The test loss and accuracy is not further improving with this one.

[LuluW8071]

• Increase hidden_units, conv_layer
• Use LeakyReLU activation
• Use AdamW optimizer with 3e-3 or SGD with 1e-3
• Zoomed Grain Image also would be more better for feature extraction
• Reasearch on Focal loss as well since one or two calsses are imbalanced.

IMO, this classification dataset doesn't need a big model.

[asadiv]

HEY THERE,
thanks for the feedback. I tried what you suggested but it didnt improve.
It got worst, there were sharp spikes in loss functions of train and test.

[LuluW8071]

import torch
from torch import nn

# Squeeze & Excitation Block 
class SEBlock(nn.Module):
    def __init__(self, channels, reduction=8):
        super().__init__()
        self.squeeze = nn.AdaptiveAvgPool2d(1)
        self.excite = nn.Sequential(
            nn.Linear(channels, channels // reduction),
            nn.LeakyReLU(0.1),
            nn.Linear(channels // reduction, channels),
            nn.Sigmoid()
        )

    def forward(self, x):
        b, c, _, _ = x.size()
        y = self.squeeze(x).view(b, c)
        y = self.excite(y).view(b, c, 1, 1)
        return x * y


class TinyVGG(nn.Module):
    def __init__(self, input_shape: int, hidden_units: int, output_shape: int) -> None:
        super().__init__()

        self.conv_block_1 = nn.Sequential(
            nn.Conv2d(input_shape, hidden_units, 3, padding=1),
            nn.LeakyReLU(0.1),
            nn.Conv2d(hidden_units, hidden_units, 3, padding=1),
            nn.SiLU(),
            SEBlock(hidden_units),
            nn.MaxPool2d(2),
            nn.Dropout(0.15)
        )

        self.conv_block_2 = nn.Sequential(
            nn.Conv2d(hidden_units, hidden_units * 2, 3, padding=1),
            nn.LeakyReLU(0.1),
            nn.Conv2d(hidden_units * 2, hidden_units * 2, 3, padding=1),
            nn.SiLU(),
            SEBlock(hidden_units * 2),
            nn.MaxPool2d(2),
            nn.Dropout(0.2)
        )

        # Extra conv block 
        self.conv_block_3 = nn.Sequential(
            nn.Conv2d(hidden_units * 2, hidden_units * 4, 3, padding=1),
            nn.LeakyReLU(0.1),
            nn.Conv2d(hidden_units * 4, hidden_units * 4, 3, padding=1),
            nn.SiLU(),
            SEBlock(hidden_units * 4),
            nn.MaxPool2d(2),
            nn.Dropout(0.25)
        )

        self.classifier = nn.Sequential(
            nn.Flatten(),
            nn.Dropout(0.3),
            nn.Linear(hidden_units * 4 * 28 * 28, output_shape)
        )

    def forward(self, x: torch.Tensor):
        x = self.conv_block_1(x)
        x = self.conv_block_2(x)
        x = self.conv_block_3(x)
        return self.classifier(x)

• Use horizontal and vertical random flip
• batch_size: 32
• hidden_units: 32
• optimizer (I didnt use scheduler) :

optimizer = torch.optim.AdamW(
    params=model_0.parameters(), 
    lr=2e-3,
    betas=(0.9, 0.999),  
    eps=1e-8,
    weight_decay=1e-4

[asadiv]

hey could you please tell me how did you learn these things in depth. i am having a hard time finding resources (specifically for computer vision.)

[LuluW8071]

Hey there @asadiv, learning is always part of the process in the life. you stumble upon obstacles and you tackle it. I used to implement similar things as you are currently doing using different Kaggle datasets.

There are lots of resources available in the internet. Kaggle Notebooks, Github, YouTube, Medium, Research Papers and many more. Also, you can ask AI or chat with different ppl to share and learn from them. Basically, To know more, you must research and experiment in the AI field.

[mrdbourke]

Hey @asadiv , @LuluW8071 has provided some good advice.

Have you tried using transfer learning? Perhaps this might help.

See the notebook here: https://www.learnpytorch.io/06_pytorch_transfer_learning/

Transfer learning brings in an existing model which has seen a lot of images and adjusts it to your specific task.