I recently finished a project where I built a basic image classifier from scratch without using TensorFlow or PyTorch – just Numpy. I wanted to really understand how image classification works by coding everything by hand. It was a challenge, but I learned a lot.

The goal was to classify images into three categories – cats, dogs, and random objects. I collected around 5,000 images and resized them to be the same size. I started by building the convolution layer, which helps detect patterns in the images. Here’s a simple version of the convolution code:

python

import numpy as np

def convolve2d(image, kernel):
    output_height = image.shape[0] - kernel.shape[0] + 1
    output_width = image.shape[1] - kernel.shape[1] + 1
    result = np.zeros((output_height, output_width))
    
    for i in range(output_height):
        for j in range(output_width):
            result[i, j] = np.sum(image[i:i+kernel.shape[0], j:j+kernel.shape[1]] * kernel)
    
    return result

The hardest part was getting the model to actually learn. I had to write a basic version of gradient descent to update the model’s weights and improve accuracy over time:

python

def update_weights(weights, gradients, learning_rate=0.01):
    for i in range(len(weights)):
        weights[i] -= learning_rate * gradients[i]
    return weights

At first, the model barely worked, but after a lot of tweaking and adding more data through rotations and flips, I got it to about 83% accuracy. The whole process really helped me understand the inner workings of convolutional neural networks.

If anyone else has tried building models from scratch, I’d love to hear about your experience :)