This is a neural network implementation in python from scratch. In this notebook, multiple hidden layers can be added to the Neural Network. I have used the mean squared error loss function at the last layer for calculating gradient. The neural network is trained using the backpropagation algorithm.
The network will start with the number of input features and then the size of hidden layers, then at the end add size of output layer. Each layer will contain same activation function which can be given in input, by default. Multiple activation functions can be found like relu, tanh, sigmoid, leaky relu and softmax can be choosen for layers. For initialization of weights initialization with zero's, with random number between -0.1 to 0.1 or normal distribution(0, 1) can be opted.
Dense Class is a skeleton for the Neural Network which contains forward and backward propogation function of the layers and the initialization of weights and biases.
Activation Class is skeleton for ofcourse activation functions containing definition of activation functions which will be used in forward propogation after hidden layer with which it is assosiated.
Neural Network Class creates/initialize the Neural network by creating layers and fit function train the model on the input data based on given output lable. The predict function predicts the output lable for any given input by doing forward pass through the trained neural network.
The neural network has input neurons, hidden neurons and output neurons. The input layer is connected to the hidden layer and the hidden layer is connected to the output layer. The neural network is trained using the backpropagation algorithm. The weights are updated using the gradient descent algorithm. The Mean Square Error is used at the output layer to check the difference in the predicted and actual labels and its derivatve is used in improving the weights of the hidden layer using back-propogation.
I have used the MNIST dataset (https://www.kaggle.com/datasets/hojjatk/mnist-dataset) for training and testing the neural network. The MNIST database of handwritten digits(between 0 - 9) has a training set of 60,000 examples, and a test set of 10,000 examples. Each input has 784 features and the ouput contains single integer which it is supposed to be. The neural network is trained to classify the MNIST dataset into their corresponding class, by calculting the probability of it being all possible number between 0 - 9 and returning the number corresponding to the highest probability.
You can run this notebook and train the neural network on the MNIST dataset. You can also change the number of hidden neurons and the learning rate to see how it affects the performance of the neural network. With the current setup I am getting accuracy of 93.27%.
I have implemented a neural network from scratch in this notebook. Testing of neural network is done on the MNIST dataset and is able to classify the digits into the correct classes. The neural network can be further improved by using different activation functions, loss functions, and optimization algorithms.