Week 6 (July 1 –July 7) Progress

 Added tests to classdef  "ClassificationSVM " and "fitcsvm", Started studying algorithms and mathematics for implementing ClassificationNeuralNetwork classdef

This week, I added some tests to the ClassificationSVM.m and fitcsvm.m files. I also started studying the mathematics behind neural network classifiers. After studying the MATLAB documentation, I identified the following components for the ClassificationNeuralNetwork class:

  • Weights initializer: Layer Weights are initialized by 'glorot' or 'he'
  • Biases initializer: Biases are initialized by 'zeros' or 'ones'
  • Loss function: Cross entropy
  • Solver used: L-BFGS (Limited-memory quasi-Newton code for bound-constrained optimization)

After some research, I have outlined the workflow that I will be following:

  1. Initialize weights and biases using the initializerParameter function
  2. Implement forward propagation
  3. Implement the predict function (using the initialized weights and biases)
  4. Implement the training portion of the neural network (the most challenging part). Depending on the complexity, I may choose between normal gradient descent or L-BFGS.

Resources I found helpful include:

  • "Neural Network from scratch in Python" by Sentdex
  • Andrew Ng's lecture, where he mentions that understanding the detailed mathematics behind the L-BFGS optimization technique is not necessary; we just need to know how to use it.

After searching for some built-in Octave functions that might help in implementing the training part, I found the bfgsmin function from the Optim package.

Our next meeting is scheduled for July 9th at 16:30 UTC. Before the meeting, I will try to wind up any remaining work of ClasssificationSVM. Following the meeting, I will make a pull request and start working on the ClassificationNeuralNetwork class.

I will also have a Midterm Evaluation as part of GSoC, which will take about 10 minutes to complete. The deadline for it is July 12th.

Comments

Post a Comment

Popular posts from this blog

My Plans for Google Summer of Code 2024: A Brief Overview of My Project and Timeline.

  My Plans for Google Summer of Code 2024: A Brief Overview of My Project and Timeline. Hello everyone! I am Pallav Purbia from Rajasthan, India currently pursuing my Btech in Electrical and Electronics Engineering with a minor degree in Artificial Intelligence and Machine learning from National Institute of Technology Delhi. I am excited to share that I will be contributing to the  Statistics package  of  GNU Octave  this summer.  Andreas Bertsatos  will be mentoring me throughout this project. I feel incredibly fortunate to be working with him. My Fork of the repository can be found  here  where you can check out the progress of the project. Weekly updates and the project’s overall progress will be documented in  my blog . Overview of the Project: Octave is a pioneer in open-source mathematical-oriented programming language for scientific computing and numerical computation and...
Read more

Week 1 (May 27 - June 2) Progress

Week 1 (May 27 - June 2) Progress Implemented the constructor of the ClassificationSVM and added the predict method to the classdef . This week, I started my work by creating the necessary files ( ClassificationSVM , ClassificationNeuralNetwork , fitcsvm , fitcnet ) and placing them in the appropriate folders. Note: I have added ClassificationNeuralNetwork and fitcnet to my fork, but not much work has been done on them yet. According to the timeline, I will be focusing on these files at the end of July and August. I wrote the help documentation for ClassificationSVM and fitcsvm in Texinfo, within their respective files. The fitcsvm Texinfo contains help related to the expected inputs from the user. The ClassificationSVM Texinfo contains help related to the returned object properties. I studied the svmtrain and svmpredict functions from LIBSVM for Octave. I will be using them for the background algorithm. I was implementing the following tasks in the classdef : Taking the inp...
Read more

Week 7 (July 8 –July 14) Progress

Started studying algorithms and mathematics for implementing the ClassificationNeuralNetwork class This week, I started work on the ClassificationNeuralNetwork.m and fitcnet.m files. I also began studying the mathematics behind neural network classifiers. After reviewing the  MATLAB documentation , I identified the following components for the ClassificationNeuralNetwork class: Weights initializer: Layer weights are initialized using either 'glorot' or 'he'. Biases initializer: Biases are initialized using either 'zeros' or 'ones'. Loss function: Cross entropy. Solver used: L-BFGS (Limited-memory quasi-Newton code for bound-constrained optimization). Initially, I am focusing on getting the forward propagation to work properly. You can see my work  here . Resources I found helpful include: " Neural Network from scratch in Python " by Sentdex (unfortunately, the playlist was never completed). Andrew Ng's  lecture , where he mentions t...
Read more