Principal Component Analysis is a statistical method that converts a set of possibly correlated variables to a set of linearly uncorrelated values by using orthogonal transformations. In simple words given a dataset with multiple dimensions, it helps reduce the number of dimensions hence making the data easier to read.

Step 1: Original Plans

I came into this class with the idea that I wanted to understand and hopefully write an algorithm that would be able to perform facial recognition when supplied with images. I had no prior experience or knowledge of anything to do with facial recognition and had no idea how hard it is to achieve something like this. After talking to professor Malloch I realized that I have to learn a lot of things before being able to understand fully the task I eventually planned on achieving.

After a little bit of research, I finally decided that more than anything I needed to learn linear algebra and some basics of machine learning and settled on PCA (principal component analysis) to be my goal for this class.

Step 2: Research

The first step was to visit the Library and find any book that introduced me to machine learning and more specifically image processing. This turned out to be much harder than I thought and I ended up with nothing much from it. I then decided to ask a friend who worked in the Vision Lab who asked me to look into linear algebra and more specifically eigenvectors and eigenvalues. I had some experience with linear algebra from a class that I had taken in my sophomore year but did not understand how eigenvectors or eigenvalues could be useful when dealing with images. As I researched more I understood that images were nothing but huge data sets and could hence be treated as matrices and it became a little bit clearer to me as to why eigenvectors were relevant to what I was doing. At this point, I decided that I should learn how to read images using python as I was going to use python for my project. Initially, I started out by using CV2.imread to read the images but that turned out to be really slow and hence I decided to use glob and PIL.image.open to do so as this is much faster. This process on paper seems relatively non-time consuming but actually took a good amount of time as I had to learn how to install and import different libraries onto PyCharm (IDE) and then read through the documentation online for each library. In the process of doing so, I also learned how to use pip install statements in command prompt.

After this, the next step was figuring out what exactly I wanted to do and learn in image processing and initially, I was planning on doing template matching but while researching for it I learned about PCA and found it to be more interesting so I decided to go with PCA instead. The first term that kept popping up was the K-NN (K- nearest neighbor) algorithm. This was my first ever exposure to a machine learning algorithm. I learned about the training and testing data and what the 'training' of an algorithm means. Understanding the K-NN algorithm was also challenging but it was very satisfying to finally understand how it works. I am currently working on having the code for K-NN working and I am very close to completion.

Step 3: Difficulties Faced and Lessons Learnt

The first big difficulty was the scope of the project itself. This was more of a research-oriented than a physical one. As the weeks went by some times I would look at the progress my peers were making and feel that I was not doing enough or that I was not making fast enough progress and that was sometimes very demotivating. Talking to Professor Malloch and just reassuring myself that I was indeed learning stuff that was very new to me helped me keep going. Another problem was that knowing theoretical stuff and applying it are two different things. Although I knew what I had to, actually coding it in python was a different story. This is where just reading documentations online and asking friends who knew about it more helped a lot to finally figure out a plan of action.

I personally think that having a larger library of books and documents at M5 could help people who are working on projects. Also having a real-time digital record of the projects being done by students so that other students and staff can look at it and get involved if it interests them is a good idea for M5.

As the project comes to an end I have learned so much in such a short span of time. I have gained a very working knowledge of machine learning and feel like I have taken the first steps to get more involved in it. I have realized that I like computer vision and that I might want to pursue this even in the future. Most importantly I have learned what PCA is, why its so important and how to make use of it.

Step 4: Next Steps

For me, this was just scratching the surface of something much more vast and something that's very important in today's world, i.e machine learning. I plan on taking courses related to machine learning in the near future. I also plan on building my way up to facial recognition as that is where this whole project started. I also have ideas for a security system which utilizes a combination features (one of them being the face of the person) to make it really secure and this is something that I want to work on in the future when I have a wider understanding of things.

For anyone like me who is interested in machine learning and image processing but has no prior experience, I would highly suggest first learning and understanding linear algebra along with statistics (especially distributions). Second I would suggest reading Pattern Recognition and Machine Learning by Christopher M. Bishop. This book helped me understand the basics of what I was getting into and is structured very well.