Data Science

While I was working in my master’s degree thesis with Accenture we used to work with image of a defined fixed size. In particular, we were working on a very well known Machine Learning challenge which is called fast MRI that was proposed by Facebook AI research (now Meta AI).

I won’t go into much more detail but we had a specific goal that was improving the quality of these images. Once we managed to do that we started asking ourselves if there was a way to store these big images in a compressed form to save some space in the storage.

We decided to apply Principal Component Analysis (PCA).

I hope that this (pretty long) introduction gave you an idea about how PCA is used both in the industry and in the research, as it is a very powerful yet reasonably simple algorithm to reduce the dimensionality of your image and save some space without actually destroying your image’s quality.

Let’s dive in.

1. Intuitive Explanation

Ok, let’s say that you have this kind of data:

If you look at this data you have a specific distribution around a diagonal line. In some way, if you consider the diagonal line only you might be able to change the dimensional of your data switching from 2 dimensions to 1.

The Principal Component Analysis method does exactly that. It finds the directions where you have the most information of the dataset and project your data on them.

Let’s show the result:

And as you could see the component 0 is really informative, and you could just use this component to plot the data.

Imagine you have M dimensions in your dataset. When you use the PCA you end up having:

• A P<M dimensional matrix
• A model that permits you to switch back to the original dimensions

This permits to have an important computational advantage when P<<M because, obviously, storing a model and a small matrix is computationally cheaper than storing a huge matrix.

Imagine that your data is an image. By all means, this is just a matrix of numbers, let’s say N rows and M columns. What you can do is reduce the columns of your matrix and have a N x P matrix with P <<M. In this way you will have a smaller matrix and a model to convert your image back to the original matrix.

The next chapter is pretty technical and explains exactly how PCA does what we said it does. Nonetheless, above there should be all you really need to know to go on and produce a practical implementation of the PCA method. So you should be able to implement the PCA and understand it even if you skip chapter 2.

2. Technical Explanation

So that was the idea. Let’s try to make it more technical. Let’s say that the matrix (image) is A. Let’s say that A is m x p Then, let’s define the matrix B:

The eigenvalue decomposition is the following:

These decomposed matrices are:

• V is an p x p orthonormal matrix of eigenvectors of the matrix.
• Lambda is a p x p square diagonal matrix. The diagonal entries are the eigenvalues that are non-negative real numbers.

The PCA defined matrix is the following:

which is the projection of on these p orthonormal eigenvectors. It has the same shape as the original that is m x p.

One may re-construct A using the following formula, if using all the eigenvectors.

This is because eigenvectors are orthonormal. It is often that the first few (ranked by the value of the eigenvalues in descending order) eigenvectors contain most of the overall information of the original matrix A. In this case, we can use only a small number of eigenvectors to reconstruct the matrix A. For example we can use k<<p number of eigenvectors. The k vectors are ordered in terms of the variance values. Using them, you can have a smaller matrix A_PCA and a model (this matrix multiplication we talked about) to convert it to the original sized matrix.

3. Practical Implementation

Even if performing the operation described above is not terribly difficult, there are some very practical tools that do it for you and accelerate the process. The great sklearn is probably the most famous one in the area of Data Science. Here is the documentation.

Let’s use a free download image of my team’s people (❤):

We are going to do these things:

1. Apply the PCA methods with different numbers of components
2. Reconstruct the images with lower numbers of components
3. Evaluate the quality of the reconstructions comparing them to the original one

We analyzed how 1. and 2. work in detail, so we are just going to spend some time into the evaluation.

There are lots of ways to evaluate the quality of a reconstructed image having the ground truth. Nonetheless, to keep it simple, let’s use a pretty generic but useful method that is the Normalized Frobenius Norm of the difference.

Let’s use 4 different numbers of components: 0%, 33%, 66% and 100%.

Let’s plot the reconstructed images:

I find this beautiful. The first reconstruction is absolutely nonsense. While we increase the number of components, of course, the reconstruction becomes similar to the original image. Nonetheless, it means that we are reducing our computational advantage as well, because we are storing a larger number of component (the reduced image’s dimensionality increases).

Let’s plot the distance (Frobenius Norm) between the original image and the reconstructed ones as the number of components vary:

If I was in the position of using it in a professional or research environment, I’ll probably use far more points to fit the curve better and understand an optimal number of components. However, I feel it is enough for our purposes, as it is showing that the reconstruction gets better while we add the number of components, which is totally reasonable.

4. Conclusions

Dimensionality reduction is something that is, in some cases, a must. When you have to preprocess high dimensionality image, it is always a good practice to see if you can use PCA to store lower dimensionality images and the code that makes you do it is incredibly easy.

If you liked the article and you want to know more about Machine Learning, or you just want to ask me something you can:

A. Follow me on Linkedin, where I publish all my stories B. Subscribe to my newsletter. It will keep you updated about new stories and give you the chance to text me to receive all the corrections or doubts you may have. C. Become a referred member, so you won’t have any "maximum number of stories for the month" and you can read whatever I (and thousands of other Machine Learning and Data Science top writer) write about the newest technology available.

Ciao 🙂

Share This Article

• Share on Facebook
• Share on LinkedIn
• Share on X