MIT’s Open Source Algorithm Automates Object Detection in Images (with GitHub link)

Overview

• MIT’s CSAIL researchers have unveilved an approach that automates certain parts of image editing, including object detection
• The approach is called Semantic Soft Segmentation (SSS)
• It combines the color and texture of images with information produced by a trained neural network

Introduction

Fixing corrupt or bad images, filling the gaps in existing images, translating the background from one image to another – these are all applications of computer vision that have transcended imagination to become a reality this year. And now researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have thrown their hat into this ring with their latest study.

They have unveiled an approach called ‘Semantic Soft Segmentation (SSS)’ that uses machine learning to automate certain parts of the image editing process, including detecting objects! What takes an expert editor several minutes (or even hours) and involves tweaking and analyzing images pixel-by-pixel can now be done in a matter of seconds thanks to SSS. The below image shows how the algorithm works to detect objects in a given image:

👁 Image

SSS works by analyzing the texture and color of the given image. It then combines these attributes with data provided by a trained neural network model that information about what kind of objects are present in the image.

As mentioned in the research paper (link below), the algorithm “generates soft segments that correspond to semantically meaningful regions in the image by fusing the high-level information from a neural network with low-level image features fully automatically”. This makes tasks like parsing objects, editing backgrounds, etc. quite trivial and removed the need for expertise (at least as far as casual users are concerned!).

While the current version of SSS is heavily focused on static images, there is an acknowledgement by the researchers that this will be fine-tuned for video applications in the future.

I have mentioned a few resources below to help you get acquainted with this study and also try it out by yourself:

• Research paper
• GitHub repository
• MIT’s blog post

Also do check out the short video below which shows SSS in it’s full glory:

Our take on this

Another week, another breakthrough study in computer vision. Deep learning has carved a niche for itself in this field and you can expect to see more and more of these projects coming soon, especially in video editing. CGI effects that we see in movies could easily be done using techniques like SSS (once they have been improved a bit more).

Should expert artists be worried? Deep learning does have the potential to generate a decent work of art but that human touch and intuition continues to elude even the finest works of machines.

If this field interests you, you can take our Computer Vision using Deep Learning course which aims to help you dip your toes and come out a master in this exciting and upcoming field.

Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!

Login to continue reading and enjoy expert-curated content.

Free Courses