Exercise

E3: Background and Blobs

Objective 1

Write your own background subtraction algorithm using simple frame differencing.

  1. Use the code from E2 to capture and display frames from your camera. You may want to use median blur and/or other image processing filters to clean up the captured frame (remove noise, correct lighting, flip, etc.).

  2. Add code so that when the user presses 'space', the current frame is saved to a variable as the background frame.

  3. Add code to calculate the absolute difference between the current frame and the captured background frame (for all frames captured after the background frame is saved).

  4. Use image processing functions like morphological opening and thresholding to convert the absolute difference into a clean foreground mask. You'll have to experiment to get a good mask.

  5. Test your code with different backgrounds. In my example below you can see that skin colour is similar to the brown envelope, so background subtraction is not as robust. Notice how shadows increase the size of blobs.

HINT: This will work much better if you turn off all the auto-exposure settings on your camera. You can try to do this in OpenCV, but it's easier to use the camera settings panel in your OS. If you're using a built-in camera on a Mac, you may need something like Sight Control.

objective 1 example

Objective 2

Track an object based on it's area using 'blobs' in the form of contours.

  1. Add code to your objective 1 code to find blobs in the foreground mask. With OpenCV, I used findContours.

  2. Calculate the area of each blob (i.e. each contour in OpenCV) and for all blobs larger than some minimum area cutoff, calculate the area and display the area on the blob. With OpenCV, I used contourArea and moments to find the blob center of mass to display the text for each area measurement. You will find it useful to draw all contours in your frame to see what's being tracked (I used drawContours).

  3. Find an object to track and set it in the scene. Set it down in multiple locations to measure the range of area measurements for that object's blob. If you have a built in camera and it's hard to aim it down on a desk, you can tape an object to a wall and move it around.

  4. Use those thresholds to try and track that object when it's set down. In my code I outlined the object I wanted to track in thick red and all other objects in thin yellow.

objective 2 example

Optional Things to Try

Experiment with the built-in background subtraction algorithms in OpenCV. This tutorial provides a good overview.

Experiment with OpenCV's SimpleBlobDetector algorithm. It's less flexible than findContours, but for some cases provides a good solution. This tutorial gives a good introduction.

Submit

Follow the submission instructions on the course information page. Include your code and one screen capture from each objective. In your solution description, write about any problems you ran into, and the resource(s) you used (blog posts, online tutorials, stackoverflow posts, papers, textbooks, etc.). These resources should be annotated with brief descriptions of what the resource is and how it helped you.

You can combine your code, screen captures, and description together into one PDF or hand in the code and description separately. You should be able to submit multiple files to Learn.