Exercise

E4: Colour

Objective 1

Write a colour tracker using hue range testing.

  1. Use the code from E2 to capture and display frames from your camera (you don't need to do any background subtraction). You may want to use median blur and/or other image processing filters to clean up the captured frame (remove noise, correct lighting, mirror, etc.).

  2. Add code to convert each frame from BGR to HSV colour space. I used cv2.cvtColor and cv2.COLOR_BGR2HSV.

  3. Find an object with an area of consistent bright colour. You should avoid objects that are planar (like a piece of paper) or red. I used a pen with a bright green coloured cap.

  4. Capture a frame of the object using your code and find the range of hue values in the coloured area you want to track using Photoshop or an online colour picker like http://imagecolorpicker.com.

  5. Add code to create a mask of all pixels within the hue range of the object you want to track, as well as a reasonable range of saturation and value to avoid very dark or grey objects. I used cv2.inRange. Remember that OpenCV parametrizes hue in the range 0 to 180, while colour pickers will almost always give you hue as a value between 0 and 360 degrees.

  6. Add code from E3 to find the blobs in the colour mask. I used findContours, contourArea, and moments. Display the coordinates and the centre point of all blobs greater than a threshold area. I set my area threshold to 100 for my 640 x 480 frames to track the small end of my pen.

###Optional Experiments

  • Try different colours including red and darker colours.

  • Change your code to track all brightly coloured objects, regardless of hue.

  • Try different objects including planar ones. Why do planar objects not work as well?

  • Add code to do something with your tracked object like draw lines, perhaps with different thicknesses depending on the size of the blob.

  • Add code to track two or more objects with different colours.

HINT: This will work much better if you turn off all the auto-exposure settings on your camera. You can try to do this in OpenCV, but it's easier to use the camera settings panel in your OS. If you're using a built-in camera on a Mac, you may need something like Sight Control.

objective 1 example

Objective 2

Write a trainable colour tracker using histogram backprojection.

  1. Add code to your objective 1 to define a smallish area in your frame as a training area. Do this my drawing a square somewhere.

  2. Add code to train a 2D histogram. When the space bar is pressed grab the region of interest in the frame corresponding to the square you drew, and calculate a hue and saturation histogram from this region. I used cv2.calcHist with 16 bins. You should normalize the histogram to increase the probability density.

  3. For debug purposes, display or write out images for the region you used to train and the 2D histogram (which is just a very small greyscale image).

  4. Once trained, calculate the backprojection of each frame using the trained histogram. I used cv2.calcBackProject.

  5. Turn the backprojection into a mask and use image processing operations to clean it up. I also used cv2.inRange to avoid masking very dark or grey pixels in the original image.

  6. Use the same code from objective 1 to track the blobs in this colour mask.

Training

objective 2 training example

Tracking

objective 2 tracking example

###Optional Experiments

  • Try diffent bin sizes.

  • Try training on hue and value.

  • Try going one step further and use Meanshift or CamShift to track the object.

Submit

Follow the submission instructions on the course information page. Include your code and one screen capture from each objective. In your solution description, write about any problems you ran into, and the resource(s) you used (blog posts, online tutorials, stackoverflow posts, papers, textbooks, etc.). These resources should be annotated with brief descriptions of what the resource is and how it helped you.

You can combine your code, screen captures, and description together into one PDF or hand in the code and description separately. You should be able to submit multiple files to Learn.