Exercise

E6: Features

Use SIFT to track an object.

Objective 1

Write code to compute SIFT keypoints and descriptors for a distinctive object. I used a card from a child's game with lots of image details.

  • Make sure you capture the image of object using the same camera you'll use to track it in objective 2.

  • Crop the image to remove any background texture: you want SIFT to only use the actual object.

  • I converted my object to grayscale before computing SIFT features.

  • Display the original object image and the image with the keypoints (using drawKeypoints). You should see many keypoints!

objective 1 output

Objective 2

Track the object in a video feed by using one of the flann matching methods (like knnMatch) to test if the image you trained in objective 1 appears in the the video frame.

  • You don't need to use background subtraction in this exercise.

  • I used Lowe's "ratio test" as explained in the original paper. This tutorial explains how.

  • Draw all matched keypoints in the video frame to show when the object is being tracked. I used yellow circles. Note that the other card has no yellow circles, so it isn't being tracked.

objective 2 output

Optional Experiments

  • Train SIFT on more than one object and identify each object. You could draw text on the image.

  • Compute a position for the object, perhaps using the mean keypoint location.

  • Time each frame to see how long SIFT takes to run (it's quite slow).

  • Compare it with SURF or one of the other methods.

Submit

Follow the submission instructions on the course information page. Include your code and one screen capture from each objective. In your solution description, write about any problems you ran into, and the resource(s) you used (blog posts, online tutorials, stackoverflow posts, papers, textbooks, etc.). These resources should be annotated with brief descriptions of what the resource is and how it helped you.

You can combine your code, screen captures, and description together into one PDF or hand in the code and description separately. You should be able to submit multiple files to Learn.