E5: Tags

Build on the code from the fiducial marker tutorial to identify and track the 2D position of a (slightly) more robust set of fiducial markers. See examples below and this PDF.

These markers have the following geometric and topological constraints:

The contour hierarchy has exactly 3 levels.
The top level (the root, level 1) is a square.
Inside that square is a single square (this is level 2).
Inside the single level 2 square are one or more blobs (these are all on level 3). The ID of the marker is determined by the number of blobs.

Testing for a perfect square may over constrain the search because the square will distort in a perspective view. A suitable loose approach is to consider any convex quadrilateral as a “square.” You can test for convex quadrilaterals using approxPolyDP and isContourConvex. There are no other size or position constraints for the marker.

Objective 1

Write a function to detect these kinds of markers in a test image. Each detected marker must be highlighted with a red border (use drawContours with the outer marker contour) and the marker ID rendered near the marker centroid.

objective 1 output

Objective 2

Track these markers in live video and discuss the performance.

How does lighting affect tracking?
Do you get false positive marker detections if the video scene shows many arbitrary objects, shadows, etc.?

objective 2 output

Optional Experiments

Detect the angle of marker by calculating a vector from the centroid of the outer square to the centroid of the black dots in the centre. Note that I drew my markers with black dots offset to make this scheme reliable.
Think of more constraints to add to the marker detection such as the length of the four sides or the corner angles of the quadrilateral, the relative size of the inner white square, the relative position or size of the inner black dots, etc.

Submit

Follow the submission instructions on the course information page. Include your code and at least one frame capture from each objective that clearly shows everything was working. In your solution notes, write about any problems you ran into, and the main resource(s) you used (blog posts, online tutorials, stackoverflow posts, papers, textbooks, etc.). These resources should include brief descriptions of what the resource is and how it helped you. The writeup can be short, bullet points are recommended.

You must combine your code, screen captures, and notes together into a single PDF. The best way to do this is by doing everything in a single Jupyter notebook using markdown cells for your notes and inline images for your output. Then just export as PDF.