Black Spot Firmware
the marker in the next frame can be estimated. The ROI is centred on the new estimated position for the next frame. The marker centroid is sent to the hub during the tracking phase.
Initial marker detection
To enable markers to be tracked using ROIs, the approximate location of each marker needs to be determined so that it can be segmented from the background by the creation of a ROI. The algorithm designed for this initial detection looks for bright pixels in a captured image. If a pixel is above a specified threshold then it is flagged as belonging to a marker. ROI is centred around this pixel and the centroid of the region is calculated after the next frame is captured. The threshold is a hard-coded constant. For the shutter period of 300 µs used
during testing, an intensity value of 20 works well.
To reduce computational requirements, the algorithm is implemented using a grid. Only pixels on the corners of this grid are tested to determine whether they are bright pixels. For an 8 by 8 pixel grid, this reduces the number of pixels that must be checked by a factor of 64 and for a 16 by 16 pixel grid this is a factor of 256 times fewer pixels. This is a significant reduction in computation. During testing the grid was set to 16 pixels wide.
fter each frame the grid is repositioned. This is an attempt to ensure all markers are located when the camera is stationary. For example, if a marker falls between the pixels on the grid then it will not be detected while the camera is motionless. If the grid is moved so that after a number of frames every pixel has been sampled then at some point the marker that was initially invisible will be discovered. For the 16 pixels wide grid it will take at the very most 256 frames before every pixel has been checked for a new marker. Therefore, at 60 fps, the worst case is that it will take (16 × 16)/60 = 4.3 s to locate a marker assuming that it only covers 1 pixel. Based on the smallest marker image size at 3 m of approximately 2.5 by 2.5 pixels the grid can be moved by 2 pixels each time and still locate a marker of this size resulting in a further reduction in time of a factor of 4. This gives a figure of (16 × 16)/(2 × 2 × 60) = 1.1 s.
Figure 4.2 shows a screen-shot of the algorithm running on a PC. ROIs are shown as boxes with the centroid number inside the box. The grid is shown as the array of regularly spaced rectangles in the figure (these are enlarged for clarity). There are two further markers in the lower right hand corner, both of which coincide with grid pixels. The detection algorithm will detect these markers and ROIs will be created for them. The screen shot comes from a video stream that was created using a Point Grey Research FireflyMV  camera. This camera contains the same image sensor used by the Black Spot and was used to create test videos for the firmware before the Black Spot hardware was developed.