Image Registration

Image Registration is the process of realigning an image (the map) to a set coordinate system. If we are allowing the viewer to move the map we will need to use image registration to rotate and translate the image back to the original coordinates. Basically, we need to calculate the change in x the change in y and the change in angle θ which minimizes the change in pixel value from one frame to the next.

Here is a paper explaing Image Registration.

Two ImageJ plugins that might do this for us:
TurboReg: Well documented and I think the Rigid Body Transformation would accomplish what we need.
TurboReg also works with StackReg for processing all images in a stack.

bUnwarpJ: I think this would do what we want to only look into this one as a back-up idea for TurboReg and StackReg.

Update (5/20/09): Altering two lines of code in StackReg to keep the initial frame (the frame the algorithm was started on) as the target image for comparison was successful in transforming a video of a moving map to a static map. This has not been thoroughly tested but produced outstanding results on a B/W video with a moving map. The reason for the video being transformed to B/W was to limit the amount of computation during the tuning of the algorithm. Further tests are needed. Also, it is important that the video aspect ratio is maintained since videos squeezed from wide frame to 4:3 aspect ratio will transform a moving rectangle into a parallelogram and will not register properly using the Rigid Body Transformation. New StackReg code is saved in files as StackReg_.java.

New ImageJ plugin: StackReg_.java

Minimum Software Requirements: ImageJ and the TurboReg plugin.

Video Requirements: Avi format with original aspect ratio and at least one frame with only the map in the image (no body parts).

To run the program:

Install the new StackReg_ plugin.

Open an avi video in ImageJ.

If neccessary, advance the video until a frame with only the map is in the image, no body parts.

From the Plugin menu, choose StackReg.

In the pop-up box, choose "Rigid Body" and select "OK."

Extracting a moving image from a static background

For a static map situation, we might try to subtract the previous frame image from the current frame image. That will reduce most of the pixel values to very small values, and the ones around the moving object (hand) would be much larger as they are changing from hand to background or background to hand. Taking the absolute value (since some will be negative) and performing segmentation on those values (or using it as a mask) may greatly reduce our problem space.

Active contours

One really nice feature of the object we are looking for (a hand) is the symmetry (we roughly know the shape) and the significance within the image (it is not small relative to the image space). We can utilize these features with an active contours algorithm, sometimes referred to as snakes. ImageJ Has a plug-in for active contours which would make finding a hand, in an image really easy. [Plus I want to see active contours in use, so I admit it is a selfish suggestion.] On the plus side, this would eliminate the need for tape on a finger.

Here is a paper explaining active contours: Snakes:Active Contour Models. Look at figure 8 on page 9 for a relevent appllication.

K-means and C-means

K-means clustering was an early idea considered for image segmentation on this project. Due to the computational complexity alone, usually considered NP-hard, it would not be a good choice for processing streaming data such as video. Also, the number of clusters fed into the algorithm must be predetermined as there is no automatic way to calculate the number of clusters which would yield satisfactory results. The number of clusters could change between camera environments and therefore would need to be tuned each time the algorithm is used. For a non-technical user, this solution is unsatifactory. Another drawback is the way k-means clusters the data. Based on the Euclidean distance between RGB values, this is insufficient for good clustering of an area which is tight in physical distance. Since more weight should be added to the pixels in close proximity (physical distance) to the center, spacial-fuzzy-c-means becomes the better choice. Unfortunately, the computational complexity does not improve and becomes less attractive as a possible solution.