Programming: Java: Video Tracking: Color Recognition

[ Effect of Sobel Edge Detection Shown in Yellow ]

[ and Color Detection Shown in Blue ]

Intro:
I first experimented with Video Tracking about a year ago, at the time I was programming primarily in VB.net 2008. A powerful language yes, but when it comes to graphics and "time expensive" processes, its bad side begins to shine through.

About a week ago I decided to reopen this project and give it another go, this time in Java. My reasoning? Java runs a lot faster and is much better at handling color arrays, the basis for image recognition.

Summary of technique:
There are many methods for scanning video feeds, none of them perfect. They all have there downfalls. The key of a good tracking program is to use multiple of these methods together so that one methods downfall can be picked up by another ones strong point.

After much research, the types of image recognition I chose to develop upon were:

• Sobel Edge Detection
• Color Recognition
• Motion Detection

[ Effect of Color Detection Shown in Blue ]

Color Detection Algorithm:

Color Based Tracking is a method by which an image or video feed is scanned for a given color. Being the most common method of object recognition it has been interpreted in many different ways. After researching what the method actually needs to do I decided on a system by which the user clicks on the the part of the image where the color they want to track is, then the program uses this color reference in its scan.

To do this I had to write multiple systems into the program. First, a system by which to get the color when the user clicks on the image feed. Second, a system to display this color choice to the user along with the threshold for the scan. And thirdly, the SCAN!

The scan is what I'll focus on in this article. It consists of a system by which we loop through each X and Y pixel of the image and compare the GrayScale (Black and White) version of the image to the GrayScale version of the given color. We do this because it is faster than checking the Red, Green, and Blue Channels of every pixel in the image.

Once we have decided that a pixel is likely to contain the search color, we flag it by coloring it with a semi transparent blue pixel. Then we continue to scan to check if the Red Channels of the pixel and the Search Color are with the threshold of each other. If that checks out then we check the Green Channels, and finally, if that checks out we try the Blue Channels.

Once we have confirmed that the given pixel is the correct color we mark it with a solid blue mark.

Color Example Code: Java

public void Update(BufferedImage buf) {
        img = buf; //Store the image
        int w = img.getWidth(); //Image Width
        int h = img.getHeight(); //Image Height
       
        FlagMap= new BufferedImage(w,h,BufferedImage.TYPE_INT_ARGB); //Create Overlay
       
        for (int x = 0; x < img.getWidth(); x+=scan) { //Scan through each X of the image
            for (int y = 0; y < img.getHeight(); y+=scan) { //For each X scan through each Y of the image
               
                Color C = new Color(img.getRGB(x,y)); //Get the pixel color at the current position
                int GS = ((C.getRed() + C.getGreen() + C.getBlue()) /3); //Generate the GrayScale                                          
                //Color Detection
                if ((AvgT - ct) < GS && (AvgT + ct) > GS) { //Is the GrayScale is within threshold?
                    FlagMap.setRGB(x,y,FlagColor.getRGB()); //If so mark with a semi transparent blue mark
                  
                    //Is each color channel within threshold?
                    if ((TColor.getRed() - ct) < C.getRed() && (TColor.getRed() + ct) > C.getRed()) {
                        if ((TColor.getGreen() - ct) < C.getGreen() && (TColor.getGreen() + ct) > C.getGreen()) {
                            if ((TColor.getBlue() - ct) < C.getBlue() && (TColor.getBlue() + ct) > C.getBlue()) {
                                FlagMap.setRGB(x,y,DetectColor.getRGB()); //If R,G, and B Channels are with threshold mark with a solid blue mark
                            }
                        }
                    }
                }
                           
            }
        }

}

Notes on example code:

• The variable "scan" is an integer and refers to the accuracy of the scan. eg. scan = 1 would mean that every pixel of the image would be scanned. scan = 2 would mean every other pixel would be scanned. scan = 5 would mean every fifth pixel... ect, ect...
• The variable "ct" is an integer and refers to the Threshold for pixel comparison. This is set in the program via the Trackbar in the Top right of the Screen which allows for differences of 0 to 40.
• The variable "TColor" is a color and refers to the color the user wants the algorithm to scan.
• The variable "AvgT" is an integer and refers to the average of variable "TColor"'s R, G, and B Channels.
  
• My program has a separate class for displaying the image to the window, it draws the actual image first then draws what are called "overlays" on top of it. In this case the overlay is the image "FlagMap" which is transparent everywhere except where an edge has been detected.
• For best results, downsize your image to around 200x200 pixels before passing it to this algorithm and set "scan" to 1. If you want to scan a larger image you will need to set "scan" to a higher value to maintain performance.

I hope this helps anyone who is trying to do something in this field, it's a tricky one. Please subscribe, more like this is on the way!