Databases
StackOverflow2013
Postsdbo
POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
    primarykey
    data
    text
    <p>One of the most interesting projects I've worked on in the past couple of years was a project about <a href="https://en.wikipedia.org/wiki/Image_processing" rel="noreferrer">image processing</a>. The goal was to develop a system to be able to recognize Coca-Cola <strong>'cans'</strong> (note that I'm stressing the word 'cans', you'll see why in a minute). You can see a sample below, with the can recognized in the <em>green rectangle</em> with scale and rotation.</p> <p><img src="https://i.stack.imgur.com/irQtR.png" alt="Template matching"></p> <p>Some constraints on the project:</p> <ul> <li>The background could be very noisy.</li> <li>The <em>can</em> could have any <em>scale</em> or <em>rotation</em> or even orientation (within reasonable limits).</li> <li>The image could have some degree of fuzziness (contours might not be entirely straight).</li> <li>There could be Coca-Cola bottles in the image, and the algorithm should only detect the <em>can</em>!</li> <li>The brightness of the image could vary a lot (so you can't rely "too much" on color detection).</li> <li>The <em>can</em> could be partly hidden on the sides or the middle and possibly partly hidden behind a bottle.</li> <li>There could be no <em>can</em> at all in the image, in which case you had to find nothing and write a message saying so.</li> </ul> <p>So you could end up with tricky things like this (which in this case had my algorithm totally fail):</p> <p><img src="https://i.stack.imgur.com/Byw82.png" alt="Total fail"></p> <p>I did this project a while ago, and had a lot of fun doing it, and I had a decent implementation. Here are some details about my implementation:</p> <p><strong>Language</strong>: Done in C++ using <a href="http://opencv.org" rel="noreferrer">OpenCV</a> library.</p> <p><strong>Pre-processing</strong>: For the image pre-processing, i.e. transforming the image into a more raw form to give to the algorithm, I used 2 methods:</p> <ol> <li>Changing color domain from RGB to <a href="http://en.wikipedia.org/wiki/HSL_and_HSV" rel="noreferrer">HSV</a> and filtering based on "red" hue, saturation above a certain threshold to avoid orange-like colors, and filtering of low value to avoid dark tones. The end result was a binary black and white image, where all white pixels would represent the pixels that match this threshold. Obviously there is still a lot of crap in the image, but this reduces the number of dimensions you have to work with. <img src="https://i.stack.imgur.com/ktdAB.png" alt="Binarized image"> </li> <li>Noise filtering using median filtering (taking the median pixel value of all neighbors and replace the pixel by this value) to reduce noise.</li> <li>Using <a href="http://en.wikipedia.org/wiki/Canny_edge_detector" rel="noreferrer">Canny Edge Detection Filter</a> to get the contours of all items after 2 precedent steps. <img src="https://i.stack.imgur.com/F9319.png" alt="Contour detection"></li> </ol> <p><strong>Algorithm</strong>: The algorithm itself I chose for this task was taken from <a href="http://rads.stackoverflow.com/amzn/click/0123725380" rel="noreferrer">this</a> awesome book on feature extraction and called <a href="http://en.wikipedia.org/wiki/Generalised_Hough_transform" rel="noreferrer">Generalized Hough Transform</a> (pretty different from the regular Hough Transform). It basically says a few things:</p> <ul> <li>You can describe an object in space without knowing its analytical equation (which is the case here).</li> <li>It is resistant to image deformations such as scaling and rotation, as it will basically test your image for every combination of scale factor and rotation factor.</li> <li>It uses a base model (a template) that the algorithm will "learn".</li> <li>Each pixel remaining in the contour image will vote for another pixel which will supposedly be the center (in terms of gravity) of your object, based on what it learned from the model.</li> </ul> <p>In the end, you end up with a heat map of the votes, for example here all the pixels of the contour of the can will vote for its gravitational center, so you'll have a lot of votes in the same pixel corresponding to the center, and will see a peak in the heat map as below:</p> <p><img src="https://i.stack.imgur.com/wxrT1.png" alt="GHT"></p> <p>Once you have that, a simple threshold-based heuristic can give you the location of the center pixel, from which you can derive the scale and rotation and then plot your little rectangle around it (final scale and rotation factor will obviously be relative to your original template). In theory at least...</p> <p><strong>Results</strong>: Now, while this approach worked in the basic cases, it was severely lacking in some areas:</p> <ul> <li>It is <strong>extremely slow</strong>! I'm not stressing this enough. Almost a full day was needed to process the 30 test images, obviously because I had a very high scaling factor for rotation and translation, since some of the cans were very small.</li> <li>It was completely lost when bottles were in the image, and for some reason almost always found the bottle instead of the can (perhaps because bottles were bigger, thus had more pixels, thus more votes)</li> <li>Fuzzy images were also no good, since the votes ended up in pixel at random locations around the center, thus ending with a very noisy heat map.</li> <li>In-variance in translation and rotation was achieved, but not in orientation, meaning that a can that was not directly facing the camera objective wasn't recognized.</li> </ul> <p>Can you help me improve my <strong>specific</strong> algorithm, using <strong>exclusively OpenCV</strong> features, to resolve the <strong>four specific</strong> issues mentioned?</p> <p>I hope some people will also learn something out of it as well, after all I think not only people who ask questions should learn. :)</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USCharles Menguy
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
      1. USbtown
      1. VTFavorite
    3. VO
      singulars
      1. POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
      1. USzengr
      1. VTFavorite
    1. COIt might be said that this question is more appropriate at dsp.stackexchange.com, or stats.stackexchange.com, and you certainly should consider re-asking at those sites too.
      singulars
      1. POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
      1. USely
    2. COThe first thing to do here is to analyze *why* the different failure cases are happening. E.g., isolate examples of places where bottles win, where the images are fuzzy, etc., and perform some statistical analysis to learn the difference between their Hough representations and the ones you wish it would detect. Some great places to learn about alternative approaches are [here](http://www.svcl.ucsd.edu/projects/discriminantsaliency/) and [here](http://www.eecs.berkeley.edu/~lbourdev/poselets/)
      singulars
      1. POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
      1. USely
    3. CO@linker Wouldn't be extracting SIFT or SURF features much faster than the hough-transformation ? Why only detect cans when you could detect more registered objects?
      singulars
      1. POImage Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition
      1. USstacker
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload