OpenCV shape detection

shape_detection_results

This tutorial is the second post in our three part series on shape detection and analysis.

Last week we learned how to compute the center of a contour using OpenCV.

Today, we are going to leverage contour properties to actually label and identify shapes in an image, just like in the figure at the top of this post.

Looking for the source code to this post?
Jump right to the downloads section.

OpenCV shape detection

Before we get started with this tutorial, let’s quickly review our project structure:

As you can see, we have defined a pyimagesearch  module. Inside this module we have shapedetector.py  which will store our implementation of the ShapeDetector  class.

Finally, we have the detect_shapes.py  driver script that we’ll use to load an image from disk, analyze it for shapes, and then perform shape detection and identification via the ShapeDetector  class.

Before we get started, make sure you have the imutils package installed on your system, a series of OpenCV convenience functions that we’ll be using later in this tutorial:

Defining our shape detector

The first step in building our shape detector is to write some code to encapsulate the shape identification logic.

Let’s go ahead and define our ShapeDetector . Open up the shapedetector.py  file and insert the following code:

Line 4 starts the definition of our ShapeDetector  class. We’ll skip the __init__  constructor here since nothing needs to be initialized.

We then have our detect  method on Line 8 which requires only a single argument, c , the contour (i.e., outline) of the shape we are trying to identify.

In order to perform shape detection, we’ll be using contour approximation.

As the name suggests, contour approximation is an algorithm for reducing the number of points in a curve with a reduced set of points — thus the term approximation.

This algorithm is commonly known as the Ramer-Douglas-Peucker algorithm, or simply the split-and-merge algorithm.

Contour approximation is predicated on the assumption that a curve can be approximated by a series of short line segments. This leads to a resulting approximated curve that consists of a subset of points that were defined by the original cruve.

Contour approximation is actually already implemented in OpenCV via the cv2.approxPolyDP  method.

In order to perform contour approximation, we first compute the perimeter of the contour (Line 11), followed by constructing the actual contour approximation (Line 12).

Common values for the second parameter to cv2.approxPolyDP  are normally in the range of 1-5% of the original contour perimeter.

Note: Interested in a more in-depth look at contour approximation? Be sure to check out the PyImageSearch Gurus course where I discuss computer vision and image processing fundamentals such as contours and connected-component analysis in detail.

Given our approximated contour, we can move on to performing shape detection:

It’s important to understand that a contour consists of a list of verticesWe can check the number of entries in this list to determine the shape of an object.

For example, if the approximated contour has three vertices, then it must be a triangle (Lines 15 and 16).

If a contour has four vertices, then it must be either a square or a rectangle (Line 20). To determine which, we compute the aspect ratio of the shape, which is simply the width of the contour bounding box divided by the height (Lines 23 and 24). If the aspect ratio is ~1.0, then we are examining a square (since all sides have approximately equal length). Otherwise, the shape is a rectangle.

If a contour has five vertices, we can label it as a pentagon (Line 31 and 32).

Otherwise, by process of elimination (in context of this example, of course), we can make the assumption that the shape we are examining is a circle (Lines 35 and 36).

Finally, we return the identified shape to the calling method.

Shape detection with OpenCV

Now that our ShapeDetector  class has been defined, let’s create the detect_shapes.py  driver script:

We start off on Lines 2-5 by importing our required packages. Notice how we’re importing our implementation of the ShapeDetector  class from the shapedetector  sub-module of pyimagesearch .

Lines 8-11 handle parsing our command line arguments. We only need a single switch here, --image , which is the path to where the image we want to process resides on disk.

Next up, let’s pre-process our image:

First, we load our image from disk on Line 15 and resize it on Line 16. We then keep track of the ratio  of the old height to the new resized height on Line 17 — we’ll find out exactly why we do this later in the tutorial.

From there, Lines 21-23 handle converting the resized image to grayscale, smoothing it to reduce high frequency noise, and finally thresholding it to reveal the shapes in the image.

After thresholding, our image should look like this:

Figure 1: Thresholding reveals the shapes in our image.

Figure 1: Thresholding reveals the shapes in our image.

Notice how our image has been binarized — the shapes appear as a white foreground against a black background.

Lastly, we find contours in our binary image, handle grabbing the correct tuple value from cv2.findContours  based on our OpenCV version, and finally initialize our ShapeDetector  (Lines 27-30).

The last step is to identify each of the contours:

On Line 33 we start looping over each of the individual contours. For each of them, we compute the center of the contour, followed by performing shape detection and labeling.

Since we are processing the contours extracted from the resized image (rather than the original image), we need to multiply the contours and center (x, y)-coordinates by our resize ratio  (Lines 43-45). This will give us the correct (x, y)-coordinates for both the contours and centroid of the original image.

Lastly, we draw the contours and the labeled shape on our image (Lines 44-48), followed by displaying our results (Lines 51 and 52).

To see our shape detector in action, just execute the following command:

Figure 2: Performing shape detection with OpenCV.

Figure 2: Performing shape detection with OpenCV.

As you can see from the animation above, our script loops over each of the shapes individually, performs shape detection on each one, and then draws the name of the shape on the object.

Summary

In today’s post blog, we learned how to perform shape detection with OpenCV and Python.

To accomplish this, we leveraged contour approximation, the process of reducing the number of points on a curve to a more simple approximated version.

Then, based on this contour approximation, we examined the number of vertices each shape has. Given the vertex count, we were able to accurately label each of the shapes.

This lesson is part of a three part series on shape detection and analysis. Last week we covered how to compute the center of a contour. Today we covered shape detection with OpenCV. And next week we’ll discuss how to label the actual color of a shape using color channel statistics.

Be sure to enter your email address in the form below to be notified when the next post goes live — you won’t want to miss it!

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 11-page Resource Guide on Computer Vision and Image Search Engines, including exclusive techniques that I don’t post on this blog! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , ,

61 Responses to OpenCV shape detection

  1. leena February 9, 2016 at 5:59 am #

    Why it is scanning and labeling from bottom to top?

    How to to scan and label top to bottom?

    • Adrian Rosebrock February 9, 2016 at 3:54 pm #

      That is how the cv2.findContours method is implemented. If you would like to sort contours, see this post.

      • leena February 17, 2016 at 4:57 am #

        Thanks Adrian. It worked and I am able to scan from top to bottom.

        Please help me in identifying lines connected with two shapes like in a flowchart, 2 boxes are connected with line/arrow

        with regards.

  2. leena February 9, 2016 at 6:11 am #

    I have done the same with

    shape factor= area / (peri * peri)

    if shapefactor >= 0.06 and shapefactor = 0.0484 and shapefactor = 0.050 and shapefactor = 0.032 and shapefactor = 0.95 and ar <= 1.05 else "rectangle"

    • Adrian Rosebrock February 9, 2016 at 3:55 pm #

      Is there a particular reason you are taking the ratio of the area to the perimeter squared? It seems to make the rule more complicated.

      • leena February 10, 2016 at 11:11 pm #

        Actually I do not know the reason, just it got solved my problem, so I took it. You or somebody can help me understanding this and the better solution .
        Thanks

        • bitflip June 21, 2016 at 11:30 am #

          Given you have following triangle:

          The bounding rect of it would have aspect-ratio about 1:1.
          So, better take the area() of the contour and compare it to width*height of the bounding rect. when the error is too hight –> rectangle.

  3. Vincent February 18, 2016 at 3:41 pm #

    Hi Adrian,

    First and foremost, thank you for this excellent tutorial. Very useful and informative.

    I have used the logic here to detect red triangles in a webcam feed. I have also tweaked the shapedetector class to identify only triangles. I am able to successfully identify a red triangle.
    http://imgur.com/6Z9CnBA

    I’ve noticed that sometimes a very messy contour will be classified incorrectly as a triangle.
    http://imgur.com/4a06psM

    What would be a good way to tweak this?
    http://pastie.org/10727912
    http://pastie.org/10727915

    • Adrian Rosebrock February 18, 2016 at 4:05 pm #

      Keep in mind that the code is only as good as the images that you put into it. The code detailed in this post assumes simple shapes that can be recognized utilizing contour properties. For more advanced shapes, or shapes that have substantial variances in how they appear (such as noisy contours), you might need to train your own custom object detector.

      Anyway, the reason sometimes even messy contours get classified differently is due to the contour approximation. Play around with the percentage used in cv2.approxPolyDP and you’ll be able to see the differences.

  4. Peng March 4, 2016 at 7:49 pm #

    Hi Adrian,

    Thank you for making this. A little feedback on the image file.

    I notice that if using a .jpg file as the source, the moment(cnt) will not get a correct value.

    It report an error :

    cntX = int(M[“m10”] / M[“m00”])
    ZeroDivisionError: float division by zero

    Any ideas on this?

    Thanks

    • Adrian Rosebrock March 6, 2016 at 9:20 am #

      Version version of OpenCV and Python are you using?

      In either case, you can resolve the issue by doing:

      This if statement will take care of the divide by zero error.

      Alternatively, you can add a tiny value to the moment to ensure there is no divide by zero problem:

      cX = (M["c10"] / (M["m00"] + 1e-7))

  5. Euan March 10, 2016 at 10:00 pm #

    Hi Adrian,

    Firstly thanks for a great tutorial and site. I’m a mechanical engineer and noob to openCV, python and linux and have managed to get openCV 3.0 and python2.7 setup thanks to your tutorial here…. http://www.pyimagesearch.com/2015/06/22/install-opencv-3-0-and-python-2-7-on-ubuntu/

    In order to get this code running on my setup, I needed to modify “float” on line 17 to “int” as it was causing cast problems on line 43 “c *=ratio”. I believe this is probably due to an update of how python works from how it worked when you wrote this tutorial. Is this the case?

    • Adrian Rosebrock March 13, 2016 at 10:29 am #

      Interesting, Brandon mentioned this issue in a comment above. Which version of OpenCV and NumPy are you using?

  6. brandon March 11, 2016 at 4:20 pm #

    Adrian, great stuff. I’ve learned a lot from your blog, and plan on purchasing the book soon. I was working through this post for now, and I’m getting the following traceback error:

    • Adrian Rosebrock March 13, 2016 at 10:21 am #

      I personally haven’t seen this error message before. Can you let me know which OpenCV and NumPy versions you are using?

      • Brandon March 14, 2016 at 12:28 pm #

        numpy is 1.10.4, and it happens with both OpenCV 2.4.12 and 3.1.0 (in a virtualenv, thanks to another of your tutorials) under Python 2.7 on OS X 10.11.3

        The workaround was simply to adjust data types pre and post multiplication:

        c=c.astype(np.float_)
        c *= ratio
        c=c.astype(np.int32)

        It works now.

        • Adrian Rosebrock March 14, 2016 at 3:15 pm #

          Thanks for the tip Brandon! I’ll be sure to dive into this more. I’m using NumPy 1.9.3, so perhaps this is an issue with NumPy 1.10.X.

  7. Ahmed Abdeldaim March 24, 2016 at 10:30 am #

    Great work Mr. Adrian
    but is there a way to make the selection more softer, for example by reduce point size ??
    or this is the best result??

    • Adrian Rosebrock March 24, 2016 at 5:10 pm #

      Absolutely, you just need to apply contour approximation first. I detail contour approximation in a few blog posts, but I would start with this one.

      • Ahmed Abdeldaim March 26, 2016 at 4:05 pm #

        Thanks for your help.

  8. darshan March 26, 2016 at 2:09 am #

    how to install imutils module
    I used pip install imutils I’m getting error

    • Adrian Rosebrock March 27, 2016 at 9:14 am #

      Please see the “OpenCV shape detection” section of this blog post. You just need to use pip

      $ pip install imutils

  9. firoz khan April 23, 2016 at 10:42 am #

    hi adrian it is only detcting one pentagon and nothing else

    • Adrian Rosebrock April 25, 2016 at 2:09 pm #

      Make sure you click on the window and press any key on your keyboard — this will advance the script. Right now a keypress is required after each detection.

  10. Diego Fernando Barrios April 29, 2016 at 5:08 pm #

    Good afthernoon!

    Thanks very much for this tutotrial, you´re doing a great and util work.

    Friend, I have a problem with contour detection, when I change the image, the project don’t work (I’m using a black background, I take the image from USB camera). I not have problem with the image path.

    The python scripts should recognize (9 “nine” rectangles”) but just one is recognized

    Sorry for the writing, my english is not so good.

    I would like that you can help me. I’m working in my work grade.

    Thanks very much!

    • Adrian Rosebrock April 30, 2016 at 3:55 pm #

      Depending on your image, this could be an issue with segmentation and/or the contours. Make sure that after thresholding your 9 rectangles have been clearly segmented. From there, move on to the contour approximation and see how many points are returned by the approximation. You might need to tweak the cv2.approxPolyDP parameters.

  11. itai May 8, 2016 at 4:24 am #

    Hey Adrian,
    I was wondering why did you use the Ramer-Douglas-Peucker algorithm to reduce the set of points, instead of using the convex hull ?

    Thanks

    • Adrian Rosebrock May 8, 2016 at 8:12 am #

      The contour approximation algorithm and Convex Hull algorithm are used for two separate purposes. As the name implies, contour approximation is used to reduce the number of points along a contour by “simplifying” the contour based on a percentage of the perimeter. Your resulting contour approximation is this a simplification of the shape by utilizing points that are already part of the shape.

      The convex hull on the other hand is the smallest convex set that contains all points along the contour — it is, by definition, not a simplification of the contour shape and the resulting convex hull actually contains points that are not part of the original shape.

      In this case, I used contour approximation because I wanted to reduce the number of (x, y)-coordinates that comprise the contour, while ensuring that all points in the resulting approximation were also part of the original shape.

  12. Armin June 30, 2016 at 2:56 pm #

    Hello Adrian

    thanks for tutorial

    I want to show detected shapes in seperate windows( each shape on each window), what should I do?
    also tried cropping them (using ROI) but I didn’t able to work it out.

    tnx

    • Adrian Rosebrock July 1, 2016 at 2:59 pm #

      Hey Armin — you’re on the right track. You should be applying array slicing to extract the ROI, then using the cv2.imshow function on each ROI. An example of ROI slicing can be found in this blog post as well as Practical Python and OpenCV.

  13. Alex Hopper July 24, 2016 at 5:04 pm #

    Hello,
    I’m new openCV-Python user.. I have a question about it:
    I have a database containing pre-processed images by kinect, and I need use Deep Learning to analyse these images. Is there a best way to start?

    http://www.pyimagesearch.com/2016/02/08/opencv-shape-detection/

    Can I use it to start?

    Thanks.

    • Adrian Rosebrock July 27, 2016 at 2:36 pm #

      What type of images are you working with? You mentioned they were pre-processed by the Kinect. Are they depth images? RGB images?

  14. Neal July 27, 2016 at 8:58 am #

    hi

    I’m looking for advice in shape detection. I want to use a camera to detect different kinds of shapes on a microcontroller. What would be the best method to approach this?

    Your help will be much appreciated.
    Thanks

    • Adrian Rosebrock July 27, 2016 at 1:55 pm #

      Hey Neal — to start, you need to segment each of the components of the microcontroller. Do you have any example images that you’re working with?

      • Neal August 2, 2016 at 8:23 am #

        well i’m going to be using different shape cut outs of wooden blocks as my different objects.

        • Adrian Rosebrock August 2, 2016 at 2:57 pm #

          I would use a similar approach as detailed in this blog post. Cut out your wooden blocks and place the camera such that its “looking down” on the blocks. Apply thresholding or edge detection to find the blocks. And from there, you can use the contour approximation technique to label the shapes.

  15. Anupam September 4, 2016 at 3:40 am #

    Can you please help me out with detecting overlapping shapes ?

  16. Leena October 6, 2016 at 8:25 am #

    How can we use the shapedetector to classify polygon as rectangle/diamond(decision box)/ parallelogram….
    please help

  17. Poehe October 31, 2016 at 11:09 am #

    Hi Adrian, thank you so much for the tutorial, it’s a great starting point for me to dive into OpenCV.

    I noticed when processing images using your code that in pictures with a white background the engine also shows the contours/edges of the whole input picture as being a shape, while in images with a black background (as in your example) the engine ignores the outside contours of the whole input pic and only shows the contours of the objects within the input pic itself (which is the way it should work, I suppose).

    Could you think of a solution that makes the engine not classify the picture edges as contours?

    Your help will be much appreciated!

    Cheers

    • Adrian Rosebrock November 1, 2016 at 8:59 am #

      So if I understand your question correctly, you are using a background that is lighter than the shapes themselves? And after thresholding your shapes appear as “black” on a “white” background? Am I understanding that correctly? If so, simply invert the threshold step to make the shapes “white” on a “black” background.

  18. Megha Maheshwari November 21, 2016 at 12:42 am #

    Hi Adrian

    How can we differentiate between rectangle and trapezoid. I have detected that the polygon contains 4 corners and hence is either a square, rectangle or trapezoid as per my image. However, for square can easily check the width and height, but how do i differentiate between rectangle and trapezoid.

    • Adrian Rosebrock November 21, 2016 at 12:27 pm #

      There are many ways to do this. I would consider computing the extent (contour area / bounding box area). A perfect rectangle will have an extent of near 1.0 while trapezoid will have an extent much less than 1.0. You can also compute the angle between each corner of the shape. A rectangle would have near perfect 90 degree angles. Either one will work.

  19. Mohamad November 23, 2016 at 8:12 am #

    Hi Great Man. Mr. Adrian
    I guess your offer in this tutorial use the webcam for raspberry pi. its Ok? other question is that for detect other shapes such as (H) similar or (L) similar or … How doing it? Is this method work with edge or point detection?

    • Adrian Rosebrock November 23, 2016 at 8:30 am #

      You can certainly use a webcam or Raspberry Pi camera module to perform shape detection. You would just need to read the frame from the camera and process it. I provide tutorials on how to access webcams here.

      As for detecting an “H” or “L” you can do that using contour properties (extent, solidity, etc.), template matching, or image descriptors such as Histogram of Oriented Gradients. I would suggest taking a look at Practical Python and OpenCV along with the PyImageSearch Gurus for more advanced demonstrates of recognizing objects in images.

  20. Luís Serrador November 25, 2016 at 5:48 am #

    Hi Adrian,

    I tried two of your tutorials (this one and ‘OpenCV center of contour’) and when I execute the command to run the .py file my result is not the same as you show at the end of the tutorials. My final image only recognize the first shape/center, and doesn’t recognize more shapes. Is there anything that could be wrong?

    • Adrian Rosebrock November 28, 2016 at 10:41 am #

      Hi Luís — what versions of OpenCV and Python are you using?

      • Luís Serrador November 28, 2016 at 11:34 am #

        Hi Adrian! I’m using OpenCV 3.0 and Python 2.7. Any mistake of mine?

        • Adrian Rosebrock November 28, 2016 at 2:42 pm #

          Nope, the Python and OpenCV version shouldn’t be an issue. I was just curious about the setup to confirm it wasn’t an outlier situation. I personally haven’t ran into this problem when executing the code. I assume you downloaded the code using the “Downloads” section of this post rather than copying and pasting the code? Sometimes that causes problems with reader’s code as well.

  21. Jan December 4, 2016 at 1:08 am #

    Hi Adrian,
    Thanks for the tutorial,
    I did the same, but for certain cicrles the vertices were shown as 4 and hence were displayed as squares , can you suggest a way to increase the number of detected vertices in the picture.
    Thanks

    • Adrian Rosebrock December 5, 2016 at 1:31 pm #

      You’ll want to play with the following line:

      approx = cv2.approxPolyDP(c, 0.04 * peri, True)

      The smaller the value passed in for peri, the more vertices you’ll obtain.

      • Preethi December 23, 2016 at 6:16 am #

        Hi Adrian,
        From Shape detection i should detect circle alone. even though rectangle,square and etc present. In your Same example i need this modification

        • Adrian Rosebrock December 23, 2016 at 10:50 am #

          For circle detection, take a look at this blog post. Otherwise, you can use this code to determine a circle as well. A circle will have many more approximated contour vertices than all other shapes. You basically need to create an if statement for this.

  22. Niel January 5, 2017 at 12:52 am #

    Hi, Adrian i’m mechatronic students and now i build an obstacle avoidance robot i like to ask how can i create a detection square box in OpenCV for detect colour so it will give feedback to th robot to move or to stop

    • Adrian Rosebrock January 7, 2017 at 9:41 am #

      Hey Niel — can you elaborate on what you mean by a “detection square box”? I’m not sure what you mean.

  23. Chandu January 18, 2017 at 1:23 am #

    Hi Adrian

    i’m getting an error

    Traceback (most recent call last):
    File “”, line 2, in
    from pyimagesearch.shapedetector import ShapeDetector
    ImportError: No module named pyimagesearch.shapedetector

    • Adrian Rosebrock January 18, 2017 at 7:08 am #

      Hey Chandu — make sure you download the source code to this blog post using the “Downloads” section. It’s likely that your project directory structure does not match mine (perhaps missing a __init__.py file. Please download my code and compare it to yours.

  24. Arturo January 19, 2017 at 9:48 pm #

    Hi, i´m having a problem with the code, Sorry for my ignorance but where do I have to put the path of the image?
    I can´t understand the lines 9-12 where you say that we have to place the path, the (– image) part

  25. tal January 25, 2017 at 12:14 pm #

    Hi, i´m having a problem with the code, i have insert the path into the
    ap.add_argument(“-i”, “–image”, required=True, help=”rec.jpg”)
    and i’m getting this error.

    usage: detect_shapes.py [-h] -i IMAGE
    detect_shapes.py: error: argument -i/–image is required.
    thanks alot

    • Adrian Rosebrock January 26, 2017 at 8:21 am #

      You do not have to modify the code at all. You just need to supply the --image switch to the Python script via command line argument:

      $ python detect_shapes.py --image shapes_and_colors.png

      Please read up on command line arguments before continuing.

Leave a Reply