Zero-parameter, automatic Canny edge detection with Python and OpenCV

Figure 2: Applying automatic Canny edge detection to a picture of a camera. Left: Wide Canny edge threshold. Center: Tight Canny edge threshold. Right: Automatic Canny edge threshold.

Today I’ve got a little trick for you, straight out of the PyImageSearch vault.

This trick is really awesome — and in many cases, it completely alleviates the need to tune the parameters to your Canny edge detectors. But before we get into that, let’s discuss the Canny edge detector a bit.

Looking for the source code to this post?
Jump right to the downloads section.

OpenCV and Python versions:
This example will run on Python 2.7/Python 3.4+ and OpenCV 2.4.X/OpenCV 3.0+.

The Canny Edge Detector

In previous posts we’ve used the Canny edge detector a fair amount of times. We’ve used it to build a kick-ass mobile document scanner and we’ve used to find a Game Boy screen in a photo, just two name a couple instances.

The Canny edge detector was developed way back in 1986 by John F. Canny. And it’s still widely used today was one of the default edge detectors in image processing.

The Canny edge detection algorithm can be broken down into 5 steps:

  • Step 1: Smooth the image using a Gaussian filter to remove high frequency noise.
  • Step 2: Compute the gradient intensity representations of the image.
  • Step 3: Apply non-maximum suppression to remove “false” responses to to edge detection.
  • Step 4: Apply thresholding using a lower and upper boundary on the gradient values.
  • Step 5: Track edges using hysteresis by suppressing weak edges that are not connected to strong edges.

If you’re familiar with the OpenCV implementation of the Canny edge detector you’ll know that the function signature looks like this:

cv2.canny(image, lower, upper)

Where image  is the image that we want to detect edges in; and lower  and upper  are our integer thresholds for Step 4, respectively.

The problem becomes determining these lower and upper thresholds.

What is the optimal value for the thresholds?

This question is especially important when you are processing multiple images with different contents captured under varying lighting conditions.

In the remainder of this blog post I’ll show you a little trick that relies on basic statistics that you can apply to remove the manual tuning of the thresholds to Canny edge detection.

This trick will save you time parameter tuning — and you’ll still get a nice Canny edge map after applying the function.

To learn more about this zero-parameter, automatic Canny edge detection trick, read on.

Zero-parameter, automatic Canny edge detection with Python and OpenCV

Let’s go ahead and get started. Open up a new file in your favorite code editor, name it auto_canny.py , and let’s get started:

The first thing we’ll do is import our necessary packages. We’ll use NumPy to for numerical operations, argparse  to parse command line arguments, glob  to grab the paths to our images from disk, and cv2  for our OpenCV bindings.

We then define auto_canny , our automatic Canny edge detection function on Line 7. This function requires a single argument, image , which is the single-channel image that we want to detect images in. An optional argument, sigma  can be used to vary the percentage thresholds that are determined based on simple statistics.

Line 9 handles computing the median of the pixel intensities in the image.

We then take this median value and construct two thresholds, lower  and upper  on Lines 12 and 13. These thresholds are constructed based on the +/- percentages controlled by the sigma  argument.

A lower value of sigma  indicates a tighter threshold, whereas a larger value of sigma  gives a wider threshold. In general, you will not have to change this sigma  value often. Simply select a single, default sigma  value and apply it to your entire dataset of images.

Note: In practice, sigma=0.33  tends to give good results on most of the dataset I’m working with, so I choose to supply 33% as the default sigma value.

Now that we have our lower and upper thresholds, we then apply the Canny edge detector on Line 14 and return it to the calling function on Line 17.

Let’s keep moving with this example and see how we can apply it to our images:

We parse our command line arguments on Lines 20-23. We only need a single switch here, --images , which is the path to the directory containing the images we want to process.

We then loop over the images in our directory on Line 26, load the image from disk on Line 28, convert the image to grayscale on Line 29, and apply a Gaussian blur with a 3 x 3 kernel to help remove high frequency noise on Line 30.

Lines 34-36 then apply Canny edge detection using three methods:

  1. wide threshold.
  2. A tight threshold.
  3. A threshold determined automatically using our auto_canny  function.

Finally, our resulting images are displayed to us on Lines 39-41.

The auto_canny function in action

Alright, enough talk about code. Let’s see our auto_canny  function in action.

Open up a terminal and execute the following command:

You should then see the following output:

Figure 1: Applying automatic Canny edge detection. Left: Wide Canny edge threshold. Center: Tight Canny edge threshold. Right: Automatic Canny edge threshold.

Figure 1: Applying automatic Canny edge detection. Left: Wide Canny edge threshold. Center: Tight Canny edge threshold. Right: Automatic Canny edge threshold.

As you can see, the wide Canny edge threshold not only detects the dolphin, but also many of the clouds in the image. The tight threshold does not detect the clouds, but misses out on the dolphin tail. Finally, the automatic method is able to find all of the dolphin, while removing many of the cloud edges.

Let’s try another image:

Figure 2: Applying automatic Canny edge detection to a picture of a camera. Left: Wide Canny edge threshold. Center: Tight Canny edge threshold. Right: Automatic Canny edge threshold.

Figure 2: Applying automatic Canny edge detection to a picture of a camera. Left: Wide Canny edge threshold. Center: Tight Canny edge threshold. Right: Automatic Canny edge threshold.

The wide Canny threshold on the left includes high frequency noise based on the reflection of the light on the brushed metal of the camera, whereas the tight threshold in the center misses out on many of the structural edges on the camera. Finally, the automatic method on the right is able to find many of the structural edges while not including the high frequency noise.

One more example:

Figure 3: Applying automatic Canny edge detection to a picture of a cup. Left: Wide Canny edge threshold. Center: Tight Canny edge threshold. Right: Automatic Canny edge threshold.

Figure 3: Applying automatic Canny edge detection to a picture of a cup. Left: Wide Canny edge threshold. Center: Tight Canny edge threshold. Right: Automatic Canny edge threshold.

The results here are fairly dramatic. While both the wide (left) and the automatic (right) Canny edge detection methods perform similarly, the tight threshold (center) misses out on almost all of the structural edges of the cup.

Given the examples above, it’s clear that the automatic, zero-parameter version of the Canny edge detection obtains the best results with the least effort.

Note: The three example images were taken from the CALTECH-101 dataset.

Summary

In this blog post I showed you a simple trick to (reliably) automatically detect edges in images using the Canny edge detector, without providing thresholds to the function.

This trick simply takes the median of the image, and then constructs upper and lower thresholds based on a percentage of this median. In practice, sigma=0.33  tends to obtain good results.

In general, you’ll find that the automatic, zero-parameter version of the Canny edge detection is able to obtain fairly decent results with little-to-no effort on your part.

With this in mind, why not download the source code to this post and give it a shot on your own images? I would be curious to hear about your results!

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 11-page Resource Guide on Computer Vision and Image Search Engines, including exclusive techniques that I don’t post on this blog! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , ,

63 Responses to Zero-parameter, automatic Canny edge detection with Python and OpenCV

  1. Ryan Jansekok April 6, 2015 at 11:00 pm #

    Hey I just wanted say this is a tremendous post! I’ve implemented something similar to this and it really streamlines processing bulk images.

    Also after reading the line:

    ‘np.hstack([wide, tight, auto])’

    I laughed out load because of how simple/awesome this method is for joining images. Do you have any other awesome ‘cheats’ like that one?
    Thanks again for a quality blog!

    • Adrian Rosebrock April 7, 2015 at 7:23 am #

      Hi Ryan, thanks so much for the comment and kind words, I really appreciate it 😀 I do have some other little cheats like these scattered across the blog. At some point I should really compile them all into a single post.

  2. Michele April 7, 2015 at 2:09 pm #

    You can find something similar here:https://gist.github.com/egonSchiele/756833

    Bests

    • Adrian Rosebrock April 7, 2015 at 3:11 pm #

      Nice, thanks for sharing!

  3. Yves Daoust April 8, 2015 at 2:21 am #

    Thank you for this post. I fully agree with a method that relies on statistics of the gradient intensity and I am sure yours gives convincing results.

    Interestingly, the ratio of the thresholds you chose is precisely 2, the value that is usually recommended for hysteresis thresholding. Adjusting a single threshold is not always that easy; adjusting two at a time is a challenge.

    I know from experience that textured/noisy areas do pollute the histogram of intensities and can displace the median (think of enlarging a textured area indefinitely). Using the histogram of only those pixels that survive after non-maxima suppression does not fix this, as in textured areas a significant fraction of the pixels do survive. I a still looking for a robust statistic.

    • Adrian Rosebrock April 8, 2015 at 6:19 am #

      Hi Yves, thanks so much for the reply! And yes, I absolutely agree — this is not a “magic bullet” approach. But it does obtain good results (in many cases) with literally no effort. And in the case of textured areas, you’ll probably struggle to tune the Canny thresholds manually as well. Have you tried applying a series of bilateral filters to your images to smooth the texture while still preserving the lines? This could help in your situation.

    • Dislaire January 31, 2017 at 4:19 am #

      Hey Yves 😉
      I would prefer to take the part of the histogram of interest (in images with background this one has not to be taken in consideration) and use the percentile 5 and 95 as the limit of the distribution.
      The Threshold High would be (P95+P5) /2 + 0.33 * (P95-P5) /2

  4. Yves Daoust April 8, 2015 at 5:44 pm #

    Indeed, the bilateral filter does a very good job (much better than the median); if I could I would use the non-local means filter, but for its horrible running time. My situation is everybody’s situation, isn’t it: in more than 25 years of practice, I doubt I have ever seen a clean image where edge detection really works 😉

    On second thoughts, noise and flat/textured zones are largely dominant in an image (otherwise the image is just a mess), so that edges can be seen as “outlier” pixels. So it could be that the goal of automatic threshold computation is to just capture the distribution of the noise. This is a partly hopeless task as the noise can be correlated to image intensities and textures are location-dependent, and the threshold should be adaptive.

    • Adrian Rosebrock April 8, 2015 at 6:51 pm #

      It sounds like you might want to take a look at Local Binary Patterns. You should be able to capture the distribution of edge, flat, and corner regions with a little bit of trial and error.

  5. Tyler April 11, 2015 at 9:04 am #

    Is it possible to get the output from edge detection as a series of line functions? I built a v-plotter and would like to draw the results of the edge detection. Thx.

    • Adrian Rosebrock April 11, 2015 at 10:30 am #

      To get the edge map as a series of lines, you would probably have to use something like cv2.findContours and then fit a line to each contour.

  6. vin May 8, 2015 at 11:48 pm #

    hi adrian,

    great post, as usual! did you consider using an automatically calculated threshold (eg otsu)?

    • Adrian Rosebrock May 9, 2015 at 7:51 am #

      Otsu’s method is really awesome, but assumes a bi-modal histogram to get decent threshold results. This method does not assume any knowledge of the distribution and instead uses the median, which is less sensitive to outliers than other statistical methods, to derive the edge thresholds. In practice, it works quite well.

  7. Bosmart September 14, 2015 at 12:53 pm #

    The way I see it, you have exchanged two parameters for a new one + heuristic. So “zero” is a strong word here… 🙂

    • Adrian Rosebrock September 15, 2015 at 6:04 am #

      I have exchanged two parameters that have to be manually set for an optional one that works well on natural images. Yes, it’s still technically a parameter, but it’s an optional one. The real power of auto_canny is when you go to apply it to a dataset of images, allowing you to obtain a consistent edge map without hardcode parameters.

  8. Tim Clemans October 10, 2015 at 3:50 pm #

    Is there a way to get all the lines to connect?

    • Adrian Rosebrock October 11, 2015 at 8:09 am #

      What do you mean by “get all the lines to connect”?

      • Tim Clemans October 11, 2015 at 4:31 pm #

        The camera’s edges aren’t connected so I would be unable to fill in the camera with say a blur right?

        • Adrian Rosebrock October 12, 2015 at 6:57 am #

          Yes, that would be correct. You can look into morphological operations, specifically the closing operation to help close gaps between lines — but if the gaps are too big, you might have to modify your actual object detection procedure.

  9. static November 2, 2015 at 9:32 pm #

    Once you do edge detection, how can it be used to find an object?

    • Adrian Rosebrock November 3, 2015 at 10:05 am #

      Once you find edges in your images, you can use the cv2.findContours function to find the actual objects that correspond to these edges. For an example of doing object detection with contours, please see this post.

      • Brian November 15, 2015 at 5:00 pm #

        Your post on finding contours to detect squares, rectangles, and circles was very simple and straight forward. How would you detect the dolphin in your example above?

        • Adrian Rosebrock November 16, 2015 at 6:36 am #

          If you wanted to detect the actual dolphin, a simple contour approach would not work. You would likely need to train a custom object detector using something like the HOG + Linear SVM framework.

  10. ehsan December 29, 2015 at 10:16 am #

    hi
    I want apply edge detection with opencv (python) on live video on raspberry pi2 but all samples what i found aplle the edge detect on the image naot real time live video

    • Adrian Rosebrock December 30, 2015 at 7:06 am #

      Are you using a Raspberry Pi camera or a USB camera? If you’re using a Raspberry Pi camera, start here. Once you start looping over the frames of the video stream, you can perform edge detection on them. Once you know how to apply edge detection to a single image, it’s not that different to apply it to a video.

  11. ehsan January 4, 2016 at 12:53 pm #

    hi
    I use rasspberry pi camera and thanks for your link but i dont know how can apply the edge detect on evry frame please send a simple example link about how apply the edge detect on frame of the video

    • Adrian Rosebrock January 4, 2016 at 2:37 pm #

      I would suggest starting with this tutorial: Unifying picamera and cv2.VideoCapture into a single class with OpenCV

      You can then modify the code inside the loop to do:

      I hope that helps!

      • ehsan January 25, 2016 at 2:23 pm #

        hi adrian
        i use this code for real time edge detect on frame of video and with print command show the matrix of frame but how can i determine the size of matrix of edge detect ?
        output from this function imutils.auto_canny(frame)
        plese help me this is my important problem

        • Adrian Rosebrock January 25, 2016 at 4:03 pm #

          I’m not sure I understand what you mean by “determine size of matrix of edge detect”? If you’re looking to simply display the output of edge detection process, you could do:

          cv2.imshow("Edges", imutils.auto_canny(frame))

          If you’re looking to get the shape of the image, you can use the .shape of the array:

          print(edge.shape)

          If you want to process the edges returned by the edge detection process, you need to use the cv2.findContours method.

  12. ehsan January 5, 2016 at 4:44 pm #

    Thannks for your reply

    I use the code of your link but when type python [file name].py picamera is not start and show error :videostream is not define

    • Adrian Rosebrock January 6, 2016 at 6:28 am #

      Make sure you have the latest version of imutils installed:

      $ pip install --upgrade imutils --no-cache-dir

      This will ensure the latest version is pulled down and installed.

  13. ehsan January 10, 2016 at 3:15 pm #

    hi
    thanks for reply
    finally i success apply the edge detct on live video with your code thank you
    but i need to metrix of frame and mertix of edge on evry frame how can i solve this problem ?

  14. Satej March 9, 2016 at 11:20 am #

    i want to count the number of object using edge detection
    can u help me in the code

    • Adrian Rosebrock March 9, 2016 at 4:38 pm #

      Once you have the edges detected, you should use the cv2.findContours function which can be used to find the “outlines” of each object in the edge map. This will also enable you to “count” the number of objects in the image. If you’re just getting started with OpenCV and computer vision, I would definitely suggest going through Practical Python and OpenCV, which has a chapter dedicated to counting objects based on thresholding and edge detection.

  15. David March 19, 2016 at 8:55 pm #

    Hi Adrian,

    Wouldn’t you want to use the median of the gradient magnitude image rather than the median of the image itself as ‘v’? I’ve found that if I change the Sobel kernel size, the values of the gradient magnitude change, so a threshold set based on the image median would not adapt to that change. Any thoughts?

    Thanks!

    • Adrian Rosebrock March 20, 2016 at 10:40 am #

      The median, by definition, is less sensitive to outliers than other aggregates, such as the average. And yes, if you change the size of the Sobel kernel, the more pixels are considered in the convolution. But again, due to the median being less sensitive to outliers, this shouldn’t have a substantial impact. However, that is also worth testing 🙂

  16. David March 20, 2016 at 12:00 pm #

    After thinking about it further it’s not just that it doesn’t adapt to the size of the Otsu kernel, but rather that the Otsu threshold will be between 0 and 255, and the gradient magnitude image will be valued between 0 and 1500 or so, so the point of my question really was WHICH image to compute statistics on (the original or the gradient magnitude), not so much which statistics to compute.

    David

  17. Chuck May 6, 2016 at 5:37 am #

    As usual, a wonderful post.

    But I’ve also a question. Sometimes is pretty difficult to find the edges ’cause there isn’t a big contrast between the object and the background. I read in another post you suggest to use a series of dilations to close the gaps in the outline.
    Could you please to name some dilations we could use? I was using these here: http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_morphological_ops/py_morphological_ops.html#morphological-ops, but without satisfactory results.
    Thanks a lot again!!

    • Adrian Rosebrock May 6, 2016 at 4:30 pm #

      To perform dilations and erosions, you’ll need to use the cv2.dilate and cv2.erode functions, respectively. These are examples of morphological operations.

  18. Dita Cihlářová July 21, 2016 at 10:41 am #

    Hello! I love your website. It have really helped me with my bachelor’s thesis. However, I would like to ask about those steps for Canny edge detection. Aren’t steps 4 and 5 one and the same?
    “Step 4: Apply thresholding using a lower and upper boundary on the gradient values.
    Step 5: Track edges using hysteresis by suppressing weak edges that are not connected to strong edges.”
    The way I understand it (and according to this: http://users.ecs.soton.ac.uk/msn/book/new_demo/thresholding/), hysteresis thresholding is an advanced type of thresholding, that is using two thresholds instead of one. So it seems to me that Step 4 is useless here. Am I missing something?

    • Adrian Rosebrock July 21, 2016 at 12:37 pm #

      Correct, hysteresis is more of an “advanced” type of thresholding, more akin to non-maxima suppression. However, they are two separate steps. The first thresholding steps give you the edge regions. Then you apply hysteresis along each of the thresholded regions, which is essentially non-maxima suppression, and is used to prune the thresholded regions. Step #4 is actually a requirement, the output of which is then fed into hysteresis.

      • Dita Cihlářová July 21, 2016 at 1:25 pm #

        Thank you for you quick response! So Step 5 needs an image with edges and then hysteresis is applied to remove some of them (false positives). But does Step 3 not produce such image? I thought that after applying non-maxima suppresion in Step 3, I get all candidate edges. Is this incorrect?
        And second question: If you need two thresholds for Step 4 and two thresholds for hysteresis in Step 5, are they the same pair?
        Thank you very much for your time.

        • Adrian Rosebrock July 22, 2016 at 11:04 am #

          My apologies regarding my previous comment, I misspoke regarding the hysteresis step. To clarify:

          In Step #3, you’ll apply NMS to get what I would call “edge candidates”. These edge candidates may or may not be edges — we need to apply Step #4 to threshold them based on their gradient values (the threshold values). Since we are using two thresholds, this is the first part of hysteresis. But there is also a second step (Step #5): the connected-component analysis step. This gives us another set of edge candidates. We are pretty confident as to whether or not these regions are actual edges or not, but in order to determine this, we need to track the actual edges and determine how they are connected.

  19. Sam July 29, 2016 at 2:01 am #

    Thank you for your post Adrian, as a beginner i always find your post the easiest to understand and learn.
    i have 2 simple questions: first why do we need to convert the image to grayscale before blurring it? i only notice that it seems to take less time for the program to blur a grayscale image than a BGR image, is that all?
    Second is that what’s the difference between grayscale and HSV? does converting the image to HSV affect the result? cuz i can’t really see the difference myself 0.0
    Hope my English is good enough for you to understand my questions XD

    • Adrian Rosebrock July 29, 2016 at 8:29 am #

      You can blur the image in either the grayscale or the RGB colorspace. Blurring in grayscale is faster since there is only one channel to blur versus the 3 channels in the RGB space. As for the HSV color space, I would suggest reading up on it here.

  20. Olawale November 23, 2016 at 7:53 am #

    Dear Adrian,
    Thank you for posting this code and detailed explanation.
    I ran into this problem ‘auto_canny.py : error : argument -i/ –images is required ‘while using this code.
    I called the image to be used but no output was displayed even after waiting for an hour.
    What do I have to do?

    • Adrian Rosebrock November 23, 2016 at 8:31 am #

      You need to supply the --images command line argument when executing your Python script via the terminal as I did in the blog post:

      $ python auto_canny.py --images images

  21. Olawale November 23, 2016 at 8:51 am #

    Yes, I did i.e # python auto_canny.py –images images/moon.tif
    Still giving me the same empty output

    • Adrian Rosebrock November 28, 2016 at 10:50 am #

      If that’s the case then I think your version of OpenCV was compiled without TIFF image support. I would suggest following one of my OpenCV install tutorials.

    • Yashas May 17, 2017 at 2:39 am #

      I have been facing a similar problem. I downloaded the .zip file which contained both, the images and the code. But I’m still getting an empty output.

      • Yashas May 17, 2017 at 2:42 am #

        I’m using Ubuntu 16.04 and working on the CV environment i.e. I have executed the “workon cv” command.

  22. sanna March 13, 2017 at 3:35 am #

    np.hstack([wide, tight, auto])) is the best trick to combine image on single window. Is there any cheats that can combine color and binary image together?

    • Adrian Rosebrock March 13, 2017 at 12:11 pm #

      In my opinion np.hstack and np.vstack are the best ways to horizontally/vertically stack images into a single image. For combining color and gray you need np.dstack:

  23. Harsh March 21, 2017 at 1:59 am #

    Can we use it as an extra feature for the CBIR you taught earlier?

    • Adrian Rosebrock March 21, 2017 at 7:07 am #

      Canny edge detection? Edge detection maps can be used as features, but using Histogram of Oriented Gradients will likely give you better results.

  24. sami alfarra March 22, 2017 at 9:43 pm #

    thank you so much . please if i want to detect edges in 45 degree direction not in all direction to know the object oreintation . i want to use canny not sobel what should i do please

  25. Amey March 29, 2017 at 1:45 am #

    Hey Adrian,

    Thanks for the solution for automatic calculation of threshold. I am facing a small problem though. I am working on very large images. Even for working this code I have used Gaussian blur matrix of size (25,25) then only the noise is reducing. But I am getting very small non continuous edges. I think I have to increase the matrix size or some parameters which checks the gradient. I am unable to find that parameter. My intention is to perform perspective transformation and for that I have to obtain exact 4 coordinates automatically. And I cant resize or compress the image as I cant afford the loss of information. So what could be the solution?

    • Adrian Rosebrock March 31, 2017 at 2:04 pm #

      How large are your input images? We typically don’t process images that are larger than 600 pixels along their largest dimension. Resize your image to a smaller size and then process it there.

  26. Hannah August 9, 2017 at 8:16 am #

    Hey Adrian.
    Thank you for your great post. It’s really simple and can be implemented well.

    Can you explain more about how you use sigma?
    Is this sigma the same as standard deviation on Gaussian filter in Canny algorithm step?

    • Adrian Rosebrock August 10, 2017 at 8:48 am #

      Hi Hannah — no, the sigma value is not the same as your variance/standard deviation (perhaps a better name for this parameter would have been epsilon). I’ve found (empirically) that leaving at 0.33 returns reasonable results across multiple datasets; however, you should consider testing various values and visually examining the results.

  27. Dawn Rabor September 9, 2017 at 5:45 am #

    Hi Adrian!
    This post was very helpful to me. May I know how did you get your sigma value?

    • Adrian Rosebrock September 11, 2017 at 9:19 am #

      I tuned it empirically by running the edge detector on a dataset and visually inspecting the results. I would suggest you to do the same for your own datasets.

Leave a Reply