Watershed OpenCV

watershed_output_coins_02

The watershed algorithm is a classic algorithm used for segmentation and is especially useful when extracting touching or overlapping objects in images, such as the coins in the figure above.

Using traditional image processing methods such as thresholding and contour detection, we would be unable to extract each individual coin from the image — but by leveraging the watershed algorithm, we are able to detect and extract each coin without a problem.

When utilizing the watershed algorithm we must start with user-defined markers. These markers can be either manually defined via point-and-click, or we can automatically or heuristically define them using methods such as thresholding and/or morphological operations.

Based on these markers, the watershed algorithm treats pixels in our input image as local elevation (called a topography) — the method “floods” valleys, starting from the markers and moving outwards, until the valleys of different markers meet each other. In order to obtain an accurate watershed segmentation, the markers must be correctly placed.

In the remainder of this post, I’ll show you how to use the watershed algorithm to segment and extract objects in images that are both touching and overlapping. To accomplish this, we’ll be using a variety of Python packages including SciPy, scikit-image, and OpenCV.

Looking for the source code to this post?
Jump right to the downloads section.

Watershed OpenCV

Figure 1: An example image containing touching objects. Our goal is to detect and extract each of these coins individually.

Figure 1: An example image containing touching objects. Our goal is to detect and extract each of these coins individually.

In the above image you can see examples of objects that would be impossible to extract using simple thresholding and contour detection, Since these objects are touching, overlapping, or both, the contour extraction process would treat each group of touching objects as a single object rather than multiple objects.

The problem with basic thresholding and contour extraction

Let’s go ahead and demonstrate a limitation of simple thresholding and contour detection. Open up a new file, name it contour_only.py , and let’s get coding:

We start off on Lines 2-7 by importing our necessary packages. Lines 10-13 then parse our command line arguments. We’ll only need a single switch here, --image , which is the path to the image that we want to process.

From there, we’ll load our image from disk on Line 17, apply pyramid mean shift filtering (Line 18) to help the accuracy of our thresholding step, and finally display our image to our screen. An example of our output thus far can be seen below:

Figure 2: Output from the pyramid mean shift filtering step.

Figure 2: Output from the pyramid mean shift filtering step.

Now, let’s threshold the mean shifted image:

Given our input image , we then convert it to grayscale and apply Otsu’s thresholding to segment the background from the foreground:

Figure 3: Applying Otsu's automatic thresholding to segment the foreground coins from the background.

Figure 3: Applying Otsu’s automatic thresholding to segment the foreground coins from the background.

Finally, the last step is to detect contours in the thresholded image and draw each individual contour:

Below we can see the output of our image processing pipeline:

Figure 4: The output of our simple image processing pipeline. Unfortunately, our results are pretty poor -- we are not able to detect each individual coin.

Figure 4: The output of our simple image processing pipeline. Unfortunately, our results are pretty poor — we are not able to detect each individual coin.

As you can see, our results are pretty terrible. Using simple thresholding and contour detection our Python script reports that there are only two coins in the images, even though there are clearly nine of them.

The reason for this problem arises from the fact that coin borders are touching each other in the image — thus, the cv2.findContours  function only sees the coin groups as a single object when in fact they are multiple, separate coins.

Note: A series of morphological operations (specifically, erosions) would help us for this particular image. However, for objects that are overlapping these erosions would not be sufficient. For the sake of this example, let’s pretend that morphological operations are not a viable option so that we may explore the watershed algorithm.

Using the watershed algorithm for segmentation

Now that we understand the limitations of simple thresholding and contour detection, let’s move on to the watershed algorithm. Open up a new file, name it watershed.py , and insert the following code:

Again, we’ll start on Lines 2-7 by importing our required packages. We’ll be using functions from SciPy, scikit-image, and OpenCV. If you don’t already have SciPy and scikit-image installed on your system, you can use pip  to install them for you:

Lines 10-13 handle parsing our command line arguments. Just like in the previous example, we only need a single switch, the path to the image --image  we are going to apply the watershed algorithm to.

From there, Lines 17 and 18 load our image from disk and apply pyramid mean shift filtering. Lines 23-25 perform grayscale conversion and thresholding.

Given our thresholded image, we can now apply the watershed algorithm:

The first step in applying the watershed algorithm for segmentation is to compute the Euclidean Distance Transform (EDT) via the distance_transform_edt  function (Line 31). As the name suggests, this function computes the Euclidean distance to the closest zero (i.e., background pixel) for each of the foreground pixels. We can visualize the EDT in the figure below:

Figure 5: Visualizing the Euclidean Distance Transform.

Figure 5: Visualizing the Euclidean Distance Transform.

On Line 32 we take D , our distance map, and find peaks (i.e., local maxima) in the map. We’ll ensure that is at least a 20 pixel distance between each peak.

Line 37 takes the output of the peak_local_max  function and applies a connected-component analysis using 8-connectivity. The output of this function gives us our markers  which we then feed into the watershed  function on Line 38. Since the watershed algorithm assumes our markers represent local minima (i.e., valleys) in our distance map, we take the negative value of D .

The watershed  function returns a matrix of labels , a NumPy array with the same width and height as our input image. Each pixel value as a unique label value. Pixels that have the same label value belong to the same object.

The last step is to simply loop over the unique label values and extract each of the unique objects:

On Line 43 we start looping over each of the unique labels . If the label  is zero, then we are examining the “background component”, so we simply ignore it.

Otherwise, Lines 51 and 52 allocate memory for our mask  and set the pixels belonging to the current label to 255 (white). We can see an example of such a mask below on the right:

Figure 6: An example mask where we are detecting and extracting only a single object from the image.

Figure 6: An example mask where we are detecting and extracting only a single object from the image.

On Lines 55-57 we detect contours in the mask  and extract the largest one — this contour will represent the outline/boundary of a given object in the image.

Finally, given the contour of the object, all we need to do is draw the enclosing circle boundary surrounding the object on Lines 60-63. We could also compute the bounding box of the object, apply a bitwise operation, and extract each individual object as well.

Finally, Lines 66 and 67 display the output image to our screen:

Figure 7: The final output of our watershed algorithm -- we have been able to cleanly detect and draw the boundaries of each coin in the image, even though their edges are touching.

Figure 7: The final output of our watershed algorithm — we have been able to cleanly detect and draw the boundaries of each coin in the image, even though their edges are touching.

As you can see, we have successfully detected all nine coins in the image. Furthermore, we have been able to cleanly draw the boundaries surrounding each coin as well. This is in stark contrast to the previous example using simple thresholding and contour detection where only two objects were (incorrectly) detected.

Applying the watershed algorithm to images

Now that our watershed.py  script is finished up, let’s apply it to a few more images and investigate the results:

Figure 8: Again, we are able to cleanly segment each of the coins in the image.

Figure 8: Again, we are able to cleanly segment each of the coins in the image.

Let’s try another image, this time with overlapping coins:

Figure 9: The watershed algorithm is able to segment the overlapping coins from each other.

Figure 9: The watershed algorithm is able to segment the overlapping coins from each other.

In the following image, I decided to apply the watershed algorithm to the task of pill counting:

Figure 10: We are able to correctly count the number of pills in the image.

Figure 10: We are able to correctly count the number of pills in the image.

The same is true for this image as well:

Figure 11: Applying the watershed algorithm with OpenCV to count the number of pills in an image.

Figure 11: Applying the watershed algorithm with OpenCV to count the number of pills in an image.

Summary

In this blog post we learned how to apply the watershed algorithm, a classic segmentation algorithm used to detect and extract objects in images that are touching and/or overlapping.

To apply the watershed algorithm we need to define markers which correspond to the objects in our image. These markers can be either user-defined or we can apply image processing techniques (such as thresholding) to find the markers for us. When applying the watershed algorithm, it’s absolutely critical that we obtain accurate markers.

Given our markers, we can compute the Euclidean Distance Transform and pass the distance map to the watershed function itself, which “floods” valleys in the distance map, starting from the initial markers and moving outwards. Where the “pools” of water meet can be considered boundary lines in the segmentation process.

The output of the watershed algorithm is a set of labels, where each label corresponds to a unique object in the image. From there, all we need to do is loop over each of the labels individually and extract each object.

Anyway, I hope you enjoyed this post! Be sure download the code and give it a try. Try playing with various parameters, specifically the min_distance  argument to the peak_local_max  function. Note how varying the value of this parameter can change the output image.

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 11-page Resource Guide on Computer Vision and Image Search Engines, including exclusive techniques that I don’t post on this blog! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , ,

48 Responses to Watershed OpenCV

  1. Pranav November 2, 2015 at 1:58 pm #

    Hi Adrian,

    (Particularly for detecting circles say for example red blood cells) How does watershed algorithm compare to hough_circles?

    -Pranav

    • Adrian Rosebrock November 3, 2015 at 10:06 am #

      For detecting red blood cells, this method will likely perform better than Hough circles. The parameters to Hough circles can be tricky to tune and even if you get them right, overlapping red blood cells can still be missed.

  2. JetC November 2, 2015 at 5:00 pm #

    Does this only work with round objects, or will it also work with squarish/oblong shapes? Thanks

    • Adrian Rosebrock November 3, 2015 at 10:05 am #

      It will work with square/oblong objects as well.

  3. C.W. Predovic November 6, 2015 at 10:43 am #

    Does this accurately work for 3-D images?

    • Adrian Rosebrock November 7, 2015 at 6:20 am #

      Yes, the watershed algorithm is intended to work with both 2D and 3D images. However, I’ve never tried using watershed with 3D images within OpenCV, only the ImageJ implementation.

  4. Alexandre de Siqueira November 12, 2015 at 6:44 am #

    Awesome post, Adrian! Simple and killing! Learned a couple things on this one!
    I’d like to ask you two questions.
    1) Do you know if there is a relation between “pyramid mean shift filtering” (PMSF) and “discrete wavelet transforms” (Mallat cascade algorithm)?
    2) Could you tell what paper originated PMSF?
    Thank you very much!

    • Adrian Rosebrock November 12, 2015 at 12:26 pm #

      Hey Alexandre — I’m glad you enjoyed the blog post, that’s great! To answer your questions:

      1. Pyramid mean-shift filtering is not related to wavelet transforms. Perhaps you are thinking about Haar cascades for object detection?
      2. As for the original paper, you’ll want to look up Comanicu and Meer’s 2002 paper, Mean shift: A robust approach toward feature space analysis

      • Alexandre de Siqueira December 9, 2015 at 10:05 pm #

        Hey Adrian,
        thank you for that references! I downloaded them and will check when time is available 🙂
        Thanks again! Have a nice one!

  5. Taufiq December 7, 2015 at 12:45 pm #

    Hi, can try this source code in android sir ?

    • Adrian Rosebrock December 8, 2015 at 6:32 am #

      If you need to use this code for your Android device, you’ll need to convert it from Python to Java (or another suitable language for Android). This is mainly a Python blog and I don’t do much Java development.

  6. ghanendra March 28, 2016 at 10:42 am #

    Hi Adrian
    while installing scipy its showing this
    It gets stuck at Running setup.py install for scipy What to do??

    • Adrian Rosebrock March 28, 2016 at 1:29 pm #

      What platform are you installing SciPy on? If it’s a Raspberry Pi, it can take up to 45 minutes to 1 hour to compile and install SciPy. be patient with the install.

  7. Jaime Lopez May 21, 2016 at 9:29 am #

    Hi Adrian,

    How could I used Watershed algorithm on remote sensing image to detect objects, because I have too many different objects so I can not apply simple thresholding?

    Thanks, Jaime

    • Adrian Rosebrock May 23, 2016 at 7:32 pm #

      It really depends on what your image contents are. Normally, you apply watershed on an image that you have already thresholded. If you cannot apply thresholding, you might want to consider applying a more advanced segmentation algorithm such as GrabCut. Otherwise, you could look into training a custom object detector.

  8. Jon May 27, 2016 at 1:35 pm #

    Hi Adrian,

    Great tutorial! I’m using watershed to segment touching objects so that I can track them frame by frame using nearest neighbor distances. Everything works pretty good except that sometimes there are too many new contours formed after watershed and I know that I can decrease this by increasing the min_distance parameter in peak_local_max but I need to have a low value because the objects are really small and I start losing contours if I increase the parameter.

    The problem is that the labels (for tracking) for the objects get switched up because I’m comparing the current object’s centroid to contour centroid’s that aren’t part of the same object. Do you have any advice for combining contours on a single object and getting an average centroid to compare to? Any help is much appreciated!

    • Adrian Rosebrock May 27, 2016 at 1:38 pm #

      That is quite the problem to have! Merging contours together is normally done by heuristics. You can compare adjacent watershed regions and compare them based on their appearance, such as texture or color. Regions with similar appearances can be merged together. In this case, you would generate a new mask for the merged objects and compute their corresponding centroid. Alternatively, if you have both contour variables handy, you should be able to compute the weighted (x, y) spatial coordinates to form the new centroid.

  9. Wanderson September 15, 2016 at 2:33 pm #

    Hi Adrian,

    Can I use the watershed algorithm to segment a group of people walking together? The images must be captured by a video camera installed on the ceiling. I performed tests with GMM and KNN, but I got no success.

    Thanks, Wanderson

    • Adrian Rosebrock September 16, 2016 at 8:24 am #

      If you have a mask that represents the foreground (the people) versus the background, then yes, I would give the watershed algorithm a try. However, you might need a more powerful approach depending on your scene. I could foresee utilizing a custom object detector to detect each of the people individually instead of background subtraction/motion detection.

      • Wanderson September 16, 2016 at 1:53 pm #

        I appreciate your reply. I will direct my research from here.

        Thank you!

  10. Tim Brooks January 4, 2017 at 1:29 pm #

    Great article, Adrian
    I am getting sometimes wrong results and would like to debug. What was used to visualize the Euclidean Distance Transform (fig. 5).

    Thanks

    Tim

    • Adrian Rosebrock January 7, 2017 at 9:44 am #

      I actually used matplotlib for that visualization.

  11. David February 21, 2017 at 2:01 pm #

    Been reading your tutorials and will be purchasing the opencv book, really good stuff.I have one question:

    The watershed works by specifying a starting point to the algorithm. In your case this is done by an Euclidean distance from the background color (which is very dark) compared to the objects of interest (coins, pills).

    I would like to use the watershed, but have a somewhat uneven specular background (clear plastic) which goes from almost white to very dark (even in the best diffuse lighting).

    Any suggestions as to segmenting pills, coins or candy in such a scenario?

    Thanks!

    • Adrian Rosebrock February 22, 2017 at 1:33 pm #

      Hey David — it’s great to hear you are enjoying the PyImageSearch blog! Regarding your question, do you have any example images of what you’re working with? That might be easier to provide a solution on techniques to try.

  12. Philip Hahn March 17, 2017 at 7:31 pm #

    David – How did you generate the distance map in “Figure 5: Visualizing the Euclidean Distance Transform.”? An imshow of D looks identical to thresh. Thanks!

    • Adrian Rosebrock March 21, 2017 at 7:39 am #

      Are you asking me or David? Figure 5 was generated using matplotlib and a plot of the distance map.

  13. Miguel March 22, 2017 at 5:04 pm #

    Hi Adrian,

    Great article. I’m trying to segment touching bean seed using the code that you posted,
    in some cases seeds are well segmented, but in others the beans are splited.
    I was decreasing and increasing the min_distance parameter, but i could not segmented the beans. Please, can you suggest me what can i do in that cases

    these are my images:
    https://s14.postimg.org/7371ox9sx/beans.png
    https://s27.postimg.org/vk0x2zo37/Img0878.png

    Thanks

    • Adrian Rosebrock March 23, 2017 at 9:30 am #

      Hey Miguel — I can clearly see the beans touching in the second image. But what is the first image supposed to represent? The beans after segmentation?

      • Miguel March 23, 2017 at 11:05 am #

        Hi Adrian

        Yes, the first image represents the beans after segmentation. After obtaining the contours, i draw the segmented beans one by one. As I told you before, in some cases the beans are segmented correctly.

        Thanks

        • Adrian Rosebrock March 25, 2017 at 9:36 am #

          You’ll likely have to continue to fiddle with the thresholding parameters along with the Watershed parameters. There isn’t a one-size-fits-all solution when using these parameters. More advanced solutions would include using machine learning to do a pixel-wise segmentation of the image, but that’s a bit of a pain and I would try to avoid that.

  14. Nada March 27, 2017 at 8:48 am #

    Hi Adrian,
    Hi i’m a beginner in opencv with python, I’m trying to use the code that you posted but i get this error :
    error: argument -i/–image is required
    Please, can you tell me what can i do

    Thanks

    Your comment is awaiting moderation.

  15. Ian V. May 30, 2017 at 3:48 pm #

    Hi Adrien,

    I have to do a documentation about a programm that i have written in python. For a clean documentation, i would like to know how you displayed codefragments in a box with line numbering?

    Thanks

    • Adrian Rosebrock May 31, 2017 at 1:08 pm #

      Hi Ian — the code fragments displayed in this blog post are handled by a WordPress plugin I use.

  16. shyam June 2, 2017 at 5:05 am #

    hi adrian,
    is there any solution for objects( irregular shape) other than coins

  17. Akash Kumar June 26, 2017 at 6:04 am #

    Hi Adrian,

    It’s a great and perfect tutorial. I would like to know what does that [1] mean and even in the contours [-2]? I am new to opencv.

    thresh = cv2.threshold(gray, 0, 255,cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
    cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[-2]

    What is the difference between cv2.THRESH_BINARY|cv2.THRESH_OTSU and cv2.THRESH_BINARY+cv2.THRESH_OTSU?
    Thanks for the help.

    • Adrian Rosebrock June 27, 2017 at 6:26 am #

      The cv2.threshold function returns a 2-tuple of the threshold value T used (computed via Otsu’s method) and the actual thresh image. Since we are only interested in the thresh image, we grab the value via [1]. This is called Python array indexing. I would suggest reading up on it.

      Also, my recommended way to extract contours via OpenCV 3 and OpenCV 2.4 is now:

      This will make it compatible with both OpenCV 2.4 and OpenCV 3.

      As for your last question the vertical pipe “| is a bitwise OR.

  18. max September 28, 2017 at 3:03 pm #

    This site is an invaluable resource. Thanks for the thorough and lucid explanation of the watershed algorithm. I’m wondering if you can help me filter the set of contours returned by cv2.findContours(). Essentially, what I want is the set of contours that _do not_ share a boundary with other contours. I know this sounds contrary to the problem watershed is meant to solve, but my requirement is similar to the following problem: Given a picture with a number of coins (as in your example) some touching and some completely isolated, return the set of contours for the isolated coins only and exclude the return of any contours that are touching each other. Thanks for your help!

    • Adrian Rosebrock October 2, 2017 at 10:30 am #

      I’m happy to hear you are enjoying the PyImageSearch blog, Max!

      My suggestion here is to take the output contours, draw them, and then apply a connected component analysis. This will help you determine which contours touch.

      I don’t like OpenCV’s connected component analysis function as much as the scikit-image one, so I would suggest starting there.

  19. Carl.C October 6, 2017 at 5:32 am #

    Hi, Adrian.

    Great tutorial. I would like to ask why two identical pictures get different result?

    I downloaded the coins picture straight from this website (Figure1, jpg format), and ran on it. It turned out to be 10 coins instead of 9, and #3 is missing, also said “[INFO] 10 unique segments found”.

    Then I downloaded your source code and ran on the original picture(png format), it’s 9 coins.

    I can’t figure it out because they really look identical.

    It is normal for random mistakes? Or the result somehow relys on the picture format/ picutre quality?

    • Adrian Rosebrock October 6, 2017 at 4:53 pm #

      Which version of OpenCV are you using? There are minor differences between the versions that can cause slight differences in results. Furthermore, keep in mind that OpenCV is heavily dependent on a number of pre-req libraries, such as optimization packages, libraries used to load various image file formats, etc. Unless explicitly configured, no two computer vision development environments are 100% exact, so these differences can compound and sometimes lead to different results.

      • John Goodman November 14, 2017 at 3:33 pm #

        Hi Adrian,

        I’m running Python 3.6.1 and OpenCV 3.2.0 and I’m seeing the same results. What’s happening is that the top left nickel is being counted twice as (#2 and #3).

        Is there a way to tune for this by tweaking the filtering, thresholding or something else?

        example: https://i.imgur.com/vHlh4mU.png

        Anyway, Thanks for the great blog and book!

        • Adrian Rosebrock November 15, 2017 at 12:58 pm #

          Thanks for sharing the screenshot, John. I appreciate it. You would need to do some tweaking to the parameters here, I would have to play with the code to determine what actually needs to be changed. I’ll try to check this out and get back to you.

  20. Luana de Oliveira October 10, 2017 at 5:01 pm #

    Hello Adrian,
    Thank you very much for the tutorial. It’s great and it has helped me a lot up here.
    I added at the end of the code a simple function to get the coordinates (x, y, and r) of the centroid of the circles
    My current problem is that I’m trying to use this code in georeferenced images (tiff), to get the UTM coordinates (x, y and radius in meters) of each centroid at the end. I tried the gdal, but I could not. Would you have any tips?
    Thank you!

    • Adrian Rosebrock October 13, 2017 at 9:02 am #

      Hi Luana — unfortunately my experience with georeferenced images is pretty minimal, so I’m not sure what the best solution to the problem is. Sorry I couldn’t be of more help here!

  21. shubham November 16, 2017 at 11:15 am #

    hello Adrian! i have also tried this code but after running the first segment of code its giving output but no image is showing at all.only a window with gray background.
    Please help me…

    • Adrian Rosebrock November 18, 2017 at 8:20 am #

      That is indeed strange behavior! What version of OpenCV are you using? And how did you install OpenCV on your system?

Leave a Reply