Real-time barcode detection in video with Python and OpenCV

Today’s post is a followup to a previous (extremely popular) article on detecting barcodes in images using Python and OpenCV.

In the previous post we explored how to detect and find barcodes in images. But today we are going to refactor the code to detect barcodes in video.

As an example, check out the screenshot below (captured using a video stream from my webcam) of me holding up the back cover of Modern Warfare 3 and the barcode being detected successfully.

Figure 1: Detecting barcodes in video streams using Python and OpenCV.

Figure 1: Detecting barcodes in video streams using Python and OpenCV.

Note: Big thanks to Jason who commented on the original post and mentioned that it would be really cool to see barcode detection applied to video. Thanks for the suggestion! And you’re 100% right, it is really cool to see barcode detection applied to video.

For example, let’s pretend that we are working at GameStop on the 26th of December. There are a line of kids ten blocks long outside our store — all of them wanting to return or exchange a game (obviously, their parents or relatives didn’t make the correct purchase).

To speedup the exchange process, we are hired to wade out into the sea of teenagers and start the return/exchange process by scanning barcodes. But we have a problem — the laser guns at the register are wired to the computers at the register. And the chords won’t reach far enough into the 10-block long line!

However, we have a plan. We’ll just use our smartphones instead!

Using our trusty iPhones (or Androids), we open up our camera app, set it to video mode, and head into the abyss.

Whenever we hold a video game case with a barcode in front of our camera, our app will detect it, and then relay it back to the register.

Sound too good to be true?

Well. Maybe it is. After all, you can accomplish this exact same task using laser barcode readers and a wireless connection. And as we’ll see later on in this post that our approach only works in certain conditions.

But I still think this a good tutorial on how to utilize OpenCV and Python to read barcodes in video — and more importantly, it shows you how you can glue OpenCV functions together to build a real-world application.

Anyway, continue reading to learn how to detect barcodes in video using OpenCV and Python!

Looking for the source code to this post?
Jump right to the downloads section.

Real-time barcode detection in video with Python and OpenCV

So here’s the game plan. Our barcode detection in video system can be broken into two components:

  • Component #1: A module that handles detecting barcodes in images (or in this case, frames of a video) Luckily, we already have this. We’ll just clean the code up a bit and reformat it to our purposes.
  • Component #2: A driver program that obtains access to a video feed and runs the barcode detection module.

We’ll go ahead and start with the first component, a module to detect barcodes in single frames of a video.

Component 1: Barcode detection in frames of a video

I’m not going to do a complete and exhaustive code review of this component, that was handled in my previous post on barcode detection in images.

However, I will provide a quick review for the sake of completeness (and review a few minor updates). Open up a new file, name it simple_barcode_detection.py , and let’s get coding:

If you read the previous post on barcode detection in images then this code should look extremely familiar.

The first thing we’ll do is import the packages we’ll need — NumPy for numeric processing and cv2  for our OpenCV bindings.

From there we define our detect  function on Line 6. This function takes a single argument, the image  (or frame of a video) that we want to detect a barcode in.

Line 8 converts our image to grayscale, while Lines 12-18 find regions of the image that have high horizontal gradients and low vertical gradients (again, if you would like more detail on this part of the code, refer to the previous post on barcode detection).

We then blur and threshold the image on Lines 21 and 22 so we can apply morphological operations to the image on Lines 25-30. These morphological operations are used to reveal the rectangular region of the barcode and ignore the rest of the contents of the image.

Now that we know the rectangular region of the barcode, we find its contour (or simply, its “outline”) on Lines 33-35.

If no outline can be found, then we make the assumption that there is no barcode in the image (Lines 38 and 39).

However, if we do find contours in the image, then we sort the contours by their area on Line 43 (where the contours with the largest area appear at the front of the list). Again, we are making the assumption that the contour with the largest area is the barcoded region of the frame.

Finally, we take the contour and compute its bounding box (Lines 44-46). This will give us the (x, y) coordinates of the barcoded region, which is returned to the calling function on Line 49.

Now that our simple barcode detector is finished, let’s move on to Component #2, the driver that glues everything together.

Component #2: Accessing our camera to detect barcodes in video

Let’s move on to building the driver to detect barcodes in video. Open up a new file, name it detect_barcode.py , and let’s create the second component:

Again, we’ll start our by importing the packages we need. I’ve placed our simple_barcode_detection  function in the pyimagesearch  module for organizational purposes. Then, we import argparse  for parsing command line arguments and cv2  for our OpenCV bindings.

Lines 9-12 handle parsing our command line arguments. We’ll need a single (optional) switch, --video , which is the path to the video file on desk that contains the barcodes we want to detect.

Note: This switch is useful for running the example videos provided in the source code for this blog post. By omitting this switch you will be able to utilize the webcam of your laptop or desktop.

Lines 16-22 handle grabbing a reference to our vs  feed whether it is the webcam (Lines 16-18) or a video file (Lines 21 and 22).

Now that the setup is done, we can move on to applying our actual barcode detection module:

On Line 25 we start looping over the frames of our video — this loop will continue to run until (1) the video runs out of frames or (2) we press the q  key on our keyboard and break from the loop.

We query our vs  on Line 29, which returns a 2-tuple. We handle whether we’re using VideoStream  or cv2.VideoCapture  on Line 30.

If the frame was not successfully grabbed (such as when we reach the end of the video file), we break from the loop on Lines 34 and 35.

Now that we have our frame, we can utilize our barcode detection module to detect a barcode in it — this handled on Line 38 and our bounding box is returned to us.

We draw our resulting bounding box around the barcoded region on Line 42 and display our frame to our screen on Line 45.

Finally, Lines 46-50 handle breaking from our loop if the q  key is pressed on our keyboard while Lines 53-61 cleanup pointers to our video stream object.

So as you can see, there isn’t much to our driver script!

Let’s put this code into action and look at some results.

Successful barcode detections in video

Let’s try some some examples. Open up a terminal and issue the following command:

The video at the top of this post demonstrates the output of our script. And below is a screenshot for each of the three successful barcode detections on the video games:

Figure 2: Successfully detecting the barcode of three XBox video games in a video stream.

Figure 2: Successfully detecting the barcode of three XBox video games in a video stream.

Let’s see if we can detect barcodes on a clothing coupon:

Here’s an example screenshot from the video stream:

Figure 3: Another successful barcode detection using Python and OpenCV.

Figure 3: Another successful barcode detection using Python and OpenCV.

And the full video of the output:

Of course, like I said that this approach only works in optimal conditions (see the following section for a detailed description of limitations and drawbacks).

Here is an example of where the barcode detection did not work:

Figure 4: An unsuccessful barcode detection. The barcode is too far away from the camera.

Figure 4: An unsuccessful barcode detection. The barcode is too far away from the camera.

In this case, the barcode is too far away from the camera and there are too many “distractions” and “noise” in the image, such as large blocks of text on the video game case.

This example is also clearly a failure, I just thought it was funny to include:

Figure 5: My ear is clearly not a barcode.

Figure 5: My ear is clearly not a barcode.

Again, this simple implementation of barcode detection will not work in all cases. It is not a robust solution, but rather an example of how simple image processing techniques can give surprisingly good results, provided that assumptions in the following section are met.

Limitations and Drawbacks

So as we’ve seen in this blog post, our approach to detecting barcodes in images works well — provided we make some assumptions regarding the videos we are detecting barcodes in.

The first assumption is that we have a static camera that is “looking down” on the barcode at a 90-degree angle. This will ensure that the gradient region of the barcoded image will be found by our simple barcode detector.

The second assumption is that our video has a “close up” of the barcode, meaning that we are holding our smartphones directly overtop of the barcode, and not holding the barcode far away from the lens. The farther we move the barcode away from the camera, the less successful our simple barcode detector will be.

So how do we improve our simple barcode detector?

Great question.

Christoph Oberhofer has provided a great review on how robust barcode detection is done in QuaggaJS. And my friend Dr. Tomasz Malisiewicz has written a fantastic post on how his VMX software can be utilized to train barcode detectors using machine learning. If you’re looking for the next steps, be sure to check out those posts!

Recognizing and decoding barcodes

Figure 6: ZBar coupled with OpenCV and Python makes for a great Raspberry Pi barcode project. My name, “AdrianRosebrock” is encoded in this CODE128 barcode.

Today we detected presence of barcodes. If you’re hoping to actually recognize and decode barcodes, then look no further than the following blog post: An OpenCV barcode and QR code scanner with ZBar. Using a package called ZBar, you’ll be able to decode barcodes into human readable text very easily.

Summary

In this blog post we built upon our previous codebase to detect barcodes in images. We extended our code into two components:

  • A component to detect barcodes in individual frames of a video.
  • And a “driver” component that accesses the video feed of our camera or video file.

We then applied our simple barcode detector to detect barcodes in video.

However, our approach does make some assumptions:

  • The first assumption is that we have a static camera view that is “looking down” on the barcode at a 90-degree angle.
  • And the second assumption is that we have a “close up” view of the barcode without other interfering objects or noise in the view of the frame.

In practice, these assumptions may or may-not be guaranteeable. It all depends on the application you are developing!

At the very least I hope that this article was able to demonstrate to you some of the basics of image processing, and how these image processing techniques can be leveraged to build a simple barcode detector in video using Python and OpenCV.

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , , , , , , ,

50 Responses to Real-time barcode detection in video with Python and OpenCV

  1. tita June 25, 2015 at 10:15 pm #

    thank you for the great article 🙂
    but i don’t understand about >> “I’ve placed our simple_barcode_detection function in the pyimagesearch module for organizational purposes.”

    what is pyimagesearch module?

    • Adrian Rosebrock June 26, 2015 at 5:55 am #

      The PyImageSearch module is just a directory named pyimagesearch with a __init__.py file inside it to indicate that it is a module to the Python programming language. Download the source code associated with this post to get a better idea regarding the organization of the code.

  2. Rishi October 18, 2015 at 2:30 pm #

    Is there any means to decode the barcodes using python and opencv? I have used zbar to decode qrcodes . But didnt find much about the barcodes

    • Adrian Rosebrock October 19, 2015 at 6:13 am #

      The zbar library is pretty much standard for this type of thing. If you want to decode barcodes in Python + OpenCV, you’ll likely need to roll your own method.

  3. Dmitriy November 4, 2015 at 4:51 am #

    Thanks a lot for the tutorial, it helped a lot!

    One thing to notice: if someone by some reason is using OpenCV 3 instead of OpenCV 2 (as I am), then syntax should be a bit different in the simple_barcode_detection.py.
    You can find a gist with the updated code here: https://gist.github.com/998c7e2bfd06f7fc217c

    Best,
    Dmitriy

    • Adrian Rosebrock November 4, 2015 at 6:30 am #

      Awesome, thanks so much for sharing Dmitriy!

    • Mario November 5, 2015 at 6:02 pm #

      Hi Dmitriy,

      The gist URL is giving me a 404 on github. Could you, please, check if it is correct?

      Many thanks in advance!

      Regards,

      Mário

  4. Dmitriy November 10, 2015 at 1:21 pm #

    Hi Mario,

    Thanks! It is something strange with that gist – it was correct several days ago, but now it’s disappeared.
    Please find the new one (hope this one will survive): https://gist.github.com/b1c0d453283a5305d46d

    Best,
    Dmitriy

    • Adrian Rosebrock November 11, 2015 at 6:36 am #

      Hey Dmitriy — it looks like that one is 404’ing too. If you to email me the .py file, I’ll create a Gist under my personal account and it can live there.

      • Dmitriy November 11, 2015 at 10:32 am #

        Hi Adrian,

        I’ve realized what was wrong: provided link it is a link for the gist cloning, not for the page itself, sorry. 🙂 So, you just need to remove “.git” from the end of the link and then it will work OK.

        • Adrian Rosebrock November 12, 2015 at 5:48 am #

          Done! I have updated the original comments to remove the “.git”. The links are working perfectly now. Thanks again Dmitriy!

  5. Patrick February 21, 2016 at 12:49 am #

    Hi Adrian,

    I executed the program. There wasn’t any error and the video frame did not pop up to show the detection either. What do i miss out?

    • Adrian Rosebrock February 22, 2016 at 4:26 pm #

      Where you using a builtin/USB camera? Or a Raspberry Pi camera module? If there wasn’t an error and a video frame did not pop up, the OpenCV is having trouble accessing your video stream via the cv2.VideoCapture method. Go back and make sure that OpenCV can access your video stream.

    • Miguel April 10, 2016 at 1:27 pm #

      Me too, I change threshold values(line 20) from 255, 255 to 127, 255 and work fine. I use opencv3 and python2 in W10.

      Thanks for all to every body.

  6. Edwin June 8, 2016 at 6:02 pm #

    Hi! I’m doing my project, and trying to use your code, but it doesn’t work! I’m using my Raspberry Pi with the Pi Camera module and the program start, but end real quick. What I’m doing wrong? It does’t even turn on the Pi Camera and does not trough any errors. What Can I DO?

    • Adrian Rosebrock June 9, 2016 at 5:21 pm #

      The code is this program is designed for a USB webcam, not the Raspberry Pi camera module. You can access the Raspberry Pi camera module in this post. Or better yet, modify the code to use the unifying VideoStream class.

  7. Anish jain May 20, 2017 at 10:10 am #

    Hi, Adrian

    The script runs well without any errors and a video frame also pops up but it is unable to detect any barcodes.
    Sometimes it throws random tiny rectangles but doesn’t detect any barcodes.

    PS: I’m using a RaspberryPi-3 , OpenCV-3 and Python-2.7. But I’ve made the four changes in line numbers – 11, 12, 31 and 42 , to make the code compatible with OpenCV-3.

    Please help me resolve the issue.

    Also, Can I use Zbar for barcode recognition with the above mentioned specifications?

    • Adrian Rosebrock May 21, 2017 at 5:11 am #

      Are you using the example video I provided or using your own custom videos? Keep in mind that the barcode must be horizontal in order for it to be etected.

  8. Carlos reyes August 9, 2017 at 1:27 pm #

    Hi, Adrian

    Can your code read the pdf417 barcode? Or what would I’ve to do to achieve it?
    Plis answer me, i’m desperate with this

    • Adrian Rosebrock August 10, 2017 at 8:46 am #

      I would take a look at the zbar library. According to their source code it seems like you might be able to read the pdf417 barcode.

  9. saraswati September 9, 2017 at 3:44 am #

    no any error with barcode image detection.no any output. The detect function is not work.

    I give this command python Barcode_detection.py –image barcode_01.jpg in terminal and no any outpt. and no any error

    • Adrian Rosebrock September 11, 2017 at 9:21 am #

      This blog post is on real-time barcode detection in video. Is there a particular reason why you are passing in a .JPG file instead of a video file?

  10. Atharva Saraf October 6, 2017 at 3:38 am #

    Hey! This really helped. Thanks a lot. I have just started learning opencv, and this really made my day. Can you suggest some good ways of picking up OpenCv and Image processing? Thanks once Again! 😀

    • Adrian Rosebrock October 6, 2017 at 4:55 pm #

      Hi Atharva, thanks for the comment. It’s wonderful to hear that you’ve found the PyImageSearch blog helpful.

      If you’re just getting started with OpenCV and image processing I would suggest you work through my book, Practical Python and OpenCV.

      1,000’s of readers have worked through the book and found it super helpful to learning the fundamentals.

  11. Shaun October 16, 2017 at 4:03 am #

    Is there any way I could do this with the newest OpenCV 3.3 ?

    • Adrian Rosebrock October 16, 2017 at 12:18 pm #

      Yes, a handful of the parameters need to be updated. In particular cv2.findContours and the call to the Sobel/Scharr kernels. Take a look at more recent posts on PyImageSearch and you’ll learn how it’s done.

  12. Mahesa October 19, 2017 at 12:58 am #

    hey adrian, can you teach me how to save that barcode into excel?

    • Adrian Rosebrock October 19, 2017 at 4:43 pm #

      Save the value of the barcode?

      • Mahesa October 20, 2017 at 1:38 pm #

        i mean, when decode barcode output value display on python shell. how to get that value save into excel files?
        sorry for my bad english

        • Adrian Rosebrock October 22, 2017 at 8:38 am #

          I haven’t written any tutorials on decoding the actual barcode value. Take a look at libraries like ZBar which are dedicated to decoding barcodes.

  13. syaf December 17, 2017 at 1:53 am #

    thanks for this useful tutorials..
    may i know whether this code and theory can be applied when detecting qr code?
    if not, may i know what is the change?

  14. sya December 18, 2017 at 11:16 pm #

    hi adrian,
    i wanna know if this code can be used for detecting qr code?
    if not, where does the change supposed to be made?

    • Adrian Rosebrock December 19, 2017 at 4:16 pm #

      I would suggest using zbar for QR codes.

  15. Muktar February 5, 2018 at 3:28 pm #

    Can you hepl me how to use canny edge detector instead of sobel?

  16. satyar February 17, 2018 at 6:55 am #

    I am getting an error at the line cv2.drawContours(frame,box,-1,(0,255,0),2)

    it says “cv2.error:(-215) reader.ptr !=0 in function cvDrawContours

    Could you just guide me how to resolve this error?

    • satyar February 17, 2018 at 6:58 am #

      it doesn’t make much difference even I put ‘box’ in square braces.

    • Adrian Rosebrock February 18, 2018 at 9:43 am #

      I don’t believe I have encountered that error before. Did you use this code when trying to detect the barcodes?

      • satyar February 19, 2018 at 1:02 am #

        yes.. I have used this code only. What might be the reason for it? when I run the code, it just opens up camera and shuts down and shows the above error.

        • Ava April 24, 2018 at 1:53 pm #

          I have exactly same issue. I’m using Python2.7, OpenCV3.4.1.

          • Adrian Rosebrock April 25, 2018 at 5:35 am #

            What operating system are you using?

      • satyar February 20, 2018 at 2:07 pm #

        same error is being repeated

        cv2.error:(-215) reader.ptr !=0 in function cvDrawContours

        Camera just opens and when I try to pose barcode on to webcam, it shuts down automatically and throws the above error.what is this error. I dont even get the ans in google

        • Adrian Rosebrock February 22, 2018 at 9:11 am #

          It sounds like there is a problem with the actual extraction of the contours. Can you clarify if you used the “Downloads” section of this blog post to download the code + example images or did you copy and paste? Additionally, what version of OpenCV and Python are you using?

          • Sang May 7, 2018 at 5:09 pm #

            I have downloaded thru the link on this page and encountered the same issue.

            I am using Python 3.6.5 with cv2 3.4.0.

          • kaisar khatak May 19, 2018 at 1:46 am #

            I get the same error and used the Downloads section code. The code does work on webcam (tested with receipt), but fails on the videos (coupon.mov, video_games.mov). Both video files can be played with Videos player (ubuntu).

            Python27
            OpenCV3
            Ubuntu 16

            ————————–
            python detect_barcode.py –video video/coupon.mov

            cv2.drawContours(frame, [box], -1, (0, 255, 0), 2)
            (-215) reader.ptr != NULL in function cvDrawContours

          • kaisar khatak May 19, 2018 at 2:18 am #

            I re-ran the code with a video clip I made with a book bar code and the program worked. I think the issue is that the video clips that were included do not have a barcode in them at the beginning which throws the error. So, anytime there is no barcode detected (false positives a good thing?) in a frame, the program throws the “reader.ptr != NULL” error. Might be good to check if box has a value before drawing out to the frame…

            # if a barcode was found, draw a bounding box on the frame
            cv2.drawContours(frame, [box], -1, (0, 255, 0), 2)

          • kaisar khatak May 19, 2018 at 2:41 am #

            added the following check and program works now:

            # detect the barcode in the image
            box = simple_barcode_detection.detect(frame)

            if box is None:
            continue

            # if a barcode was found, draw a bounding box on the frame
            cv2.drawContours(frame, [box], -1, (0, 255, 0), 2)

          • Adrian Rosebrock May 22, 2018 at 6:25 am #

            Congrats on resolving the issue and thanks so much for sharing the fix!

  17. Aashi Singh March 4, 2018 at 8:52 am #

    I wanted to make a program that recognises a face and emotion along with bar code detection. I know how to do these individually. I can’t work my way around to make all 3 happen in the same frame. Could you please tell me how to do that.

    • Adrian Rosebrock March 7, 2018 at 9:35 am #

      I’m not sure I understand the project. You’re trying to recognize faces + emotions that have barcodes on them?

Leave a Reply