Face detection with OpenCV and deep learning

Today I’m going to share a little known secret with you regarding the OpenCV library:

You can perform fast, accurate face detection with OpenCV using a pre-trained deep learning face detector model shipped with the library.

You may already know that OpenCV ships out-of-the-box with pre-trained Haar cascades that can be used for face detection…

…but I’m willing to bet that you don’t know about the “hidden” deep learning-based face detector that has been part of OpenCV since OpenCV 3.3.

In the remainder of today’s blog post I’ll discuss:

  • Where this “hidden” deep learning face detector lives in the OpenCV library
  • How you can perform face detection in images using OpenCV and deep learning
  • How you can perform face detection in video using OpenCV and deep learning

As we’ll see, it’s easily to swap out Haar cascades for their more accurate deep learning face detector counterparts.

To learn more about face detection with OpenCV and deep learning, just keep reading!

Looking for the source code to this post?
Jump right to the downloads section.

Face detection with OpenCV and deep learning

Today’s blog post is broken down into three parts.

In the first part we’ll discuss the origin of the more accurate OpenCV face detectors and where they live inside the OpenCV library.

From there I’ll demonstrate how you can perform face detection in images using OpenCV and deep learning.

I’ll then wrap up the blog post discussing how you can apply face detection to video streams using OpenCV and deep learning as well.

Where do these “better” face detectors live in OpenCV and where did they come from?

Back in August 2017, OpenCV 3.3 was officially released, bringing it with it a highly improved “deep neural networks” ( dnn ) module.

This module supports a number of deep learning frameworks, including Caffe, TensorFlow, and Torch/PyTorch.

The primary contributor to the dnn  module, Aleksandr Rybnikov, has put a huge amount of work into making this module possible (and we owe him a big round of thanks and applause).

And since the release of OpenCV 3.3, I’ve been sharing a number of deep learning OpenCV tutorials, including:

However, what most OpenCV users do not know is that Rybnikov has included a more accurate, deep learning-based face detector included in the official release of OpenCV (although it can be a bit hard to find if you don’t know where to look).

The Caffe-based face detector can be found in the face_detector  sub-directory of the dnn samples:

Figure 1: The OpenCV repository on GitHub has an example of deep learning face detection.

When using OpenCV’s deep neural network module with Caffe models, you’ll need two sets of files:

  • The .prototxt file(s) which define the model architecture (i.e., the layers themselves)
  • The .caffemodel file which contains the weights for the actual layers

Both files are required to when using models trained using Caffe for deep learning.

However, you’ll only find the prototxt files here in the GitHub repo.

The weight files are not included in the OpenCV samples  directory and it requires a bit more digging to find them…

Where can I can I get the more accurate OpenCV face detectors?

For your convenience, I have included both the:

  1. Caffe prototxt files
  2. and Caffe model weight files

…inside the “Downloads” section of this blog post.

To skip to the downloads section, just click here.

How does the OpenCV deep learning face detector work?

Figure 2: Deep Learning with OpenCV’s DNN module.

OpenCV’s deep learning face detector is based on the Single Shot Detector (SSD) framework with a ResNet base network (unlike other OpenCV SSDs that you may have seen which typically use MobileNet as the base network).

A full review of SSDs and ResNet is outside the scope of this blog post, so if you’re interested in learning more about Single Shot Detectors (including how to train your own custom deep learning object detectors), start with this article here on the PyImageSearch blog and then take a look at my book, Deep Learning for Computer Vision with Python, which includes in-depth discussions and code enabling you to train your own object detectors.

Face detection in images with OpenCV and deep learning

In this first example we’ll learn how to apply face detection with OpenCV to single input images.

In the next section we’ll learn how to modify this code and apply face detection with OpenCV to videos, video streams, and webcams.

Open up a new file, name it detect_faces.py , and insert the following code:

Here we are importing our required packages (Lines 2-4) and parsing command line arguments (Lines 7-16).

We have three required arguments:

  • --image : The path to the input image.
  • --prototxt : The path to the Caffe prototxt file.
  • --model : The path to the pretrained Caffe model.

An optional argument, --confidence , can overwrite the default threshold of 0.5 if you wish.

From there lets load our model and create a blob from our image:

First, we load our model using our --prototxt  and --model  file paths. We store the model as net  (Line 20).

Then we load the image  (Line 24), extract the dimensions (Line 25), and create a blob  (Lines 26 and 27).

The dnn.blobFromImage  takes care of pre-processing which includes setting the blob  dimensions and normalization. If you’re interested in learning more about the dnn.blobFromImage  function, I review in detail in this blog post.

Next, we’ll apply face detection:

To detect faces, we pass the blob  through the net  on Lines 32 and 33.

And from there we’ll loop over the detections  and draw boxes around the detected faces:

We begin looping over the detections on Line 36.

From there, we extract the confidence  (Line 39) and compare it to the confidence threshold (Line 43). We perform this check to filter out weak detections.

If the confidence meets the minimum threshold, we proceed to draw a rectangle and along with the probability of the detection on Lines 46-56.

To accomplish this, we first calculate the (x, y)-coordinates of the bounding box (Lines 46 and 47).

We then build our confidence text  string (Line 51) which contains the probability of the detection.

In case the our text  would go off-image (such as when the face detection occurs at the very top of an image), we shift it down by 10 pixels (Line 52).

Our face rectangle and confidence text  is drawn on the image  on Lines 53-56.

From there we loop back for additional detections following the process again. If no detections  remain, we’re ready to show our output image  on the screen (Lines 59 and 60).

Face detection in images with OpenCV results

Let’s try out the OpenCV deep learning face detector.

Make sure you use the “Downloads” section of this blog post to download:

  • The source code used in this blog post
  • The Caffe prototxt files for deep learning face detection
  • The Caffe weight files used for deep learning face detection
  • The example images used in this post

From there, open up a terminal and execute the following command:

Figure 3: My face is detected in this image with 74% confidence using the OpenCV deep learning face detector.

The above photo is of me during my first trip to Ybor City in Florida, where chickens are allowed to roam free throughout the city. There are even laws protecting the chickens which I thought was very cool. Even though I grew up in rural farmland, I was still totally surprised to see a rooster crossing the road — which of course spawned many “Why did the chicken cross the road?” jokes.

Here you can see my face is detected with 74.30% confidence, even though my face is at an angle. OpenCV’s Haar cascades are notorious for missing faces that are not at a “straight on” angle, but by using OpenCV’s deep learning face detectors, we are able to detect my face.

And now we’ll see how another example works, this time with three faces:

Figure 4: The OpenCV DNN face detector finds all three images without any trouble.


This photo was taken in Gainesville, FL after one of my favorite bands finished up a show at Loosey’s, a popular bar and music venue in the area. Here you can see my fiance (left), me (middle), and Jason (right), a member of the band.

I’m incredibly impressed that OpenCV can detect Trisha’s face, despite the lighting conditions and shadows cast on her face in the dark venue (and with 86.81% probability!)

Again, this just goes to show how much better (in terms of accuracy) the deep learning OpenCV face detectors are over their standard Haar cascade counterparts shipped with the library.

Face detection in video and webcam with OpenCV and deep learning

Now that we have learned how to apply face detection with OpenCV to single images, let’s also apply face detection to videos, video streams, and webcams.

Luckily for us, most of our code in the previous section on face detection with OpenCV in single images can be reused here!

Open up a new file, name it detect_faces_video.py , and insert the following code:

Compared to above, we will need to import three additional packages: VideoStream , imutils , and time .

If you don’t have imutils  in your virtual environment, you can install it via:

Our command line arguments are mostly the same, except we do not have an --image  path argument this time. We’ll be using our webcam’s video feed instead.

From there we’ll load our model and initialize the video stream:

Loading the model is the same as above.

We initialize a VideoStream  object specifying camera with index zero as the source (in general this would be your laptop’s built in camera or your desktop’s first camera detected).

A few quick notes here:

  • Raspberry Pi + picamera users can replace Line 25 with vs = VideoStream(usePiCamera=True).start() if you wish to use the Raspberry Pi camera module.
  • If you to parse a video file (rather than a video stream) swap out the VideoStream  class for FileVideoStream . You can learn more about the FileVideoStream class in this blog post.

We then allow the camera sensor to warm up for 2 seconds (Line 26).

From there we loop over the frames and compute face detections with OpenCV:

This block should look mostly familiar to the static image version in the previous section.

In this block, we’re reading a frame  from the video stream (Line 32), creating a blob  (Lines 37 and 38), and passing the blob  through the deep neural net  to obtain face detections (Lines 42 and 43).

We can now loop over the detections, compare to the confidence threshold, and draw face boxes + confidence values on the screen:

For a detailed review of this code block, please review the previous section where we perform face detection to still, static images. The code here is nearly identical.

Now that our OpenCV face detections have been drawn, let’s display the frame on the screen and wait for a keypress:

We display the frame  on the screen until the “q” key is pressed at which point we break  out of the loop and perform cleanup.

Face detection in video and webcam with OpenCV results

To try out the OpenCV deep learning face detector make sure you use the “Downloads” section of this blog post to grab:

  • The source code used in this blog post
  • The Caffe prototxt files for deep learning face detection
  • The Caffe weight files used for deep learning face detection

Once you have downloaded the files, running the deep learning OpenCV face detector with a webcam feed is easy with this simple command:

Figure 5: Face detection in video with OpenCV’s DNN module.

You can see a full video demonstration, including my commentary, in the following video:


In today’s blog post you discovered a little known secret about the OpenCV library — OpenCV ships out-of-the-box with a more accurate face detector (as compared to OpenCV’s Haar cascades).

The more accurate OpenCV face detector is deep learning based, and in particular, utilizes the Single Shot Detector (SSD) framework with ResNet as the base network.

Thanks to the hard work of Aleksandr Rybnikov and the other contributors to OpenCV’s dnn  module, we can enjoy these more accurate OpenCV face detectors in our own applications.

The deep learning face detectors can be hard to find in the OpenCV library, so
for your convenience, I have put gathered the Caffe prototxt and weight files for you — just use the “Downloads” form below to download the (more accurate) deep learning-based OpenCV face detector.

See you next week with another great computer vision + deep learning tutorial!


If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 11-page Resource Guide on Computer Vision and Image Search Engines, including exclusive techniques that I don’t post on this blog! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , , , , , , ,

56 Responses to Face detection with OpenCV and deep learning

  1. Chris Combs February 26, 2018 at 10:50 am #

    How well does the new OpenCV model recognize faces of various skin tones? Do we know how it was trained?

    • Adrian Rosebrock February 26, 2018 at 1:40 pm #

      I have not performed an exhaustive evaluation for various skin tones so I’m not sure. That would make for a interesting article. If I cannot write one I would love to see one written by a PyImageSearch reader. The GitHub repo has more information on the training process.

  2. wildan February 26, 2018 at 10:53 am #

    what’s different this method with haar cascade’s face detection ?

    • Adrian Rosebrock February 26, 2018 at 1:39 pm #

      This method uses deep learning, in particular a Single Shot Detector (SSD) with ResNet base network architecture. That is the difference.

  3. Ratman February 26, 2018 at 10:58 am #

    Sounds nice, but how is the the performance? There are a lot of face detection frameworks but they are not even near real time.

    • Adrian Rosebrock February 26, 2018 at 1:39 pm #

      Please see the video where I provide a live demonstration. This face detector can be used in real-time.

      • ratman February 28, 2018 at 3:41 pm #

        Thanks for the reply, Adran! I think it is still a bit slow for our family album including 40k pictures, but it is worth a trial then 🙂

  4. KK February 26, 2018 at 10:59 am #

    Hi Adrian, nice to know ..is this detector faster than dlib’s detector? thanks much!

    • Adrian Rosebrock February 26, 2018 at 1:38 pm #

      I haven’t tested them side-by-side, but it should be comparably fast to dlib’s HOG + Linear SVM detector.

  5. Curious Observer February 26, 2018 at 12:02 pm #

    Hey Adrian,

    First off, I want to thank you for the great work you’ve done so far. Your blog is the basis for the computer vision startup we’re founding.

    Coming back to this specific blog post, I haven’t tested it yet, but how do you think the speed of this DNN will compare to Haar cascades on a Raspberry Pi? On my computer, I’m seeing about a 25% slowdown.

    Which would you choose for detecting faces on a Pi where both speed and accuracy were equally important?

    • Adrian Rosebrock February 26, 2018 at 1:38 pm #

      Haar cascades will be the fastest here, but the deep learning face detector will give you the most accuracy. As for which one to choose, that really depends on your project. If you’re looking to detect faces that will naturally have more variability in viewing angle, use the deep learning detector. If the faces will almost always be “straight on” then the Haar cascades will likely be sufficient. Again, it really depends on the project.

    • TetsFR March 10, 2018 at 12:42 pm #

      Not sure about Haar cascades but this deepnet runs on my rpi3 at about 1fps, maybe a bit slower. So that is still usable for some projects although difficult for realtime applications. I am trying to control servos of a pan tilt camera mount and there is so much delay in the feedback loop that it is tricky to manage (if you guys have a suggestion of an algo that is robust to such delay I would take it).
      Tks Andrian for the great tutorial, as always.

  6. Matt February 26, 2018 at 12:45 pm #

    This is awesome to hear OpenCV now ships with the dnn module.

    If you wanted to take this a step further to start recognizing particular faces, would you have to go back to deep learning to actually teach the faces? Would there be a way to leverage this to assist in the collection of labeled samples?


    • Adrian Rosebrock February 26, 2018 at 1:36 pm #

      Facial recognition is a two stage process. The first stage is detecting the presence of a face in an image but not knowing “who” the actual face is. The second stage is taking each detected face and recognizing it. For this, you would need a dedicated facial recognition algorithm. I actually discuss how to create a Python script to assist in collecting labeled faces (as you suggested) inside the PyImageSearch Gurus course. From there you’ll be able to build your own facial recognizer.

  7. swapnil February 26, 2018 at 1:13 pm #

    One of the post I was eagerly looking for. Thanks adrian for this post. You are the best when it comes to computer vision.

    • Adrian Rosebrock February 26, 2018 at 1:34 pm #

      Thank you swapnil! 🙂

  8. Steve Cox February 26, 2018 at 9:04 pm #

    Very nice !!!! I look forward to playing with this example.

    Now how do we train this deep model to recognize “Our” faces 😉

    I think this is in the right direction and away from eigenfaces which I noticed don’t seem to be accurate. (Not an exhaustive test on my part) I can still see using a har cascade in front of the this deep learning SSD. Har is so fast I think the two algo stacked together make sense.

    Thanks again !!!!

    • Adrian Rosebrock February 27, 2018 at 11:33 am #

      There are a bunch of ways to perform face recognition using deep learning. One of my favorites is to use deep learning embeddings of the faces. I’ll cover this is well in the future.

  9. phillity February 26, 2018 at 11:58 pm #

    Hi Adrian. Thanks for the great tutorial!

    • Adrian Rosebrock February 27, 2018 at 11:29 am #

      Thanks so much Phillity, I’m happy you enjoyed it! 🙂

  10. ray February 27, 2018 at 10:38 am #

    great tutorial!!!!

    • Adrian Rosebrock February 27, 2018 at 11:20 am #

      Thank you Ray, I’m glad you enjoyed it! 🙂

  11. Nam Phan February 27, 2018 at 12:53 pm #

    first off, great tut as usual ! excellent !
    i wonder if there are some helper functions accompanying with the package that i can use to extract extra information of face’s parts like : eyes, nose, ears , forehead … positions .
    If not then is there any packages to do such extraction after detecting using this dnn

    • Adrian Rosebrock February 28, 2018 at 1:49 pm #

      It sounds like you’re referring to facial landmarks. See this post for more details, including code.

  12. han February 28, 2018 at 2:45 am #

    Thank you for your efforts and sharing.
    I really like to read your posts!!

    • Adrian Rosebrock February 28, 2018 at 1:50 pm #

      Thanks Han, I’m glad you’re enjoying them!

  13. GabriellaK February 28, 2018 at 5:16 am #

    Great, Is there something new for full body detection?

    • Adrian Rosebrock February 28, 2018 at 1:50 pm #

      What specifically regarding full body detection? Detecting the presence of a body in an image? Localizing each of the arms, legs, joints, etc.?

  14. Mark February 28, 2018 at 7:17 am #

    Thanks Adrian,

    Brilliant stuff as usual, I managed to replicate that in C++ but still very slow on my RPi.
    Any idea if Movidius stick can be used to boost the recognition part here?


    • Adrian Rosebrock February 28, 2018 at 1:52 pm #

      I haven’t tried this code on the Movidius but from the previous post I used a Caffe model weights + architecture for a MobileNet + SSD. It seems to reason that a ResNet + SSD would work as well. I would try loading the face detector via the Movidius but I get the feeling that you might have to work with it a bit.

  15. Chris Burns February 28, 2018 at 11:38 am #

    Adrian, thank you for this post and some excellent insight into OpenCV. Can you elaborate on why you chose to use the VideoStream feature of imutils? I have a non-traditional set up (Rpi3 with custom ARM64 (aarch64) kernel. I’ve compiled OpenCV and everything looks good there but the imutils – vs.read() call is returning null. I was thinking about going to OpenCV.videoCapture but thought I would ask the above question before I started. Thanks!

    • Adrian Rosebrock February 28, 2018 at 1:53 pm #

      The VideoStream is my implementation of a faster, threaded frame polling class. It is compatible with both USB/built-in webcams along with the Raspberry Pi camera module. You could use OpenCV’s cv2.VideoCapture function but you’ll get an extra performance boost from VideoStream which is a must when working with the Raspberry Pi. You can read more about the VideoStream class here.

  16. Anish Varsha March 1, 2018 at 12:07 pm #

    Hey Adrian,

    I find the tutorial very useful with the differences between SSD and HOG detection are night and day. Can you suggest me where I can find the Face Recognition Using Deep Learning in OpenCV? Thanks!

  17. Dominic Pritham March 3, 2018 at 3:41 pm #

    This is super cool Adrian. It is very reliable. In traditional face detection, I have had issues when the face leaves the frame and re-enters. This is so exciting. Thank you so very much for writing this blog.

    • Adrian Rosebrock March 7, 2018 at 9:42 am #

      Thanks Dominic, I’m glad you enjoyed the post! 🙂

  18. Peshmerge March 6, 2018 at 10:36 am #

    Hi Adrian,

    Thanks for your great article. It’s really helpful! I have couple of notes based on my observation while testing your code.
    I am writing now my thesis at Amsterdam University of Applied Sciences, and it’s about Facial detection and recognition on children. My target group is children aged between 0 and 12 years old.
    I am stil busy with researching, but I tried your code just to build a fast proof-of-concept and it didn’t work well in the beginning. I have adjusted 3 parameters and it did it well.
    Those parameters were:
    1) –confidence, which is now 0.40
    2) x and y while calling cv2.dnn.blobFromImage(). In your original code it’s 300*300 in my code I just changed to be the height of the input image.

    Here is the result of the running your code without changing anything

    here the result after changing the parameters (the confidence doesn’t matter 0.4 or 0.5)

    Do you have any explanation?

    Kind regards, Peshmerge

    • Adrian Rosebrock March 7, 2018 at 9:10 am #

      The confidence is the minimum probability. It is used to filter out weak(er) detections. You can set the confidence as you see best fit for your project/application.

      • Peshmerge March 12, 2018 at 7:18 am #

        Thanks for you reply! To be honest adjusting the x and y just made the difference for me! I wonder why did you choose to give it 300*300?

        • Adrian Rosebrock March 14, 2018 at 1:07 pm #

          300×300 is the typical input image size for SSDs and Faster R-CNNs.

  19. Amal March 6, 2018 at 11:02 pm #

    hey Adrian
    I applied the same code on my raspberry pi 3 but it work very slowly and reboot after few scond each time I run the code

    • Adrian Rosebrock March 7, 2018 at 9:08 am #

      Hey Amal — the OpenCV deep learning face detector will certainly run slow on the Pi. For fast, but less accurate face detection you should use Haar cascades. As for the Pi automatically rebooting, I’m not sure what the problem is. It sounds like a hardware problem or your Pi is overheating.

  20. Ed n. March 7, 2018 at 3:45 pm #

    Hi Adrian,

    Is there a good way to covert Caffe based code to Keras? Using Caffe in the production os kind of hassle.


    • Adrian Rosebrock March 9, 2018 at 9:26 am #

      I would start by going through this resource which includes a number of deep learning model/weight converters.

  21. saverio March 8, 2018 at 9:24 am #

    I tried to run the compiled graph on movidius, but I’am a little bit confused about the retruned value from graph.GetResult(): an array of shape (1407,)!

    For sure a made some mistake…

  22. Adanalı March 8, 2018 at 6:03 pm #

    Hi Adrian
    This is a bit off topic, but I was wondering if you would be so kind as to write an article on making a “people counting” with OpenCV — that is, a program that counts people going in and out of a building via a live webcam feed. There are no great resources available online for this, so if you would write one I’m sure it would drive plenty of traffic to your site. It’s a win for both of us!

    • Adrian Rosebrock March 9, 2018 at 8:54 am #

      Thank you for the suggestion. I will certainly consider it for the future.

  23. gopalakrishna March 9, 2018 at 9:26 pm #

    I am new to OpenCV it would be a great help if you tell how to add the path in argument parse (line 8 in the code)

    • Adrian Rosebrock March 14, 2018 at 1:28 pm #

      Take a look at this post to get started with command line arguments.

  24. lii March 15, 2018 at 9:25 am #

    hi, can someone help me with this error:

    [INFO] loading model…
    [INFO] starting video stream…
    Traceback (most recent call last):

    (h, w) = image.shape[:2]
    AttributeError: ‘NoneType’ object has no attribute ‘shape’

    • Adrian Rosebrock March 19, 2018 at 5:53 pm #

      OpenCV cannot access your webcam. See this post for more details.

  25. Kapil Goyal March 20, 2018 at 4:27 pm #

    This code is not working for a group photo with 50 people.

    • Adrian Rosebrock March 22, 2018 at 10:09 am #

      That’s not too surprising. If there are 50 people in the photo the faces are likely very small. Try increasing the resolution of the image before passing it into the network for detection.

  26. Kapil Goyal March 22, 2018 at 10:14 am #

    I want to create my own training set. How to train my own neural network in python for my college project?

    • Adrian Rosebrock March 22, 2018 at 10:18 am #

      I demonstrate how to train your first Convolutional Neural Network in this post. I would suggest starting there. If you’re interested in a deeper dive and understanding of how to train your own custom networks I would suggest Deep Learning for Computer Vision with Python where I discuss deep learning + computer vision in detail (including how to train your own custom networks). I hope that helps point you in the right direction!


  1. Python, argparse, and command line arguments - PyImageSearch - March 12, 2018

    […] Adrian, I just downloaded the source code to your deep learning face detection blog post, but when I execute it I get the following […]

Leave a Reply