Real-time facial landmark detection with OpenCV, Python, and dlib

Over the past few weeks we have been discussing facial landmarks and the role they play in computer vision and image processing.

We’ve started off by learning how to detect facial landmarks in an image.

We then discovered how to label and annotate each of the facial regions, such as eyes, eyebrows, nose, mouth, and jawline.

Today we are going to expand our implementation of facial landmarks to work in real-time video streams, paving the way for more real-world applications, including next week’s tutorial on blink detection.

To learn how to detect facial landmarks in video streams in real-time, just keep reading.

Looking for the source code to this post?
Jump right to the downloads section.

Real-time facial landmark detection with OpenCV, Python, and dlib

The first part of this blog post will provide an implementation of real-time facial landmark detection for usage in video streams utilizing Python, OpenCV, and dlib.

We’ll then test our implementation and use it to detect facial landmarks in videos.

Facial landmarks in video streams

Let’s go ahead and get this facial landmark example started.

Open up a new file, name it video_facial_landmarks.py , and insert the following code:

Lines 2-9 import our required Python packages.

We’ll be using the face_utils  sub-module of imutils, so if you haven’t installed/upgraded to the latest version, take a second and do so now:

Note: If you are using Python virtual environments, take care to ensure you are installing/upgrading imutils  in your proper environment.

We’ll also be using the VideoStream  implementation inside of imutils , allowing you to access your webcam/USB camera/Raspberry Pi camera module in a more efficientfaster, treaded manner. You can read more about the VideoStream  class and how it accomplishes a higher frame throughout in this blog post.

If you would like to instead work with video files rather than video streams, be sure to reference this blog post on efficient frame polling from a pre-recorded video file, replacing VideoStream  with FileVideoStream .

For our facial landmark implementation we’ll be using the dlib library. You can learn how to install dlib on your system in this tutorial (if you haven’t done so already).

Next, let’s parse our command line arguments:

Our script requires one command line argument, followed by a second optional one, each detailed below:

  • --shape-predictor : The path to dlib’s pre-trained facial landmark detector. Use the “Downloads” section of this blog post to download an archive of the code + facial landmark predictor file.
  • --picamera : An optional command line argument, this switch indicates whether the Raspberry Pi camera module should be used instead of the default webcam/USB camera. Supply a value > 0 to use your Raspberry Pi camera.

Now that our command line arguments have been parsed, we need to initialize dlib’s HOG + Linear SVM-based face detector and then load the facial landmark predictor from disk:

The next code block simply handles initializing our VideoStream  and allowing the camera sensor to warm up:

The heart of our video processing pipeline can be found inside the while  loop below:

On Line 31 we start an infinite loop that we can only break out of if we decide to exit the script by pressing the q  key on our keyboard.

Line 35 grabs the next frame from our video stream.

We then preprocess this frame by resizing it to have a width of 400 pixels and convert it to grayscale (Lines 36 an 37).

Before we can detect facial landmarks in our frame, we first need to localize the face — this is accomplished on Line 40 via the detector  which returns the bounding box (x, y)-coordinates for each face in the image.

Now that we have detected the faces in the video stream, the next step is to apply the facial landmark predictor to each face ROI:

On Line 43 we loop over each of the detected faces.

Line 47 applies the facial landmark detector to the face region, returning a shape  object which we convert to a NumPy array (Line 48).

Lines 52 and 53 then draw a series of circles on the output frame , visualizing each of the facial landmarks. To understand what facial region (i.e., nose, eyes, mouth, etc.) each (x, y)-coordinate maps to, please refer to this blog post.

Lines 56 and 57 display the output frame  to our screen. If the q  key is pressed, we break from the loop and stop the script (Lines 60 and 61).

Finally, Lines 64 and 65 do a bit of cleanup:

As you can see, there are very little differences between detecting facial landmarks in images versus detecting facial landmarks in video streams — the main differences in the code simply involve setting up our video stream pointers and then polling the stream for frames.

The actual process of detecting facial landmarks is the same, only instead of detecting facial landmarks in a single image we are now detecting facial landmarks in a series of frames.

Real-time facial landmark results

To test our real-time facial landmark detector using OpenCV, Python, and dlib, make sure you use the “Downloads” section of this blog post to download an archive of the code, project structure, and facial landmark predictor model.

If you are using a standard webcam/USB camera, you can execute the following command to start the video facial landmark predictor:

Otherwise, if you are on your Raspberry Pi, make sure you append the --picamera 1  switch to the command:

Here is a short GIF of the output where you can see that facial landmarks have been successfully detected on my face in real-time:

Figure 1: A short demo of real-time facial landmark detection with OpenCV, Python, an dlib.

I have included a full video output below as well:

Summary

In today’s blog post we extended our previous tutorials on facial landmarks and applied them to the task of real-time detection.

As our results demonstrated, we are fully capable of detecting facial landmarks in a video stream in real-time using a system with a modest CPU.

Now that we understand how to access a video stream and apply facial landmark detection, we can move on to next week’s real-world computer vision application — blink detection.

To be notified when the blink detection tutorial goes live, be sure to enter your email address in the form below — this is a tutorial you won’t want to miss!

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , , , , ,

126 Responses to Real-time facial landmark detection with OpenCV, Python, and dlib

  1. tony April 17, 2017 at 12:03 pm #

    Thanks for this tutorial . how the face landmarks can be more stable , I tried the tutorial and the points are shaky

  2. Mansoor Nasir April 17, 2017 at 1:00 pm #

    Great work Adrian, my only question is will it work with multiple faces? And will it affect the performance or accuracy?

    • Adrian Rosebrock April 19, 2017 at 12:58 pm #

      I would suggest going back and reading my previous posts facial landmarks. This method will work with multiple faces provided that each face in the image/video stream can be detected.

  3. Shravan Kumar Parunandula April 17, 2017 at 1:05 pm #

    Awaiting for this, thank you so much.

    • Adrian Rosebrock April 19, 2017 at 12:57 pm #

      Thank you Shravan! 🙂

  4. Linus April 17, 2017 at 1:40 pm #

    This one is freaking awesome! Will definitively try it out and install dlib. Thanks Adrian for this row of posts! 🙂

    • Adrian Rosebrock April 19, 2017 at 12:55 pm #

      Thanks Linus — it only gets better from here 🙂

  5. Muhammad April 17, 2017 at 2:34 pm #

    Beautiful! Thanks a lot!

  6. Levi Blaney April 17, 2017 at 9:02 pm #

    Hey this is really great stuff. I can’t wait to try it out. I want to use it for my magic mirror to tell who is standing in front of it. I’m sure I could Google a algorithm up but could you do a blog post on how to some what reliably detect the same person over and over again.

    • Adrian Rosebrock April 19, 2017 at 12:53 pm #

      I wouldn’t recommend using facial landmarks for facial recognition. Algorithms such as LBPs for face recognition, Eigenfaces, and Fisherfaces would work well for a magic mirror application. I cover LBPs for face recognition and Eigenfaces inside the PyImageSearch Gurus course.

      • tbanda April 27, 2017 at 10:33 am #

        Hello Adrian…
        what if i use dlib only for face identification because haarcascades do not identify flipped faces and from there i use LBP for face recognition?

        • Adrian Rosebrock April 28, 2017 at 9:29 am #

          As I mentioned, facial landmarks are not used for face identification. You can use either Haar cascades or the dlib built-in factor detector to detect horizontally flipped faces. To detect vertically flipped faces, simply flip your image prior to passing them into the detector.

  7. Joe April 17, 2017 at 9:53 pm #

    This is awesome! Thanks for another great blog Adrian….keep it up!

    • Adrian Rosebrock April 19, 2017 at 12:51 pm #

      Thanks Joe! 🙂

  8. kunal April 18, 2017 at 3:59 am #

    Really impressed by the way you have done with this coding of real-time facial landmark detection for usage in video streams utilizing Python. Splendid!

  9. Linus April 18, 2017 at 1:59 pm #

    And I can’t understand why you import datetime? Is this just from development?

    And the whole thing worked out just fine BTW 😀

    • Adrian Rosebrock April 19, 2017 at 12:48 pm #

      The import datetime cam be safely removed. I had it imported for a different application I was working on.

  10. David J Axelrod April 19, 2017 at 11:07 am #

    Woah, super cool Adrian! Another awesome article

    • Adrian Rosebrock April 19, 2017 at 12:42 pm #

      Thank you David!

  11. Matt Sandy April 19, 2017 at 11:30 pm #

    I really want to make a game controlled by facial expressions. I think it would be hilarious to get people to play it in public.

  12. Sidharth Patnaik April 20, 2017 at 9:49 pm #

    Just another awesome Tutorial, thanks for sharing! 🙂
    was waiting for this, the whole time.
    can you upload a tutorial based on OpenFace, please ?

    • Adrian Rosebrock April 21, 2017 at 10:51 am #

      Sure, I will certainly consider this for a future tutorial.

  13. tony April 21, 2017 at 4:18 am #

    Thanks for this tutorial , I have asked you this question and I haven’t got reply.

    how the face landmarks can be more stable , I tried the tutorial and the points are shaky

    • Adrian Rosebrock April 21, 2017 at 10:45 am #

      Hi Tony — I’m not sure what you mean by “shaky”. The facial landmark predictor included by dlib is pre-trained. You could try to train your own predictor on your own data to see if that improves your result.

      • tony April 22, 2017 at 12:52 am #

        Thanks , I mean the landmark points are not stable ( shaking) . How can I train new predictor for more than 68 landmarks ?

        • Adrian Rosebrock April 24, 2017 at 9:48 am #

          You would need to use the dlib library. This example demonstrates how to train a custom shape predictor.

  14. carlos julio pardo April 28, 2017 at 5:44 pm #

    Hi …how can i set up Dlib on visualstudio 2012 or other version?

    • Adrian Rosebrock May 1, 2017 at 1:48 pm #

      Hi Carlos — I only cover how to install dlib on Linux, macOS, and Raspbian on this blog. Setting up dlib on Windows or other Microsoft products is not covered. For that, I would suggest looking at the official dlib website.

  15. Thimira Amaratunga May 5, 2017 at 3:45 am #

    Hi Adrian,

    Thanks for another awesome tutorial.
    I noticed that you convert the image frame to grayscale before passing to the Dlib face detector. I’m assuming it was for speeding up the face detection process. Am I correct?

    If we pass a color image to the Dlib face detector, would it detect faces better while being slower (i.e. detection accuracy will be high because more feature data will be available), or have the Dlib face detector designed to work better with grayscale images?

    Thanks,

    • Adrian Rosebrock May 8, 2017 at 12:38 pm #

      It really depends on the underlying HOG + Linear SVM implementation. Dalal and Triggs (the authors of the original HOG paper) found that computing the gradient over multiple channels and taking the maximum response can increase detection accuracy. The problem is that you end up computing the gradient representation for each channel which makes the detection pipeline slower. In this case I like to explicitly convert to grayscale for speed.

  16. Ivan May 7, 2017 at 3:50 pm #

    Great stuff! How hard would it be to extract pose data (head pitch, yaw, & roll) from the features here? Thanks!

  17. Jyotsna May 18, 2017 at 1:31 am #

    I m getting error mentioned below when executing above program of real-time facial landmark detection:
    File “video_facial_landmarks.py”, line 22, in

    predictor = dlib.shape_predictor(args[“shape_predictor”])

    RuntimeError: Unable to open shape_predictor_68_face_landmarks.dat in ubuntu 16.04

    Please help.
    Thanks in advance

    • Adrian Rosebrock May 18, 2017 at 11:48 am #

      Make sure you use the “Downloads” section of this guide to download the source code + facial landmark predictor file. The issue is that you do not have this .dat file.

      • Yoko November 25, 2018 at 7:57 am #

        Hi Adrian!

        I downloaded the .dat file (the zip’s unzipped and everything) and yet I am still receiving the same error 🙁 How should I go about this?

        • Adrian Rosebrock November 25, 2018 at 8:48 am #

          Your path to the input file is incorrect. Path make sure you are correctly specifying the path to it via your command line arguments. If you are new to using command line arguments make sure you read this tutorial first.

  18. Alex May 22, 2017 at 2:29 am #

    Hi Adrian,

    Thanks a lot for your amazing work !

    I wondered if you have any idea regarding what would be the best way to go in order to create a legs detection algorithm ?

    Thanks for your help,
    Alex

  19. Vinod Ramamoorthy May 25, 2017 at 9:49 am #

    Hi Adrian – thanks for your great work – kudos 🙂

    I’m stuck and I thought I’ll ask for your help.

    Installed opencv and dlib successfully.
    site-packages are symliked to my virtualenv as well.

    when I run (i’m working within my virtualenv at this point and the files are located within /Documents/cv)

    python video_facial_landmarks.py \
    –shape-predictor shape_predictor_68_face_landmarks.dat \
    –picamera 1

    throws the following error

    File “/home/pi/.virtualenvs/facecam/lib/python3.4/site-packages/imutils/video/pivideostream.py”, line 2, in from picamera.array import PiRGBArray
    ImportError: No module named ‘picamera’

    • Vinod Ramamoorthy May 25, 2017 at 9:57 am #

      Update – figured out the problem here –

      forgot this~
      pip install “picamera[array]”

      but have another problem at hand~

      (Frame:1158): Gtk-warning **:cannot open display:

      • Vinod Ramamoorthy May 25, 2017 at 10:10 am #

        update – was running the script from CLI

        Once I rebooted the system with pixel GUI enabled – it worked 🙂

        • Adrian Rosebrock May 28, 2017 at 1:23 am #

          Congrats on resolving the issue Vinod 🙂

  20. Wenliangh Wang May 30, 2017 at 2:38 pm #

    Hi Adrian,

    I referred your above article, it works perfectly fine on my face image as well.

    I wanted to use this facial landmark detected image and match with it my other image which I have placed in train folder (Face Recognition)

    How can I effectively match these landmark features between two images?. I tried your existing LBP recognition code (Local Binary Patterns with Python & OpenCV) but the results are poor.

    https://www.pyimagesearch.com/2015/12/07/local-binary-patterns-with-python-opencv/

    Do you have any opencv code which can effectively recognize images based on facial landmarks?

    I would be very grateful if you could help me in this regard and if I could successfully complete this assignment, I will surely enroll myself for pyimagesearch gurus.

    Regards
    Wenliangh Wang

    • Adrian Rosebrock May 31, 2017 at 1:10 pm #

      Facial landmarks are not used for face recognition. Instead, you would use algorithms such as Eigenfaces and LBPs for face recognition. These face recognition algorithms are covered inside the PyImageSearch Gurus course.

  21. leminhson June 5, 2017 at 1:13 am #

    Can we use this for detection of hand landmarks ?

    • Adrian Rosebrock June 6, 2017 at 12:04 pm #

      You would need to train a custom landmark predictor for hand landmarks, but yes, it is possible.

  22. Rebecca June 14, 2017 at 4:24 pm #

    Hi, I’m trying to run the program on mac, but it is giving me an illegal instruction: 4 error. I’m not accustomed to mac and I don’t know if there’s a way to fix this. Thanks!

  23. DJ June 21, 2017 at 8:00 pm #

    Hello Adrian

    I would like to ask on where to place the “shape_predictor_68_face_landmarks.DAT” file because upon running the “video_facial_landmarks.py” file im getting the error of

    usage: video_facial_landmarks.py [-h] -p SHAPE_PREDICTOR [-r PICAMERA]
    video_facial_landmarks.py: error: argument -p/–shape-predictor is required

    • Adrian Rosebrock June 22, 2017 at 9:27 am #

      You can place the .dat file anywhere you like on your machine. You need to specify the path to it via command line arguments. I would suggest you read up on command line arguments and how they work before proceeding.

  24. Igor June 26, 2017 at 1:59 pm #

    Hi Adrian, can you explain little bit more, what I should to change to use recorded video file instead of live cam?

    • Adrian Rosebrock June 27, 2017 at 6:21 am #

      If you’re using a recorded video you can simply specify the file path to cv2.VideoCapture or use the FileVideoStream class.

      • Igor August 17, 2017 at 10:29 am #

        I just simply changed vs = VideoStream(usePiCamera=args[“picamera”] > 0).start() to vs = FileVideoStream(“rt.wmv”).start() but it doesnt help ( Can you explait what to do little bit more? Thank you.

        • Adrian Rosebrock August 17, 2017 at 10:32 am #

          I’m not sure what you mean by “it doesn’t help”. Are you getting an error message? Does the script automatically exit? The more insight you can provide the better chance that myself or other PyImageSearch readers will be able to help you.

          • Igor August 24, 2017 at 7:14 am #

            I just see this message:
            [INFO] camera sensor warming up…
            And no autoexit.

          • Adrian Rosebrock August 24, 2017 at 3:29 pm #

            I would suggest double-checking that OpenCV has access to your webcam. It sounds like OpenCV cannot access your webcam and gets stuck in a loop.

          • Igor August 29, 2017 at 6:57 am #

            I solve the problem, just reinstalled ubuntu opencv dlib and also add 8gb ram to the project. And it works great! Thank you Adrian for great job!

          • Adrian Rosebrock August 31, 2017 at 8:38 am #

            Congrats on resolving the issue, Igor. Nice job.

  25. Luciano July 3, 2017 at 10:25 am #

    Hi Adrian, I keep getting “corrupt JPEG data: extraneous bytes” error message when I run the program.
    The program seems to be working fine but I keep getting this error
    Thanks!

    • Adrian Rosebrock July 5, 2017 at 6:13 am #

      If you’re getting that error message when trying to read frames from your webcam, either (1) there is a problem with your webcam or (2) you’re missing some video codecs. Without access to your machine, I don’t know what the true problem is. Unfortunately, you might want to consider re-installing your OS and compiling OpenCV from scratch.

  26. Simon Legrand August 3, 2017 at 6:47 am #

    Hi Adrian. First of all, thank you so much for this invaluable resource and very detailed and well explained tutorials.

    I’m currently running this configuration on a raspberry pi 3. Everything works as expected with the exception of a very low frame-rate. I get about 1 frame every 2 seconds.

    Is this a limitation of the pi 3 hardware or is there a chance that if I put a bit of effort into optimising what I’m doing I could get 12fps or above?

    • Adrian Rosebrock August 4, 2017 at 6:53 am #

      It’s a limitation of the Raspberry Pi hardware. Swap out Haar cascades for the the default HOG + Linear SVM face detector. Skip frames are also useful. I’ll be covering how to do both of these in a future blog post.

      • simon legrand August 10, 2017 at 3:53 pm #

        Great thank you!

      • simon legrand August 10, 2017 at 5:05 pm #

        I was under the impression that we were already using HOG + Linear SVM in this one.

        or am I misunderstanding what you mean to swap out?

        • Adrian Rosebrock August 14, 2017 at 1:27 pm #

          My apologies, I meant to swap the order in my previous comment. You are correct, this post already uses HOG + Linear SVM provided by dlib. Swap out the HOG + Linear SVM for the face detection Haar cascades provided by OpenCV. They will be less accurate, but faster.

  27. fariborz September 3, 2017 at 7:22 am #

    Hello
    Thank you for the tutorials
    I did it but the speed of the camera frames is very slow and there is a lot of delay
    What should I do?
    In addition,
    I used the raspberry camera

  28. Tiffany September 8, 2017 at 5:47 am #

    Excellent tutorials, really appreciate it. Do you know if it would be possible to implement this on android ?

    • Adrian Rosebrock September 11, 2017 at 9:26 am #

      Yes, you can certainly apply this method to Android or iOS; however, I am not an Android developer. If you are looking for an Android + OpenCV developer I would suggest posting your project on PyImageJobs.

  29. ashish September 17, 2017 at 9:20 am #

    Please help me to install SciPy and Scikit image installation in windows 10 64-bit

  30. Laura W September 24, 2017 at 2:17 am #

    Hello Adrian, I execute
    /home/pi/Desktop/video_facial_landmarks.py –shape predictor shape_predictor_68_face_landmarks.dat –picamera 1
    and
    /home/pi/Desktop/video_facial_landmarks.py –shape predictor /home/pi/Desktop/shape_predictor_68_face_landmarks.dat –picamera 1

    But I’m getting this error (In both cases):
    .
    usage: video_facial_landmarks.py [-h] -p SHAPE_PREDICTOR [-r PICAMERA]
    video_facial_landmarks.py:error: unrecognized arguments: shape_predictor_68_face_landmarks.dat
    .
    .
    I have the files (.dat and .py) saved in /home/pi/Desktop/ . The same files we can get from the download section
    .
    I love your tutorials
    Thanks!!!!

    • Laura W September 24, 2017 at 2:52 am #

      Sorry, I was wrong, I corrected it: I was writing –shape predictor, it should be –shape-predictor, but now, when I execute:
      python /home/pi/Desktop/video_facial_landmarks.py –shape-predictor /home/pi/Desktop/shape_predictor_68_face_landmarks.dat
      It shows me
      [INFO] loading facial landmark predictor…
      [INFO] camera sensor warming up…
      Traceback (most recent call last):

      ImportError: No module named ‘picamera’

      I dont know why
      I give up 🙁

      • Adrian Rosebrock September 24, 2017 at 7:09 am #

        You haven’t installed the picamera module you:

        $ pip install picamera

        If you are using the “cv” Python virtual environment access it first and then install:

        • Laura W September 24, 2017 at 10:53 am #

          Ok, thank you, I thought it wasnt necessary because I made some examples that uses a script to access the raspberry camera

          • Adrian Rosebrock September 24, 2017 at 12:25 pm #

            No, you need the picamera module to use the VideoStream class and the Raspberry Pi camera module. As long as you have picamera installed you’ll be good to go.

  31. bash September 24, 2017 at 3:11 pm #

    Hello Adrian ! thank you for this great tutorial 🙂

    I’m having an issue every time i run the sketch i get this error even though i have upgraded imutils to the latest version . Thank you in advance if you figure out whats the issue 🙂

    File “/home/pi/Desktop/video_facial_landmarks.py”, line 6, in
    from imutils.video import VideoStream
    ImportError: No module named ‘imutils’

    • Adrian Rosebrock September 26, 2017 at 8:35 am #

      You need to install the imutils library:

      $ pip install imutils

      If you are using Python virtual environments:

  32. Todor Arnaudov September 25, 2017 at 9:58 am #

    Hi Adrian, the dlib and the new deep learning series are great!

    I’d like to report for a strange behavior regarding the threaded part, using Python 3.6.1 – the console window hangs after quitting. Last lines are executed, the camera turns off, but control is not returned to the console.

    – It hangs also when there is an exception after VideoStream.start()

    – Putting an explicit “exit()” after the last line doesn’t help.

    – Strangely *sometimes*/rarely it did end smoothly…

    Reverting to conventional sampling fixed it for me:

    vs = cv2.VideoCapture(0)

    ret, frame = vs.read()

    Best Regards

    • Adrian Rosebrock September 26, 2017 at 8:24 am #

      This does indeed sound like a threading problem. Thanks for sharing, Todor!

      • Todor Arnaudov September 27, 2017 at 7:09 am #

        I’ve just checked the drowsiness detector and the same thing happened, so I tried to debug it.

        The bug was gone after changing WebcamVideoStream’s thread daemon property to False:

        def start(self):
        # start the thread to read frames from the video stream
        t = Thread(target=self.update, args=())
        t.daemon = False

        For the record I’m using Windows 10… :-]

  33. Pavel September 28, 2017 at 5:01 am #

    Good staff, Adrian!

    Is it possible to detect facial landmarks from a part of image?
    For example, how to get a landmarks for lip of the bottom part of the face?

    • Adrian Rosebrock September 28, 2017 at 8:57 am #

      Provided you can detect the entire face you can use array slices to extract arbitrary points. See this post for more details.

  34. Ofer October 4, 2017 at 6:51 pm #

    anyone else having problem with detecting wide open mouths? the mouth points are way off

  35. Shaun October 25, 2017 at 1:10 am #

    Hi Adrian

    When you read from VideoStream, what is the exact format of the “frame” ? (@ line 35)
    Can we temporarily store this as png or jpeg format ?

  36. kELLY October 29, 2017 at 1:43 pm #

    Thank you so much Adrian–
    Everything goes smoothly but then it opens a frame that is blank and in the terminal prints: select timeout numerous times
    —–
    —Please HELP!!

    • Adrian Rosebrock October 31, 2017 at 7:58 am #

      It sounds like Python/OpenCV is having trouble accessing your video stream. What type of camera are you using? Are you using a normal computer? Or a Raspberry Pi?

      • kELLY November 4, 2017 at 4:33 am #

        Im using a normal computer not rasberry pi. Anyone who can help please…

        • Adrian Rosebrock November 6, 2017 at 10:42 am #

          If you are using a normal computer it sounds like OpenCV cannot access your webcam. Can other software on your computer properly access your webcam?

  37. Martin Safer November 19, 2017 at 8:37 am #

    Hello, very good post, but I have a problem with imutils, when I put it in the example it tells me that there is no module called imutils, and probe with pip freeze and I have version 0.4.3, what can I do?

    • Adrian Rosebrock November 20, 2017 at 4:01 pm #

      That is indeed very strange. It sounds like your “python” version and “pip” version do not match up. Make sure the “pip” you used to install imutils matches your Python version.

  38. Ad February 13, 2018 at 2:59 pm #

    Hello Adrian! First of all, thank you so much for your tutorials, they are great!
    Second, I want to ask you for some help: I’m having issues running the code since the follow error appears when I run it.

    File “/home/ad/.virtualenvs/cv/local/lib/python2.7/site-packages/imutils/convenience.py”, line 69, in resize
    (h, w) = image.shape[:2]
    AttributeError: ‘NoneType’ object has no attribute ‘shape’

    What can I do?
    Thank you so much in advance!

    • Adrian Rosebrock February 18, 2018 at 10:18 am #

      It looks like OpenCV cannot access your webcam and is returning “None” for the frame. I discuss these errors and how resolve them in this post.

  39. Saverio February 22, 2018 at 9:47 am #

    Hi Adrian,
    is there a way (model) we can use to perform a face detection with movidius?
    I’m trying both haar and hog (dlib) but on the pi we can’t reach a good fps.

    • Adrian Rosebrock February 22, 2018 at 11:05 am #

      You would need a Caffe/TensorFlow model that performs face detection. Provided you have one, yes, you can run it on the Movidius. A good starting point would be this post. Additionally, I discuss face detection speed/accuracy tradeoff on this post.

  40. Matheus Bernardi February 26, 2018 at 10:00 am #

    Hello Adrian!

    First of all, thank you for all the content in the blog, you helped me a lot! I need to track the pupils of the both eyes, do you have any tips for this implemention? I’m really stuck on this part of the algorithm.

    Thank you again for sharing all your knowledge!

    • Adrian Rosebrock February 26, 2018 at 1:41 pm #

      I do not have any tutorials on pupil detection but I know some PyImageSearch readers have had good luck with this tutorial.

  41. Matheus Bernardi February 27, 2018 at 10:44 am #

    Thank you Adrian, have a great week

    • Adrian Rosebrock February 27, 2018 at 11:20 am #

      Thank you Matheus 🙂

  42. Aysenur March 14, 2018 at 9:22 am #

    Hi Adrian
    I got this mistake. Can you help me ?

    usage: detect_face_parts.py [-h] -p SHAPE_PREDICTOR -i IMAGE
    detect_face_parts.py: error: the following arguments are required: -p/–shape-predictor, -i/–image

    • Adrian Rosebrock March 14, 2018 at 12:26 pm #

      Please see this post on command line arguments. It’s an easy fix but make sure you read the post to understand how to use command line arguments.

  43. Rohith K March 25, 2018 at 4:13 am #

    Hey man,
    I’m getting an error like,
    AttributeError: module ‘cv2’ has no attribute ‘INTER_AREA’

    Please help me…
    I desperately need this…
    Please reply soon

    • Adrian Rosebrock March 27, 2018 at 6:24 am #

      Hey Rohith — it’s odd that your “cv2” module does not have the “INTER_AREA” attribute. How did you install OpenCV? And which OpenCV version are you using?

  44. César Quiroz April 2, 2018 at 2:08 pm #

    Hi Adrian,

    I don’t know what it is, but i got this error and in internet i didn’t find any solution yet.

    [INFO] loading facial landmark predictor…
    [INFO] camera sensor warming up…
    ASSERT: “false” in file qasciikey.cpp, line 495
    Abortado (imagem do núcleo gravada) – this last part is because i’m Brazilian.
    The code works fine, but after 2 seconds…

    Could you help me please?

    Thanks

  45. Rohin April 9, 2018 at 5:21 am #

    ap.add_argument(“-p”, “–shape-predictor”, required=True,
    help=”path to facial landmark predictor”)
    ap.add_argument(“-r”, “–picamera”, type=int, default=-1,
    help=”whether or not the Raspberry Pi camera should be used”)
    args = vars(ap.parse_args())
    what changes i should make in this code?

  46. Celal May 25, 2018 at 9:40 am #

    Hi Adrian,
    Thanks for such a great work!
    I tried to put different face coordinates with another face detection method to the “rect”. But the face landmarks are not working properly when I rotate my face a bit. They slip a lot and get very shaky. How can I solve it?

    • Adrian Rosebrock May 28, 2018 at 9:55 am #

      I’m not sure what you mean by “put different face coordinates with another face detection method”. Could you elaborate?

      • Celal May 29, 2018 at 5:02 am #

        I mean I used OpenCV library to detect faces, not the dlib library. Than I converted the coordinates of detected faces to the dlib rectangle class. But I couldn’t get satisfying result. I don’t have any problem with detecting faces however, as mentioned above, landmarks are not overlapping with the correct face landmarks when I rotate my face a bit.
        I guess the ‘predictor’ method is suffering from detected face rectangle coordinates.

        • Adrian Rosebrock May 31, 2018 at 5:24 am #

          Got it. Are you sure you are converting the coordinates correctly? Take a look at this blog post where I use an OpenCV Haar cascade to detect faces and then convert them to a dlib rectangle object.

  47. Aman Gupta June 14, 2018 at 2:38 am #

    Hello Sir,
    You have done excellent work here. Thank You so much. Your code is very easy to understand.
    I have just had a question .. What exactly is landmark predictor file here. Did you code it (and is there a blog for that) or is it dlib’s standard library file??.

    • Adrian Rosebrock June 15, 2018 at 12:33 pm #

      It was trained using the dlib library and provided by the dlib creator, Davis King.

  48. ethan August 16, 2018 at 5:32 pm #

    Hi Adrian,

    Thanks a lot for your amazing work !

    I have just had a question ..How to enlarge the output frame?

    • Adrian Rosebrock August 17, 2018 at 7:18 am #

      You can use the cv2.resize or imutils.resize function to resize images/frames. If you’re new to the world of computer vision and image processing I would suggest working through Practical Python and OpenCV so you can learn the fundamentals very quickly. Be sure to check it out, I think it will really help you!

  49. Hermit September 1, 2018 at 8:51 am #

    Hi Adrian, I have a question can we track all the 68 points of the facial landmark after detection to increase the fps count.

    • Adrian Rosebrock September 5, 2018 at 9:09 am #

      I’m not sure what you mean by tracking them in between frames. Running the actual facial landmark detector after the face has been detected is super fast. You may want to skip face detection and instead apply a Kalman filter, that could help.

  50. Vaisakh September 10, 2018 at 10:09 am #

    Hi Adrian. Great job. I am working on a robot project. i need to turn the robot left and right directions if we turn our head like that. how can i add head pose into this tutorial ?

    Thanks

    • Adrian Rosebrock September 11, 2018 at 8:11 am #

      Hey Vaisakh — I don’t have any tutorials related to moving an arm/servo based on real-world coordinates but I will try to do one in the future.

  51. Kyungwon Lee October 8, 2018 at 1:53 am #

    Hello! Adrian. My name is Kyungwon Lee. I always appreciate your post. This post, as well as other posts, really helped me a lot. I am studying to prevent drowsiness by looking at your post now. In your post you are only judging your drowsiness with your eyes, but if I can not find your face first, I will let you know with a buzzer. So I came to refer to this post. But I do not know if I can refer to some posts. So I was asked this question. First, I think you should use conditional statements to make the buzzer sound when you can not find your face. But I do not know which conditional statement to use. Please teach me. Using a translator can make strange words. Say thank you again.

    • Adrian Rosebrock October 8, 2018 at 9:33 am #

      Hi Kyungwon, thanks for the comment. I wouldn’t sound the buzzer if a face cannot be found immediately. You may want to consider only sounding the buzzer after N frames (where N is a pre-defined value). Otherwise, the buzzer will sound every time the face detector fails to find a face which can happen intermittently depending on viewing angle, lighting conditions, etc.

  52. faraz November 1, 2018 at 4:02 am #

    Thank you Dr Adrian for making such contributions towards free education…

    • Adrian Rosebrock November 2, 2018 at 7:20 am #

      Thanks Faraz 🙂

  53. Jens November 16, 2018 at 9:24 am #

    Hi Adrian,

    super grateful for the posts, they have helped me a lot ! I am using the landmark detection in live video where the common situation is not a frontal face one. Do you know of any pre-trained face landmark detectors that can adapt to those kinds of situations, i.e. both frontal, profile and tilted?

Trackbacks/Pingbacks

  1. Eye blink detection with OpenCV, Python, and dlib - PyImageSearch - April 24, 2017

    […] last week’s blog post, I demonstrated how to perform facial landmark detection in real-time in video […]

Leave a Reply