Liveness Detection with OpenCV

In this tutorial, you will learn how to perform liveness detection with OpenCV. You will create a liveness detector capable of spotting fake faces and performing anti-face spoofing in face recognition systems.

Over the past year, I have authored a number of face recognition tutorials, including:

However, a common question I get asked over email and in the comments sections of the face recognition posts is:

How do I spot real versus fake faces?

Consider what would happen if a nefarious user tried to purposely circumvent your face recognition system.

Such a user could try to hold up a photo of another person. Maybe they even have a photo or video on their smartphone that they could hold up to the camera responsible for performing face recognition (such as in the image at the top of this post).

In those situations it’s entirely possible for the face held up to the camera to be correctly recognized…but ultimately leading to an unauthorized user bypassing your face recognition system!

How would you go about spotting these “fake” versus “real/legitimate” faces? How could you apply anti-face spoofing algorithms into your facial recognition applications?

The answer is to apply liveness detection with OpenCV which is exactly what I’ll be covering today.

To learn how to incorporate liveness detection with OpenCV into your own face recognition systems, just keep reading!

Looking for the source code to this post?
Jump right to the downloads section.

Liveness Detection with OpenCV

In the first part of this tutorial, we’ll discuss liveness detection, including what it is and why we need it to improve our face recognition systems.

From there we’ll review the dataset we’ll be using to perform liveness detection, including:

  • How to build to a dataset for liveness detection
  • Our example real versus fake face images

We’ll also review our project structure for the liveness detector project as well.

In order to create the liveness detector, we’ll be training a deep neural network capable of distinguishing between real versus fake faces.

We’ll, therefore, need to:

  1. Build the image dataset itself.
  2. Implement a CNN capable of performing liveness detector (we’ll call this network “LivenessNet”).
  3. Train the liveness detector network.
  4. Create a Python + OpenCV script capable of taking our trained liveness detector model and apply it to real-time video.

Let’s go ahead and get started!

What is liveness detection and why do we need it?

Figure 1: Liveness detection with OpenCV. On the left is a live (real) video of me and on the right you can see I am holding my iPhone (fake/spoofed).

Face recognition systems are becoming more prevalent than ever. From face recognition on your iPhone/smartphone, to face recognition for mass surveillance in China, face recognition systems are being utilized everywhere.

However, face recognition systems are easily fooled by “spoofing” and “non-real” faces.

Face recognition systems can be circumvented simply by holding up a photo of a person (whether printed, on a smartphone, etc.) to the face recognition camera.

In order to make face recognition systems more secure, we need to be able to detect such fake/non-real faces — liveness detection is the term used to refer to such algorithms.

There are a number of approaches to liveness detection, including:

  • Texture analysis, including computing Local Binary Patterns (LBPs) over face regions and using an SVM to classify the faces as real or spoofed.
  • Frequency analysis, such as examining the Fourier domain of the face.
  • Variable focusing analysis, such as examining the variation of pixel values between two consecutive frames.
  • Heuristic-based algorithms, including eye movement, lip movement, and blink detection. These set of algorithms attempt to track eye movement and blinks to ensure the user is not holding up a photo of another person (since a photo will not blink or move its lips).
  • Optical Flow algorithms, namely examining the differences and properties of optical flow generated from 3D objects and 2D planes.
  • 3D face shape, similar to what is used on Apple’s iPhone face recognition system, enabling the face recognition system to distinguish between real faces and printouts/photos/images of another person.
  • Combinations of the above, enabling a face recognition system engineer to pick and choose the liveness detections models appropriate for their particular application.

A full review of liveness detection algorithms can be found in Chakraborty and Das’ 2014 paper, An Overview of Face liveness Detection.

For the purposes of today’s tutorial, we’ll be treating liveness detection as a binary classification problem.

Given an input image, we’ll train a Convolutional Neural Network capable of distinguishing real faces from fake/spoofed faces.

But before we get to training our liveness detection model, let’s first examine our dataset.

Our liveness detection videos

Figure 2: An example of gathering real versus fake/spoofed faces. The video on the left is a legitimate recording of my face. The video on the right is that same video played back while my laptop records it.

To keep our example straightforward, the liveness detector we are building in this blog post will focus on distinguishing real faces versus spoofed faces on a screen.

This algorithm can easily be extended to other types of spoofed faces, including print outs, high-resolution prints, etc.

In order to build the liveness detection dataset, I:

  1. Took my iPhone and put it in portrait/selfie mode.
  2. Recorded a ~25-second video of myself walking around my office.
  3. Replayed the same 25-second video, this time facing my iPhone towards my desktop where I recorded the video replaying.
  4. This resulted in two example videos, one for “real” faces and another for “fake/spoofed” faces.
  5. Finally, I applied face detection to both sets of videos to extract individual face ROIs for both classes.

I have provided you with both my real and fake video files in the “Downloads” section of the post.

You can use these videos as a starting point for your dataset but I would recommend gathering more data to help make your liveness detector more robust and accurate.

With testing, I determined that the model is slightly biased towards my own face which makes sense because that is all the model was trained on. And furthermore, since I am white/caucasian I wouldn’t expect this same dataset to work as well with other skin tones.

Ideally, you would train a model with faces of multiple people and include faces of multiple ethnicities.  Be sure to refer to the “Limitations and further work section below for additional suggestions on improving your liveness detection models.

In the rest of the tutorial, you will learn how to take the dataset I recorded it and turn it into an actual liveness detector with OpenCV and deep learning.

Project structure

Go ahead and grab the code, dataset, and liveness model using the “Downloads” section of this post and then unzip the archive.

Once you navigate into the project directory, you’ll notice the following structure:

There are four main directories inside our project:

  • dataset/ : Our dataset directory consists of two classes of images:
    • Fake images of me from a camera aimed at my screen while playing a video of my face.
    • Real images of me captured from a selfie video with my phone.
  • face_detector/ : Consists of our pretrained Caffe face detector to locate face ROIs.
  • pyimagesearch/ : This module contains our LivenessNet class.
  • videos/ : I’ve provided two input videos for training our LivenessNet classifier.

Today we’ll be reviewing three Python scripts in detail. By the end of the post you’ll be able to run them on your own data and input video feeds as well. In order of appearance in this tutorial, the three scripts are:

  1. gather_examples.py : This script grabs face ROIs from input video files and helps us to create a deep learning face liveness dataset.
  2. train_liveness.py : As the filename indicates, this script will train our LivenessNet classifier. We’ll use Keras and TensorFlow to train the model. The training process results in a few files:
    • le.pickle : Our class label encoder.
    • liveness.model : Our serialized Keras model which detects face liveness.
    • plot.png : The training history plot shows accuracy and loss curves so we can assess our model (i.e. over/underfitting).
  3. liveness_demo.py : Our demonstration script will fire up your webcam to grab frames to conduct face liveness detection in real-time.

Detecting and extracting face ROIs from our training (video) dataset

Figure 3: Detecting face ROIs in video for the purposes of building a liveness detection dataset.

Now that we’ve had a chance to review both our initial dataset and project structure, let’s see how we can extract both real and fake face images from our input videos.

The end goal if this script will be to populate two directories:

  1. dataset/fake/: Contains face ROIs from the fake.mp4 file
  2. dataset/real/: Holds face ROIs from the real.mov file.

Given these frames, we’ll later train a deep learning-based liveness detector on the images.

Open up the gather_examples.py file and insert the following code:

Lines 2-5 import our required packages. This script only requires OpenCV and NumPy in addition to built-in Python modules.

From there Lines 8-19 parse our command line arguments:

  • --input : The path to our input video file.
  • --output : The path to the output directory where each of the cropped faces will be stored.
  • --detector : The path to the face detector. We’ll be using OpenCV’s deep learning face detector. This Caffe model is included with today’s “Downloads” for your convenience.
  • --confidence : The minimum probability to filter weak face detections. By default, this value is 50%.
  • --skip : We don’t need to detect and store every image because adjacent frames will be similar. Instead, we’ll skip N frames between detections. You can alter the default of 16 using this argument.

Let’s go ahead and load the face detector and initialize our video stream:

Lines 23-26 load OpenCV’s deep learning face detector.

From there we open our video stream on Line 30.

We also initialize two variables for the number of frames read as well as the number of frames saved while our loop executes (Lines 31 and 32).

Let’s go ahead create a loop to process the frames:

Our while  loop begins on Lines 35.

From there we grab and verify a frame  (Lines 37-42).

At this point, since we’ve read a frame , we’ll increment our  read  counter (Line 48). If we are skipping this particular frame, we’ll continue without further processing (Lines 48 and 49).

Let’s go ahead and detect faces:

In order to perform face detection, we need to create a blob from the image (Lines 53 and 54). This blob  has a 300×300 width and height to accommodate our Caffe face detector. Scaling the bounding boxes will be necessary later, so Line 52, grabs the frame dimensions.

Lines 58 and 59 perform a forward  pass of the blob  through the deep learning face detector.

Our script makes the assumption that there is only one face in each frame of the video (Lines 62-65). This helps prevent false positives. If you’re working with a video containing more than one face, I recommend that you adjust the logic accordingly.

Thus, Line 65 grabs the highest probability face detection index. Line 66 extracts the confidence of the detection using the index.

Let’s filter weak detections and write the face ROI to disk:

Line 71 ensures that our face detection ROI meets the minimum threshold to reduce false positives.

From there we extract the face ROI bounding box  coordinates and face ROI itself (Lines 74-76).

We generate a path + filename for the face ROI and write it to disk on Lines 79-81. At this point, we can increment the number of saved  faces.

Once processing is complete, we’ll perform cleanup on Lines 86 and 87.

Building our liveness detection image dataset

Figure 4: Our OpenCV face liveness detection dataset. We’ll use Keras and OpenCV to train and demo a liveness model.

Now that we’ve implemented the gather_examples.py script, let’s put it to work.

Make sure you use the “Downloads” section of this tutorial to grab the source code and example input videos.

From there, open up a terminal and execute the following command to extract faces for our “fake/spoofed” class:

Similarly, we can do the same for the “real” class as well:

Since the “real” video file is longer than the “fake” video file, we’ll use a longer skip frames value to help balance the number of output face ROIs for each class.

After executing the scripts you should have the following image counts:

  • Fake: 150 images
  • Real: 161 images
  • Total: 311 images

Implementing “LivenessNet”, our deep learning liveness detector

Figure 5: Deep learning architecture for LivenessNet, a CNN designed to detect face liveness in images and videos.

The next step is to implement “LivenessNet”, our deep learning-based liveness detector.

At the core, LivenessNet  is actually just a simple Convolutional Neural Network.

We’ll be purposely keeping this network as shallow and with as few parameters as possible for two reasons:

  1. To reduce the chances of overfitting on our small dataset.
  2. To ensure our liveness detector is fast, capable of running in real-time (even on resource-constrained devices, such as the Raspberry Pi).

Let’s implement LivenessNet now — open up livenessnet.py and insert the following code:

All of our imports are from Keras (Lines 2-10). For an in-depth review of each of these layers and functions, be sure to refer to Deep Learning for Computer Vision with Python.

Our LivenessNet  class is defined on Line 12. It consists of one static method, build  (Line 14). The build  method accepts four parameters:

  • width : How wide the image/volume is.
  • height : How tall the image is.
  • depth : The number of channels for the image (in this case 3 since we’ll be working with RGB images).
  • classes : The number of classes. We have two total classes: “real” and “fake”.

Our model  is initialized on Line 17.

The inputShape  to our model is defined on Line 18 while channel ordering is determined on Lines 23-25.

Let’s begin adding layers to our CNN:

Our CNN exhibits VGGNet-esque qualities. It is very shallow with only a few learned filters. Ideally, we won’t need a deep network to distinguish between real and spoofed faces.

The first CONV => RELU => CONV => RELU => POOL  layer set is specified on Lines 28-36 where batch normalization and dropout are also added.

Another CONV => RELU => CONV => RELU => POOL  layer set is appended on Lines 39-46.

Finally, we’ll add our FC => RELU  layers:

Lines 49-57 consist of fully connected and ReLU activated layers with a softmax classifier head.

The model is returned to the training script on Line 60.,

Creating the liveness detector training script

Figure 6: The process of training LivenessNet. Using both “real” and “spoofed/fake” images as our dataset, we can train a liveness detection model with OpenCV, Keras, and deep learning.

Given our dataset of real/spoofed images as well as our implementation of LivenessNet, we are now ready to train the network.

Open up the train_liveness.py file and insert the following code:

Our face liveness training script consists of a number of imports (Lines 2-19). Let’s review them now:

  • matplotlib : Used to generate a training plot. We specify the "Agg"  backend so we can easily save our plot to disk on Line 3.
  • LivenessNet : The liveness CNN that we defined in the previous section.
  • train_test_split : A function from scikit-learn which constructs splits of our data for training and testing.
  • classification_report : Also from scikit-learn, this tool will generate a brief statistical report on our model’s performance.
  • ImageDataGenerator : Used for performing data augmentation, providing us with batches of randomly mutated images.
  • Adam : An optimizer that worked well for this model. (alternatives include SGD, RMSprop, etc.).
  • paths : From my imutils package, this module will help us to gather the paths to all of our image files on disk.
  • pyplot : Used to generate a nice training plot.
  • numpy : A numerical processing library for Python. It is an OpenCV requirement as well.
  • argparse : For processing command line arguments.
  • pickle : Used to serialize our label encoder to disk.
  • cv2 : Our OpenCV bindings.
  • os : This module can do quite a lot, but we’ll just be using it for it’s operating system path separator.

That was a mouthful, but now that you know what the imports are for, reviewing the rest of the script should be more straightforward.

This script accepts four command line arguments:

  • --dataset : The path to the input dataset. Earlier in the post we created the dataset with the gather_examples.py  script.
  • --model : Our script will generate an output model file — here you supply the path to it.
  • --le : The path to our output serialized label encoder file also needs to be supplied.
  • --plot : The training script will generate a plot. If you wish to override the default value of "plot.png" , you should specify this value on the command line.

This next code block will perform a number of initializations and build our data:

Training parameters including initial learning rate, batch size, and number of epochs are set on Lines 35-37.

From there, our imagePaths  are grabbed. We also initialize two lists to hold our data  and class labels  (Lines 42-44).

The loop on Lines 46-55 builds our data  and labels  lists. The data  consists of our images which are loaded and resized to be 32×32 pixels. Each image has a corresponding label stored in the labels  list.

All pixel intensities are scaled to the range [0, 1] while the list is made into a NumPy array via Line 59.

Now let’s encode our labels and partition our data:

Lines 63-65 one-hot encode the labels.

We utilize scikit-learn to partition our data — 75% is used for training while 25% is reserved for testing (Lines 69 and 70).

Next, we’ll initialize our data augmentation object and compile + train our face liveness model:

Lines 73-75 construct a data augmentation object which will generate images with random rotations, zooms, shifts, shears, and flips. To read more about data augmentation, read my previous blog post.

Our LivenessNet  model is built and compiled on Lines 79-83.

We then commence training on Lines 87-89. This process will be relatively quick considering our shallow network and small dataset.

Once the model is trained we can evaluate the results and generate a training plot:

Predictions are made on the testing set (Line 93). From there a classification_report  is generated and printed to the terminal (Lines 94 and 95).

The LivenessNet  model is serialized to disk along with the label encoder on Lines 99-104.

The remaining Lines 107-117 generate a training history plot for later inspection.

Training our liveness detector

We are now ready to train our liveness detector.

Make sure you’ve used the “Downloads” section of the tutorial to download the source code and dataset — from, there execute the following command:

Figure 6: A plot of training a face liveness model using OpenCV, Keras, and deep learning.

As our results show, we are able to obtain 99% liveness detection accuracy on our validation set!

Putting the pieces together: Liveness detection with OpenCV

Figure 7: Face liveness detection with OpenCV and deep learning.

The final step is to combine all the pieces:

  1. We’ll access our webcam/video stream
  2. Apply face detection to each frame
  3. For each face detected, apply our liveness detector model

Open up the liveness_demo.py and insert the following code:

Lines 2-11 import our required packages. Notably, we’ll use

  • VideoStream  to access our camera feed.
  • img_to_array  so that our frame will be in a compatible array format.
  • load_model  to load our serialized Keras model.
  • imutils  for its convenience functions.
  • cv2  for our OpenCV bindings.

Let’s parse our command line arguments via Lines 14-23:

  • --model : The path to our pretrained Keras model for liveness detection.
  • --le : Our path to the label encoder.
  • --detector : The path to OpenCV’s deep learning face detector, used to find the face ROIs.
  • --confidence : The minimum probability threshold to filter out weak detections.

Now let’s go ahead an initialize the face detector, LivenessNet model + label encoder, and our video stream:

The OpenCV face detector is loaded via Lines 27-30.

From there we load our serialized, pretrained model ( LivenessNet ) and the label encoder (Lines 34 and 35).

Our VideoStream  object is instantiated and our camera is allowed two seconds to warm up (Lines 39 and 40).

At this point, it’s time to start looping over frames to detect real versus fake/spoofed faces:

Line 43 opens an infinite while  loop block where we begin by capturing + resizing individual frames (Lines 46 and 47).

After resizing, dimensions of the frame are grabbed so that we can later perform scaling (Line 50).

Using OpenCV’s blobFromImage function we generate a blob  (Lines 51 and 52) and then proceed to perform inference by passing it through the face detector network (Lines 56 and 57).

Now we’re ready for the fun part — liveness detection with OpenCV and deep learning:

On Line 60, we begin looping over face detections. Inside we:

  • Filter out weak detections (Lines 63-66).
  • Extract the face bounding box  coordinates and ensure they do not fall outside the dimensions of the frame (Lines 69-77).
  • Extract the face ROI and preprocess it in the same manner as our training data (Lines 81-85).
  • Employ our liveness detector model to determine if the face is “real” or “fake/spoofed” (Lines 89-91).
  • Line 91 is where you would insert your own code to perform face recognition but only on real images. The pseudo code would similar to if label == "real": run_face_reconition()  directly after Line 91).
  • Finally (for this demo), we draw the label  text and a rectangle  around the face (Lines 94-98).

Let’s display our results and clean up:

The ouput frame is displayed on each iteration of the loop while keypresses are captured (Lines 101-102). Whenever the user presses “q” (“quit”) we’ll break out of the loop and release pointers and close windows (Lines 105-110).

Deploying our liveness detector to real-time video

To follow along with our liveness detection demo make sure you have used the “Downloads” section of the blog post to download the source code and pre-trained liveness detection model.

From there, open up a terminal and execute the following command:

Here you can see that our liveness detector is successfully distinguishing real from fake/spoofed faces.

I have included a longer demo in the video below:

Limitations, improvements, and further work

The primary restriction of our liveness detector is really our limited dataset — there are only a total of 311 images (161 belonging to the “real” class and 150 to the “fake” class, respectively).

One of the first extensions to this work would be to simply gather additional training data, and more specifically, images/frames that are not of simply me or yourself.

Keep in mind that the example dataset used here today includes faces for only one person (myself). I am also white/caucasian — you should gather training faces for other ethnicities and skin tones as well.

Our liveness detector was only trained on spoof attacks from holding up a screen — it was not trained on images or photos that were printed out. Therefore, my third recommendation is to invest in additional image/face sources outside of simple screen recording playbacks.

Finally, I want to mention that there is no silver bullet to liveness detection.

Some of the best liveness detectors incorporate multiple methods of liveness detection (be sure to refer to the “What is liveness detection and why do we need it?” section above).

Take the time to consider and assess your own project, guidelines, and requirements — in some cases, all you may need is basic eye blink detection heuristics.

In other cases, you’ll need to combine deep learning-based liveness detection with other heuristics.

Don’t rush into face recognition and liveness detection — take the time and discipline to consider your own unique project requirements. Doing so will ensure you obtain better, more accurate results.

Summary

In this tutorial, you learned how to perform liveness detection with OpenCV.

Using this liveness detector you can now spot fake fakes and perform anti-face spoofing in your own face recognition systems.

To create our liveness detector we utilized OpenCV, Deep Learning, and Python.

The first step was to gather our real vs. fake dataset. To accomplish this task, we:

  1. First recorded a video of ourselves using our smartphone (i.e., “real” faces).
  2. Held our smartphone up to our laptop/desktop, replayed the same video, and then recorded the replaying using our webcam (i.e., “fake” faces).
  3. Applied face detection to both sets of videos to form our final liveness detection dataset.

After building our dataset we implemented, “LivenessNet”, a Keras + Deep Learning CNN.

This network is purposely shallow, ensuring that:

  1. We reduce the chances of overfitting on our small dataset.
  2. The model itself is capable of running in real-time (including on the Raspberry Pi).

Overall, our liveness detector was able to obtain 99% accuracy on our validation set.

To demonstrate the full liveness detection pipeline in action we created a Python + OpenCV script that loaded our liveness detector and applied it to real-time video streams.

As our demo showed, our liveness detector was capable of distinguishing between real and fake faces.

I hope you enjoyed today’s post on liveness detection with OpenCV.

To download the source code to this post and apply liveness detection to your own projects (plus be notified when future tutorials are published here on PyImageSearch), just enter your email address in the form below!

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , , , ,

39 Responses to Liveness Detection with OpenCV

  1. hamdi March 11, 2019 at 10:52 am #

    Hi Adrian.

    You are awesome . I want to hug you.

    • Adrian Rosebrock March 11, 2019 at 12:18 pm #

      Thanks Hamdi!

  2. Fabián González March 11, 2019 at 11:03 am #

    Hello Adrian, where I can talk to you in a more private way, to ask you for advice on a project I’m going to undertake.

    • Adrian Rosebrock March 11, 2019 at 12:18 pm #

      I would recommend the PyImageSearch Gurus course. I answer questions in the course, including private messages, daily. I prioritize the Gurus course forums and community over my inbox as well.

  3. David Bonn March 11, 2019 at 11:07 am #

    Thanks for another great tutorial!

    Using video to generate training data is a very promising approach for generating enormous labeled datasets rapidly. If you used youtube videos of tv news you could produce thousands of samples an hour, with a wide variety of faces, although generally under lighting conditions that could be described as “too good”.

    Using a similar camera to collect your training data as what you are going to use in the end application is also very helpful.

    Students of computer vision would do well to also acquire a collection of cheap USB web cameras for that very reason.

    Again, thanks for the tutorial.

    • Adrian Rosebrock March 11, 2019 at 12:20 pm #

      Thanks David, I’m glad you liked the tutorial. As you said, it would be easy utilize YouTube videos to gather additional data. I would suggest using two laptops:

      1. The first one responsible for playing a super long playlist of YouTube videos.
      2. The second laptop, aimed at the first, performing face detection on the screen of the first laptop.

      Bonus points if the lighting conditions of the room can automatically change as well.

      • David Bonn March 11, 2019 at 3:19 pm #

        That’s a great approach.

        One thing to be careful about when using video to generate training data is that you will end up with duplicate images, so it is important to use an image hash function at some point in your pipeline to remove those duplicates.

        For the “non-fake” side of this problem, I’d suggest using an instagram feed that is following thousands of people and one of the many scraping tools you can find on GitHub to collect images for training. In addition, I’d use a GoPro and go out and collect video of people just walking on city streets and the like.

        Using video like this is a powerful approach for generating huge quantities of training data. There isn’t really an upper limit to how much data you could collect this way.

  4. faraz March 11, 2019 at 11:07 am #

    Thank you very much Adrian for the tutorial. I was looking for it for some time now. My excitement level is that i haven’t gone through the tutorial yet but have came here straight to thank you in the comments section.

    • Adrian Rosebrock March 11, 2019 at 12:20 pm #

      Thanks Faraz!

  5. eviling March 11, 2019 at 11:17 am #

    Hi Adrian,

    Nice example 🙂
    I have a theoretical question: in the accuracy/loss plot, there is a gap between train/val loss and train/val accuracy? Does it mean the model has underfitting? because val_acc greater than train_acc.
    Also, if we have the same case but invert it, trai_acc greater than val_acc and with a gap?? Does it mean overfitting?

    How can be interpret this gap??

    • Adrian Rosebrock March 11, 2019 at 12:21 pm #

      Our dataset is very small so it’s harder to interpret the results. We have lower validation loss than training loss so the network is most certainly not overfitting. I provide my advice on how to best read plot histories and spot both overfitting and underfitting inside my book, Deep Learning for Computer Vision with Python. I would suggest starting there for more information.

  6. Alex March 11, 2019 at 11:22 am #

    I see you decided to be political in this post. Couldn’t help yourself? Not very business savvy. I hope this comment does not encourage you to ramp it up. I really like following what you do.

    • Adrian Rosebrock March 11, 2019 at 12:26 pm #

      Hm, I’m not sure I understand the point you’re trying to make. I’ve often used presidents of the USA in my tutorials and posts. For instance, if you take a look at the Practical Python and OpenCV page you’ll see I used a photo of former president Obama for an example of face detection. I don’t share my political views on this blog nor do I have any intentions to do so. Be kind on the PyImageSearch blog. Cheers friend.

    • Said March 14, 2019 at 6:59 am #

      Alex, Adrian uses pics of US Presidents in many articles.
      If you see word “Fake” and mr. Trumps pic - this doesnt mean, that the post is political.
      Maybe you read a lot of Fake news ? 🙂

  7. Dex March 11, 2019 at 11:42 am #

    Nice! It makes me wondering about something. Like image based lockers. But, btw. why is that working? Would it be correct (or correct me) to say that this works because its basically learning to distinguish “second recording/copy” from “first/original”? For example, if you would record not faces but just some environment, it would also work. Probably, just hypothesis, one can hack this by creating an inverse liveness network, trained to reconstruct “first/original” from “second recording”. What do you think?

    • Adrian Rosebrock March 11, 2019 at 12:26 pm #

      Yes, that is absolutely possible. The network is learning the difference in how images look in their “natural” captured format versus replayed.

  8. Ikbel Boulabiar March 11, 2019 at 12:30 pm #

    Thanks Adrian for this post.
    I’m very curious if more people can test this and build a bigger database of real/fake faces.

    All the available database used in research are not ethnically diverse:
    – NUAA : 100% Asian,
    – CASIA FASD : 100% Asian,
    – Idiap REPLAY-ATTACK : White 76%, Asian 22%, Black 2%
    – MSU MFSD: White 70%, Asian 28%, Black 2%
    In all of the above, Males are around 63% to 86%
    (from Face Spoof Detection with Image Distortion Analysis, Di Wen et al.)

    I have tested many algorithms based on texture analysis and similar but they all fail with dark skin people marking them as not real.

    I don’t know whether there some techniques more explainable than deep-learning that can be used and which they stay robust.

    • Adrian Rosebrock March 11, 2019 at 12:39 pm #

      Machine learning algorithms are only as good as the data they are given. And unfortunately, as you noted, many datasets are not ethnically diverse. The comment thread between myself and David Bonn discusses a method that could be used to build a larger, and ideally more diverse, dataset.

  9. Olexandr March 11, 2019 at 1:38 pm #

    Thank you!

    • Adrian Rosebrock March 11, 2019 at 1:54 pm #

      You are welcome, Olexandr! Enjoy it 🙂

  10. Taha Anwar March 11, 2019 at 3:48 pm #

    This liveness detection is an important in the face recog pipeline so great work in putting an article on it.

    However Adrian i seriously doubt this approach is gona work , i mean even for this example i dont think it can predict real faces of real people other than you with even mediocre accuracy.

    And also about adding more people to the dataset to make it more robust …i dont think that is gonna work either ….this conv net is too simple its gonna be freaking hard to converge…i mean think about it ure throwing 2d images of random people at one side and the 2d pics of those 2d images on other side and expecting conv nets to find a distinguishing pattern by only looking at the faces….i mean consider if you take proper image of person face and then take image of that pic with a high res camera and crop them to faces then even i cant tell the difference when shown 2 face images…so this approach is not gonna generalize……the best solution is to also utilize area outside of face or maybe process n frames and like you mentioned above use a number of techniques together.

    • Adrian Rosebrock March 11, 2019 at 5:42 pm #

      Hey Taha — I would suggest you try and run some experiments for yourself. Test your hypothesis. Code up the experiment. Run it. And iterate.

      Secondly, I did say that one of the limitations is that this specific model was trained on just my face. Yet the method worked well here. Would that single model, train just on my face, generalize to other people’s faces? Probably not — which is exactly why I suggested you gather more training data. The comment thread between myself and David should definitely be something to take a look at.

      Finally, there are differences in how ROIs captured between “normal” conditions and “playback” conditions. There will be different reflections, there could be artifacts in the playback, etc. A CNN, especially a sparse one, could be used to capture those. Don’t be fooled into think that a super deep network is required in all circumstances — that’s not always the case. Will the network depth likely increase with the more data you add? Yes, absolutely. Will it be ResNet50 depth? I highly doubt it. But I would suggest running those experiments to find out.

      It’s great that you are thinking critically about the problem but the next step here would be for you to start running your own experiments and noting the results.

      I’ll wrap up by saying that the best liveness detection systems will incorporate multiple methods, including classification, heuristics (eye movement, blinking, etc.), and depth. Those methods combined together will give you the best liveness results. Classification could either one the only piece of the puzzle or one of many pieces, depending on the complexity of what you’re developing.

  11. Taha Anwar March 11, 2019 at 7:36 pm #

    Hey Adrian..i will clarify one thing whrn i said the covnet is simple i meant the whole approach in general and im pretty sure depth is not the solition here….also the reflectace on playback is an environmental factor which can be controlled and made null

    And yes I absoloutely agree with you that at this point we can at best speculate but only experiments will make things clear

    Thanks for the informative answer

  12. John March 11, 2019 at 11:59 pm #

    Hi Adrian, I’ve noticed that you own what appears to be an IPhone X or XS. Knowing this, you must be familiar with how Apple’s FaceID works? They project thousands of Infrared dots onto a user’s face and then uses an Infrared camera to interpret the output. This way of facial authentication can mitigate the issue of people spoofing the camera with photos since the authentication is depth based.

    Onto the question I have. Are you planning to make a post on using computer vision techniques in tandem with depth sensors such as LIDAR? I would find such a post highly valuable. Cheers!

    • Adrian Rosebrock March 12, 2019 at 10:04 am #

      Yep, it’s an iPhone X. Depth-based liveness, in my opinion, is one of the best methods as it’s very hard to spoof. At the very least it will keep the “honest people honest” as the saying goes. I may do some depth-based tutorials here on PyImageSearch but I’m not sure if/when that may be. Thanks for the suggestion though!

  13. sharmila March 13, 2019 at 1:26 am #

    hi adrian

    Thank u so much for the post.i having small issue.How can i detect the faces form the long distance with my ip camera.can u please help me out.

    Thanks in advance

  14. Yaser Sakkaf March 14, 2019 at 1:46 am #

    Hi Adrian!

    I am new to Computer Vision. I am reading your blogs daily for the past two weeks and now I have some idea about Deep Learning and Computer Vision. I would like to be as good as you some day. Please guide me how to become like you !!!

    Thanks to you

  15. Antonio Álvarez March 17, 2019 at 5:12 am #

    Hi Adrian,

    Awesome work :-). We are working on another approach.

    Thermal cam ( AMG8833 ) image together normal cam
    using RPI3 … What do you think about this ?

    Did you tried ??

    Thanks in advance

    • Adrian Rosebrock March 19, 2019 at 10:10 am #

      A thermal camera would definitely be of help for liveness as well. I haven’t directly tried it but it would absolutely be beneficial.

  16. Maisa’a March 17, 2019 at 8:29 am #

    When I did the first command for training real video an error is shown …

    AttributeError: ‘module’ object has no attribute ‘ dnn’

    Line 30 in gather_examples.py

    Actually I don’t have an idea how to solve it 🙁

  17. Harvey Summers March 18, 2019 at 4:30 pm #

    I’m going to add this to my laptop so I can get alerts when I’m dead.

  18. lvlv19 March 20, 2019 at 3:55 am #

    Hi,Adrain.I am a newcomer in the field of computer vision.
    First,thank you for providing such a detailed demo for us to learn.
    I noticed that you introduced some methods for liveness detect and provided links to papers. As a rookie, I would like to ask, what are the main methods used in this demo? THQ

    • Adrian Rosebrock March 22, 2019 at 9:47 am #

      This method is using strictly classification.

  19. GepengJi March 21, 2019 at 8:59 pm #

    Hi! Adrian!
    How could i add depth info extracted from RGB image into the network?
    Could you give me some suggestions for it?
    (e.g., traditional methods? deep learning methods?)
    thanks!

    • Adrian Rosebrock March 22, 2019 at 8:29 am #

      If you want to add depth you would either:

      1. Need a camera capable of providing the depth
      2. Perform depth estimation

Leave a Reply

[email]
[email]