Where did SIFT and SURF go in OpenCV 3?

sift_and_surf_side_by_side

If you’ve had a chance to play around with OpenCV 3 (and do a lot of work with keypoint detectors and feature descriptors) you may have noticed that the SIFT and SURF implementations are no longer included in the OpenCV 3 library by default.

Unfortunately, you probably learned this lesson the hard way by opening up a terminal, importing OpenCV, and then trying to instantiate your favorite keypoint detector, perhaps using code like the following:

Oh no! There is no longer a cv2.FeatureDetector_create  method!

The same is true for our cv2.DescriptorExtractor_create  function as well:

Furthermore, cv2.SIFT_create  and cv2.SURF_create  will fail as well:

I’ll be honest — this had me scratching my head at first. How am I supposed to access SIFT, SURF, and my other favorite keypoint detectors and local invariant descriptors if cv2.FeatureDetector_create  and cv2.DescriptorExtractor_create  have been removed?

The cv2.FeatureDetector_create  and cv2.DescriptorExtractor_create  were (and still are) methods I used all the time. And personally, I really liked the OpenCV 2.4.X implementation. All you needed to do was pass in a string and the factory method would build the instantiation for you. You could then tune the parameters using the getter and setter methods of the keypoint detector or feature descriptor.

Furthermore, these methods have been part of OpenCV 2.4.X for many years. Why in the world were they removed from the default install? And where were they moved to?

In the remainder of this blog post, I’ll detail why certain keypoint detectors and local invariant descriptors were removed from OpenCV 3.0 by default. And I’ll also show you where you can find SIFT, SURF, and other detectors and descriptors in the new version of OpenCV.

Why were SIFT and SURF removed from the default install of OpenCV 3.0?

SIFT and SURF are examples of algorithms that OpenCV calls “non-free” modules. These algorithms are patented by their respective creators, and while they are free to use in academic and research settings, you should technically be obtaining a license/permission from the creators if you are using them in a commercial (i.e. for-profit) application.

With OpenCV 3 came a big push to move many of these “non-free” modules out of the default OpenCV install and into the opencv_contrib package. The opencv_contrib  packages contains implementations of algorithms that are either patented or in experimental development.

The algorithms and associated implementations in  opencv_contrib  are not installed by default and you need to explicitly enable them when compiling and installing OpenCV to obtain access to them.

Personally, I’m not too crazy about this move.

Yes, I understand including patented algorithms inside an open source library may raise a few eyebrows. But algorithms such as SIFT and SURF are pervasive across much of computer vision. And more importantly, the OpenCV implementations of SIFT and SURF are used by academics and researchers daily to evaluate new image classification, Content-Based Image Retrieval, etc. algorithms. By not including these algorithms by default, more harm than good is done (at least in my opinion).

How do I get access to SIFT and SURF in OpenCV 3?

To get access to the original SIFT and SURF implementations found in OpenCV 2.4.X, you’ll need to pull down both the opencv and opencv_contrib repositories from GitHub and then compile and install OpenCV 3 from source.

Luckily, compiling OpenCV from source is easier than it used to be. I have gathered install instructions for Python and OpenCV for many popular operating systems over on the OpenCV 3 Tutorials, Resources, and Guides page — just scroll down the Install OpenCV 3 and Python section and find the appropriate Python version (either Python 2.7+ or Python 3+) for your operating system.

How do I use SIFT and SURF with OpenCV 3?

So now that you have installed OpenCV 3 with the opencv_contrib  package, you should have access to the original SIFT and SURF implementations from OpenCV 2.4.X, only this time they’ll be in the xfeatures2d  sub-module through the cv2.SIFT_create  and cv2.SURF_create  functions.

To confirm this, open up a shell, import OpenCV, and execute the following commands (assuming you have an image named test_image.jpg  in your current directory, of course):

If all goes well, you should be able to instantiate the SIFT and SURF keypoint detectors and local invariant descriptors without error.

It’s also important to note that by using opencv_contrib  you will not be interfering with any of the other keypoint detectors and local invariant descriptors included in OpenCV 3. You’ll still be able to access KAZE, AKAZE, BRISK, etc. without an issue:

Summary

In this blog post we learned that OpenCV has removed the cv2.FeatureDetector_create  and cv2.DescriptorExtractor_create  functions from the library. Furthermore, the SIFT and SURF implementations have also been removed from the default OpenCV 3 install.

The reason for SIFT and SURF removal is due to what OpenCV calls “non-free” algorithms. Both SIFT and SURF are patented algorithms, meaning that you should technically be getting permission to use them in commercial algorithms (they are free to use for academic and research purposes though).

Because of this, OpenCV has made the decision to move patented algorithms (along with experimental implementations) to the opencv_contrib package. This means that to obtain access to SIFT and SURF, you’ll need to compile and install OpenCV 3 from source with opencv_contrib  support enabled. Luckily, this isn’t too challenging with the help of my OpenCV 3 install guides.

Once you have installed OpenCV 3 with opencv_contrib  support you’ll be able to find your favorite SIFT and SURF implementations in the xfeatures2d  package through the cv2.xfeatures2d.SIFT_create()  and cv2.xfeatures2d.SURF_create()  functions.

, , , , , , ,

36 Responses to Where did SIFT and SURF go in OpenCV 3?

  1. Max July 16, 2015 at 4:29 pm #

    Hi Adrian,

    First of all: Thank you very much for your effort!:) you said that you will cover the building process of opencv3 using the raspberry pi. Could you please consider including tbb when compiling opencv? Without tbb it’s not possible to leverage from all four cores. Unfortunately I was not able to compile it with tbb support, this seems to be due to a problem with raspbian wheezy.
    Thank you very much in advance:)

    • Adrian Rosebrock July 17, 2015 at 6:33 am #

      Hey Max, thanks for the comment. I already have the tutorial drafted, but I’ll go back and see if I can include tbb — at the very least it could make for a good followup post.

  2. Marcel July 20, 2015 at 5:21 pm #

    Hey Adrian,

    is using ORB, which isn’t patented and is written by the OpenCV guys, instead of SIFT or SURF no reasonable alternative? I have only a bit experience with it but so far it had given good results. I’d still have to compare them directly though to be able to tell if there’s a difference.

    • Adrian Rosebrock July 20, 2015 at 7:31 pm #

      There are many, many alternatives to SIFT and SURF. ORB is one. BRIEF, BRISK, FREAK, KAZE, and AKAZE are others. Choosing between them is very much application specific and I recommend trying the options, examining the accuracy, and then making the educated choice from there. I hope that helps!

  3. ria September 26, 2015 at 11:31 am #

    Will this method work windows too? and if firstly unzip open cv from source directly and then perform this tutorial will this work. can u provide basics steps for installing it on windows too.

    • Adrian Rosebrock September 26, 2015 at 11:52 am #

      I don’t own a Windows system and haven’t used the OS in 8+ years at this point. The tutorials covered in this blog mainly focus on OSX, Ubuntu, and Raspbian (for the Raspberry Pi). For help with Windows and OpenCV, I would consult the official OpenCV documentation.

  4. Brian November 4, 2015 at 7:49 pm #

    I just installed a fresh version of Jessie on my Raspberry Pi 2.
    Apparently OpenCV 2.4.9.1 is included in that build.

    However, like OpenCV 3.0, it appears that xfeatures2d is not there either.
    I get the following error:
    AttributeError: ‘module’ object has not attribute ‘xfeatures2d’

    I’ll try to do the install as described above to see if it resolves the problem.

    • Adrian Rosebrock November 5, 2015 at 6:41 am #

      OpenCV 2.4.9 is included in Jessie? Interesting. I did not know that.

      However, if you’re looking to use SIFT, SURF, etc. in OpenCV 2.4.X, you’ll need to use the cv2.FeatureDetector_create and cv2.DescriptorExtractor_create methods.

  5. Agnes November 24, 2015 at 3:21 am #

    Again I got this error. Sir, can you please provide me any solution.?

    • Adrian Rosebrock November 24, 2015 at 6:32 am #

      Please follow my OpenCV 3 install instructions for your particular OS and Python version here. Be sure to pay attention to the section on installing the opencv_contrib module — this is where optional modules such as xfeatures2d will be compiled and installed.

  6. Gerard January 24, 2016 at 9:09 pm #

    This is a life saver! Thanks for this post!!!!

    • Adrian Rosebrock January 25, 2016 at 4:07 pm #

      No problem Gerard, I’m happy it helped!

  7. Santiago March 28, 2016 at 1:30 pm #

    Hi, I’m really new in OpenCV and I need to use Sift and Surf in Matlab. I did every thing you say here and now I have the .cpp functions but I don’t know how to import them to Matlab. Do you know how to? I will appreciate for this.

    • Adrian Rosebrock March 28, 2016 at 1:36 pm #

      Sorry, I only use OpenCV + Python this blog. It’s been a good many years since I’ve had to use MATLAB. I would suggest using the Image Processing Toolkit for MATLAB or better yet, using David Lowe’s implementation of SIFT in MATLAB.

  8. lina May 3, 2016 at 4:42 pm #

    thanx very helpful..plz im clustering images.. how can i pass those features as vector to kmeans

    • Adrian Rosebrock May 3, 2016 at 5:40 pm #

      You would first need to construct a bag of visual words model, which involves:

      1. Extracting feature vectors from all your images.
      2. Clustering these features to build a codebook.
      3. Quantizing the features associated with each image into the codebook to form your histogram of visual words.
      4. Finally, you can take this resulting histogram and pass it on to k-means.

      I detail how to build the bag of visual words model from the ground up inside PyImageSearch Gurus. Be sure to take a look!

      • lina May 4, 2016 at 8:06 pm #

        thnxxxxxx..plz, before extracting features, image doesnt need any pre processing??

        • Adrian Rosebrock May 5, 2016 at 6:45 am #

          That depends entirely on your dataset. Normally conversion to grayscale, resizing, and perhaps a bit of Gaussian smoothing. But that is again entirely dependent on your dataset.

  9. Mayank June 14, 2016 at 6:08 pm #

    Hey adrian,

    SIFT in opencv3 is up and running smooth, thanks for the tutorial. But there are other issues too in opencv3(like cv2.cv.Scalar(10,20,0)) this is not present in new version. Is there any way i can set opencv 2.4 for my old projects in pycharm and opencv3 for new projects. thanks!!

    • Adrian Rosebrock June 15, 2016 at 12:32 pm #

      Correct, anytime you see cv2.cv this should be a red flag that the code will not run in OpenCV 3 — this is because the old cv2.cv sub-module is not included. However, a Scalar in OpenCV 3 (and OpenCV 2.4) should just be represented as a tuple. I would suggest creating a separate virtual environment for your OpenCV 3 install, and then configure it with PyCharm.

  10. Sushma September 4, 2016 at 12:12 am #

    You are rockstar….. thanks for this post.

  11. Simran Gulati September 16, 2016 at 5:48 am #

    Hi Adrian. Actually i was stuck in how to compile opencv again with opencv contrib . I am using PyCharm for building programs and my OS is Windows.What to do to compile the file opencv3 with opencv-contrib??

    • Adrian Rosebrock September 16, 2016 at 8:18 am #

      Hey Simran — I don’t support Windows on the PyImageSearch blog. I do provide support for Ubuntu, OSX, and Raspbian, and in all honesty, if you want to learn computer vision you should be using a Unix-based OS (unless there is some very specific Microsoft functionality that you need). I provide install instructions on this page. And I also offer a pre-configured Ubuntu virtual machine as part of Practical Python and OpenCV.

  12. jayant December 19, 2016 at 12:24 am #

    Hey! Thanks man! this post was really helpful. one more thing… what will be the substitute for cv2.DescriptorExtractor_create(“SIFT”) ?

    • Adrian Rosebrock December 21, 2016 at 10:41 am #

      As detailed in the blog post, the substitute is:

      sift = cv2.xfeatures2d.SIFT_create()

      You can detect keypoints via:

      kps = sift.detect(image)

      And then extract features via:

      (kps, descs) = sift.compute(image, kps)

  13. Linus Vivaldi April 2, 2017 at 1:11 pm #

    I would like to access attributes like sift.upright as well. How can I do that?

    • Adrian Rosebrock April 3, 2017 at 1:57 pm #

      I wasn’t aware sift.upright was an attribute. Can you link to the documentation you are seeing for sift.upright?

  14. Mojtaba April 21, 2017 at 8:43 am #

    Hi Adrian, I also tried to work with SIFT but gave me and error in the VirtualMachine, I wanted to ask if opencv_contrib is also installed in VirtualMachine? and if not is there any clean way to add it to the existing VirtualMachine without starting to install OpenCV from scratch?

    • Adrian Rosebrock April 21, 2017 at 10:41 am #

      Hi Mojtaba — are you referring to the VM included in Practical Python and OpenCV? If so, yes, the opencv_contrib packages are included in the OpenCV installation. If you are getting an error message, please share it so we can resolve the issue.

      • Mojtaba April 22, 2017 at 11:27 am #

        Thanks Adrian, Yes I mean the VirtualMachine in Practical Python and OpenCV course. I get this error for this line of code :
        my code : sift = cv2.SIFT()
        module object has no attribute ‘SIFT’

        I’ll appreciate if you can help me with this.
        Thanks again very much for your help

        • Adrian Rosebrock April 24, 2017 at 9:41 am #

          Hi Mojtaba — please read the contents of this blog post. In OpenCV 3, you can access SIFT via:

          sift = cv2.xfeatures2d.SIFT_create()

  15. Vitalis Asiku April 24, 2017 at 5:57 pm #

    I really appreciate the effort you have put into these tutorials. Thank you so much.

    • Adrian Rosebrock April 25, 2017 at 11:50 am #

      Thanks Vitalis!

  16. Polly Lina April 25, 2017 at 11:37 pm #

    Is there anyone could suggest how or which part to change in the SIFT Opencv to read 3D data instead of 2D images?

Trackbacks/Pingbacks

  1. How to install OpenCV 3 on Raspbian Jessie - PyImageSearch - January 3, 2016

    […] full install of OpenCV 3 (which includes features such as SIFT and SURF), be sure to grab the opencv_contrib repo as well. (Note: Make sure your opencv  and […]

  2. OpenCV panorama stitching - PyImageSearch - January 11, 2016

    […] there are major differences in how OpenCV 2.4.X and OpenCV 3.X handle keypoint detection and local invariant d… (such as SIFT and SURF), I’ve taken special care to provide code that is compatible […]

Leave a Reply