macOS for deep learning with Python, TensorFlow, and Keras

In today’s tutorial, I’ll demonstrate how you can configure your macOS system for deep learning using Python, TensorFlow, and Keras.

This tutorial is the final part of a series on configuring your development environment for deep learning. I created these tutorials to accompany my new book, Deep Learning for Computer Vision with Python; however, you can use these instructions to configure your system regardless if you bought my book or not.

In case you’re on the wrong page (or you don’t have macOS), take a look at the other deep learning development environment tutorials in this series:

To learn how to configure macOS for deep learning and computer vision with Python, just keep reading.

macOS for deep learning with Python, TensorFlow, and Keras

As you get acclimated in the deep learning domain, you’ll want to perform many experiments to hone your skills and even to solve real-world problems.

You’ll find that for experiments in the most chapters inside the Starter Bundle and half the chapters in the Practitioner Bundle can be executed on your CPU. Readers of the ImageNet Bundle will need a GPU machine in order to perform the more advanced experiments.

I definitely don’t recommend churning through large datasets and deep neural networks on your laptop, but like I said, for small experiments it is just fine.

Today, I’ll walk you through the steps to configure your Mac for deep learning.

First, we’ll install Xcode and Homebrew (a package manager). From there we will create a virtual environment called dl4cv  and install OpenCV, TensorFlow, and Keras into the environment.

Let’s get started.

Step #1: Install Xcode

For starters, you’ll need to get Xcode from the Apple App Store and install it. Don’t worry, it is 100% free.

Figure 1: Selecting Xcode from the Apple App Store.

From there, open a terminal and execute the following command to accept the developer license:

The next step is to install Apple command line tools:

Figure 2: Accepting the Xcode license.

Step #2: Install Homebrew

Homebrew (also known as Brew), is a package manager for macOS. You may already have it on your system, but if you don’t you will want to perform the actions in this section.

First we’ll install Homebrew by copying and pasting the entire command into your terminal:

Next we’ll update our package definitions:

Followed by updating your ~/.bash_profile  using the  nano  terminal editor (any other editor should do the trick as well):

Add the following lines to the file:

Figure 3: Changing ~/.bash_profile to accommodate Homebrew.

Next, simply reload your ~/.bash_profile  (this happens automatically when a new terminal is opened):

Now that Brew is ready to go, let’s get Python 3 installed.

Step #3: Install Homebrew Python 3 for macOS

This step is actually very easy, but I want to clear up some possible confusion first.

macOS comes with Python installed; however we will be installing a non-system Python using Brew. While you could use your system Python, it is actually strongly discouraged. Therefore, don’t skip this step — it is very important to your successful install.

To install Python 3 with Homebrew, simply execute this command:

Before continuing you’ll want to verify that your Python 3 installation is Homebrew’s rather than the macOS system’s:

Ensure that you see “ local ” in each path. If you don’t see this output, then you aren’t using Homebrew’s install of Python 3.

Figure 4: Executing which python3 and which pip3 to ensure that you are using the Homebrew version of each rather than the system version.

Assuming your Python 3 install worked, let’s continue on to Step #4.

Step #4: Create your Python virtual environment

As I’ve stated in other install guides on this site, virtual environments are definitely the way to go when working with Python, enabling you to accommodate different versions in sandboxed environments.

In other words, there is less of a chance that you’ll do something that is a pain in the ass to fix. If you mess up an environment, you can simply delete the environment and rebuild it.

Let’s install virtualenv and virtualenvwrapper via pip :

From there, we’ll update our ~/.bash_profile  again:

Where we’ll add the following lines to the file:

Figure 5: The bottom of our ~/.bash_profile should contain these lines for virtualenv and virtualenvwrapper.

Followed by reloading the file:

Creating the ‘dl4cv’ environment

The dl4cv  environment will house all of our software for performing experiments associated with my book. You can easily name the environment whatever you want, but from here on we’ll be referring to it as dl4cv .

To create the dl4cv environment with Python 3 simply enter the following command:

After Python 3 and supporting scripts are installed into the new environment, you should actually be inside the environment.  This is denoted by a ‘ (dl4cv) ‘ at the beginning of your bash prompt as shown in the figure below:

Figure 6: The (dl4cv) in the bash prompt signifies that we are working inside the dl4cv virtual environment. If you don’t see this, then execute workon dl4cv to activate the environment.

If you do not see the modified bash prompt then you can enter the following command at any time to enter the environment at any time:

The only Python dependency required by OpenCV is NumPy, which we can install below:

That’s it as far as creating a virtual environment and installing NumPy. Let’s continue to Step #5.

Step #5: Install OpenCV prerequisites using Homebrew

The following tools need to be installed for compilation, image I/O, and optimization:

After those packages are installed we’re ready to install OpenCV.

Step #6: Compile and Install OpenCV

First, let’s download the source code:

Then unpack the archives:

Followed by configuring the build with CMake (it is very important that you copy the CMake command exactly as it appears here, taking care to copy and past the entire command; I would suggest clicking the “<=>” button in the toolbar below to expand the entire command):

Note: For the above CMake command, I spent considerable time creating, testing, and refactoring it. I’m confident that it will save you time and frustration if you use it exactly as it appears. Make sure you click the “<=>” button in the toolbar of the code block above to expand the code block. This will enable you to copy and paste the entire command.

Your output should be similar to the screenshot below which ensures that the correct Python 3 binary/library and NumPy version are utilized:

Figure 7: The OpenCV CMake output that ensures the correct Python 3 and NumPy will be used for compiling.

Then we’re ready to perform the compilation compile OpenCV:

Note: The number ‘4’ above specifies that we have 4 cores/processors for compiling. If you have a different number of processors you can update the -j  switch. For only one core/processor simply just use the make  command (from the build directory enter make clean  prior to retrying if your build failed or got stuck).

From there you can install OpenCV:

After installing it is necessary to sym-link the cv2.so  file into the dl4cv  virtual environment:

Finally, we can test out the install:

Figure 8: OpenCV 3.3 has been installed and linked as is shown by the successful import and display of the version.

If your output properly shows the version of OpenCV that you installed, then you’re ready to go on to Step #7 where we will install the Keras deep learning library.

Step #7: Install Keras

Before beginning this step, ensure you have activated the dl4cv  virtualenv. If you aren’t in the environment, simply execute:

Then, using pip , install the required Python computer vision, image processing, and machine learning libraries:

Next install matplotlib and update the rendering backend:

Then, install TensorFlow:

Followed by keras:

To verify that Keras is installed properly we can import it and check for errors:

Keras should be imported with no errors, while stating that TensorFlow is being utilized as the backend.

Figure 9: Keras is successfully installed into the dl4cv virtual environment.

At this point, you can familiarize yourself with the ~/.keras/keras.json  file:

Ensure that the image_data_format  is set to channels_last  and that the backend  is set to tensorflow .

Congratulations! You’re now ready to go. If you didn’t open up a beer or coffee during the installation process, now is the time. It’s also the time to find a comfortable spot to read Deep Learning for Computer Vision with Python.

Summary

In today’s post, we configured our macOS box for computer vision and deep learning. The main pieces of software included Python 3, OpenCV, TensorFlow, and Keras accompanied by dependencies and installation/compilation tools.

As you can see, utilizing Homebrew, pip, and virtualenv + virtualenvwrapper made this install rather easy. I spent quite a bit of time creating and testing  the CMake command which should work easily on your computer. Be sure to give it a try.

If you encountered any problems along the way, leave a comment in the form below.

If you would like to put your newly configured macOS deep learning environment to good use, I would highly suggest you take a look at my new book, Deep Learning for Computer Vision with Python.

Regardless if you’re new to deep learning or already a seasoned practitioner, the book has content to help you reach deep learning mastery — take a look here.

, , , , , , ,

37 Responses to macOS for deep learning with Python, TensorFlow, and Keras

  1. Shannon September 29, 2017 at 11:07 am #

    Thanks Adrian, these tutorials are great. Really looking forward to the rest of the Kickstarter deliverables coming out over the next few months.

    Question regarding setup: you used brew + virtualenv to set up and configure the dl4cv environment. Would it be possible to instead use conda environments? I suppose the one potential hiccup that comes to mind would be linking the OpenCV 3.3.0 build with the environment.

    • Adrian Rosebrock October 2, 2017 at 10:14 am #

      You can use conda environments if you wish, but please note that I cannot provide any suggestions/support if you deviate from the instructions here.

    • Stephen October 4, 2017 at 7:00 am #

      Probably would need to replace -D PYTHON3_EXECUTABLE=$VIRTUAL_ENV/bin/python \ with -D PYTHON3_EXECUTABLE=$CONDA_PREFIX/bin/python \

      I’m trying it now to see if it works – otherwise Ubuntu VM.

      • Stephen October 4, 2017 at 7:16 am #

        Nah, there is obviously more to just that for the makefile using conda. Libraries and possibly packages path (PYTHON3_LIBRARY and PYTHON3_INCLUDE_DIR) need repointing.

        — Python 3:
        — Interpreter: /Users/stgreszc/anaconda/envs/python3/bin/python (ver 3.6.2)
        — Libraries: /usr/lib/libpython2.7.dylib (ver 3.6.2)
        — numpy: /Users/stgreszc/anaconda/envs/python3/lib/python3.6/site-packages/numpy/core/include (ver 1.12.1)
        — packages path: lib/python3.6/site-packages

  2. dear bar September 29, 2017 at 9:56 pm #

    Very timely tutorial, very grateful.

  3. memeka September 29, 2017 at 10:55 pm #

    Any plans to try CoreML?

    • Adrian Rosebrock September 30, 2017 at 9:38 am #

      No plans at the moment, but I will consider it for the future.

  4. Jeff September 30, 2017 at 4:18 pm #

    Your install guide worked perfectly on my Macbook Pro (10.12.6). Thank you!

    • Adrian Rosebrock October 2, 2017 at 9:48 am #

      Awesome, I’m glad to hear it Jeff! Congrats on getting your macOS machine configured for deep learning!

  5. hendrick October 1, 2017 at 5:48 am #

    I cannot run make -j4, the error said make:*** No targets specified and no makefile found. someone can help me?

    • Adrian Rosebrock October 1, 2017 at 8:47 am #

      Double-check your output from CMake. It is very likely that CMake exited with an error, hence there it did not generate a Makefile. Secondly, ensure you are in the “build” directory before executing “make”.

  6. Andreas Gounaris October 2, 2017 at 5:01 am #

    Hi Adrian, I have 3 virtual envs, with opencv 3.0 and I want to upgrade all of them to ver 3.3. If I download and compile opencv 3.3 outside of a virtual env, should my envs be informed about the upgrade?

    If not, how can I do that?

    • Adrian Rosebrock October 2, 2017 at 9:24 am #

      Are all three of your virtual environments the same Python version? Or different?

      If they are the same you can simply compile OpenCV 3.3 once, install it, and ensure the cv2.so sym-links are correct for each Python virtual environment.

  7. Andrew October 2, 2017 at 4:33 pm #

    Does not work with the latest XCode 9.0.

    nvcc warning : The ‘compute_20’, ‘sm_20’, and ‘sm_21’ architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
    nvcc fatal : The version (‘90000’) of the host compiler (‘Apple clang’) is not supported
    CMake Error at cuda_compile_generated_gpu_mat.cu.o.cmake:208 (message):

    • Andrew October 3, 2017 at 1:44 am #

      I went back to 8.1. It worked.
      Apple LLVM version 8.0.0 (clang-800.0.42.1)
      Target: x86_64-apple-darwin16.7.0
      Thread model: posix
      InstalledDir: /Library/Developer/CommandLineTools/usr/bin

  8. Joachim October 5, 2017 at 3:24 am #

    Thanks, Adrian, really great tutorial !

    I made the installation on an iMac (Retina 5K, 27-inch, late 2014) with macOS High Sierra and Xcode 9.0. I got a problem with make -j4: “fatal error tesseract/baseapi.h file not found.
    After adding “-D BUILD_opencv_text = off” it worked.

    • Adrian Rosebrock October 6, 2017 at 5:07 pm #

      Awesome, thank you for sharing Joachim!

  9. Chris October 8, 2017 at 9:14 pm #

    Thanks, Adrian, for yet another, most excellent tutorial!

    I was able to successfully install on High Sierra(10.13) using a late 2013 Mac Pro. The only hitch I had was a conflict when running one of your dl4cv scripts that imports ‘matplotlib.pyplot.’ Seems there was some sore of OSX back-end-renderer conflict. I found a fix here:

    https://stackoverflow.com/questions/21784641/installation-issue-with-matplotlib-python

    Do you have any suggestions for editing the Cmake so that tensor flow can be compiled to use SSE4.2?

    • Adrian Rosebrock October 9, 2017 at 12:18 pm #

      Thanks for sharing the matplotlib solution, Chris!

      I have not compiled TensorFlow by hand (I’ve only used the pip install) but I’ll consider looking into this and doing a future blog post on it.

  10. Gavin October 16, 2017 at 12:30 pm #

    Hi Adrian, thanks for the great tutorial.

    I am trying to install on a 2017 imac with high sierra. I have managed to get as far as the cMake command for opencv. When I check the build both numpy and libraries say NO next to them. Do I need to change something to make cmake find them?

    Many thanks

    • Adrian Rosebrock October 16, 2017 at 12:35 pm #

      Hi Gavin — can you please confirm whether or not you are in the “dl4cv” Python virtual environment when executing CMake? Also, can you confirm the full Python version?

  11. Gavin Armstrong October 16, 2017 at 3:51 pm #

    Thanks Adrian,

    I can confirm I am in the “dl4cv” and the python version is 3.6.3

    I have made a video of the install skip to the last 30 secs or so to see the cmake script running and the results.

    https://youtu.be/6Tqq_fbmzxc

    Thanks again
    Gavin

    • Gavin Armstrong October 16, 2017 at 4:26 pm #

      Its working, just installed all osx updates and deleted the build dir and tried again. Build success.

      Thanks again
      Gavin

      • Adrian Rosebrock October 17, 2017 at 9:34 am #

        Congrats on getting your deep learning environment configured, Gavin! Nice job!

  12. Andrew Baker October 26, 2017 at 12:26 pm #

    I couldn’t get the cmake file to generate properly. At the end I received the following CMake Error:
    The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files:
    CUDA_nppi_LIBRARY (ADVANCED)
    linked by target “opencv_cudev” in directory /Users/andrewbaker/opencv-3.3.0/modules/cudev …… there are many more linked by target lines which follow.

    My configuration:
    macOS Version 10.12.6
    CUDA 9.0
    XCode 9.0
    clang-900.0.38

    I tried going back to Xcode 8.1. That produced the same result. Maybe I should try going back to CUDA 8.0?

    • Adrian Rosebrock October 27, 2017 at 10:54 am #

      Hi Andrew — CUDA likely won’t work on your Mac unless you indeed have a capable NVIDIA GPU. Do you have an NVIDIA GPU?

      • Andrew Baker October 31, 2017 at 1:35 am #

        Yes I do. I went back to XCode 8.1, clang-800.0.42.1 and CUDA 8.0.61. All complied fine and installed. However now when I try to do the simlink I am getting cv2.so: No such file or directory.

        • Andrew Baker October 31, 2017 at 2:20 am #

          Ok I figured it out. No issues all is well.

          • Adrian Rosebrock October 31, 2017 at 7:40 am #

            Congrats on figuring out the issue, Andrew!

  13. Murthy October 31, 2017 at 12:10 am #

    Hi Adrian,
    Thanks for the tutorial.
    I had a successful installation.

    When I pip installed matplotlib – it mentioned it is already installed but the next command “touch” failed as there was no .matplotlib . I proceeded by created one. Hope this is not a big issue when it comes to rendering.

    • Adrian Rosebrock October 31, 2017 at 7:42 am #

      Hi Murthy — this shouldn’t be an issue.

  14. CrashDummy November 30, 2017 at 9:25 pm #

    Found that when installing and compiling OpenCV (Step #6), numpy wasn’t utilised on the build list for some reason.

    I need to manually add this line to the CMake list for it to include numpy’s directory:

    -D PYTHON_NUMPY_INCLUDE_DIR=python -c ‘import numpy as np; print(np.__path__)’ \

    Hope this helps anyone out there with the same issue 🙂

    I’m using:
    macOS 10.13.1 on MacBook Pro 13 Mid 2014
    OpenCV 3.3.0
    Numpy 1.3.3

    • Adrian Rosebrock December 2, 2017 at 7:31 am #

      Thank you for sharing! I’ll make note of this as well 🙂

  15. Michael December 12, 2017 at 5:31 pm #

    Adrian,

    Possibly a stupid question, but I previously installed opencv using one of your tutorials. Do I need to set it up again in this new environment, or can I use the one I previously installed?

    Thanks

    • Adrian Rosebrock December 15, 2017 at 8:39 am #

      You can use your previous install of OpenCV. I would recommend creating a new Python virtual environment and then sym-linking in your OpenCV bindings, but that’s entirely up to you.

  16. Andrew Baker December 14, 2017 at 12:48 pm #

    Today the new iMac Pro was introduced by Apple. The most expensive configuration came in at $13,348.00. That was with a 2.3GHz 18 core Intel Xeon W processor, 128GB RAM, 4 TB SSD and Radeon Pro Vega 64 16 GB video card. Being that most of the Deep Learning frameworks are dependent on CUDA and the NVIDIA architecture, I wonder what Apple’s thinking was for using this machine for Deep Learning? For $13k one could build a very nice multiGPU capable Ubuntu machine.

    • Adrian Rosebrock December 14, 2017 at 1:41 pm #

      I haven’t watched the announcement video or looked at the specs yet (I certainly will though) — but were they mentioning in the announcement that they expected the new iMac to be used for deep learning?

Leave a Reply