Pre-configured Amazon AWS deep learning AMI with Python

The Ubuntu VirtualBox virtual machine that comes with my book, Deep Learning for Computer Vision with Python, includes all the necessary deep learning and computer vision libraries you need (such as Keras, TensorFlow, scikit-learn, scikit-image, OpenCV, etc.) pre-installed.

However, while the deep learning virtual machine is easy to use, it also has a number of drawbacks, including:

  • Being significantly slower than executing instructions on your native machine.
  • Unable to access your GPU (and other peripherals attached to your host).

What the virtual machine has in convenience you end up paying for in performance — this makes it a great for readers who are getting their feet wet, but if you want to be able to dramatically boost speed while still maintaining the pre-configured environment, you should consider using Amazon Web Services (AWS) and my pre-built deep learning Amazon Machine Image (AMI).

Using the steps outlined in this tutorial you’ll learn how to login (or create) your AWS account, spin up a new instance (with or without a GPU), and install my pre-configured deep learning image. This will enable you to enjoy the pre-built deep learning environment without sacrificing speed.

To learn how to use my deep learning AMI, just keep reading.

Pre-configured Amazon AWS deep learning AMI with Python

In this tutorial I will show you how to:

  1. Login/create your AWS account.
  2. Launch my pre-configured deep learning AMI.
  3. Login to the server and execute your code.
  4. Stop the machine when you are done.

However, before we get too far I want to mention that:

  • The deep learning AMI is Linux-based so I would recommend having some basic knowledge of Unix environments, especially the command line.
  • AWS is not free and costs an hourly rate. Exactly how much the hourly rate depends is on which machine you choose to spin up (no GPU, one GPU, eight GPUs, etc.). For less than $1/hour you can use a machine with a GPU which will dramatically speedup the training of deep neural networks. You pay for only the time the machine is running. You can then shut down your machine when you are done.

Step #1: Setup Amazon Web Services (AWS) account

In order to launch my pre-configured deep learning you first need an Amazon Web Services account.

To start, head to the Amazon Web Services homepage and click the “Sign In to the Console” link:

Figure 1: The Amazon Web Services homepage.

If you already have an account you can login using your email address and password. Otherwise you will need to click the “Create a new AWS account” button and create your account:

Figure 2: Logging in to your Amazon Web services account.

I would encourage you to use an existing Amazon.com login as this will expedite the process.

Step #2: Select and launch your deep learning AWS instance

You are now ready to launch your pre-configured deep learning AWS instance.

First, you should set your region/zone to “US West (Oregon)”. I created the deep learning AMI in the Oregon region so you’ll need to be in this region to find it, launch it, and access it:

Figure 3: Setting your AWS region to “US West (Oregon)”.

After you have set your region to Oregon, click the “Services” tab and then select “EC2” (Elastic Cloud Compute):

Figure 4: Accessing the Amazon EC2 dashboard.

From there you should click the “Launch Instance” button:

Figure 5: Launching an Amazon AWS instance for deep learning.

Then select the “Community AMIs” and search for either “deep-learning-for-computer-vision-with-python” or “ami-ccba4ab4”:

Figure 6: Searching for the Deep Learning for Computer Vision with Python AMI.

Click “Select” next to the AMI.

You are now ready to select your instance type. Amazon provides a huge number of virtual servers that are designed to run a wide array of applications. These instances have varying amount of CPU power, storage, network capacity, or GPUs, so you should consider:

  1. What type of machine you would like to launch.
  2. Your particular budget.

GPU instances tend to cost much more than standard CPU instances. However, they can train deep neural networks in a fraction of the time. When you average out the amount of time it takes to train a network on a CPU versus on a GPU you may realize that using the GPU instance will save you money.

For CPU instances I recommend you use the “Compute optimized” c4.* instances. In particular, the c4.xlarge instance is a good option to get your feet wet.

If you would like to use a GPU, I would highly recommend the “GPU compute” instances. The p2.xlarge instance has a single NVIDIA K80 (12GB of memory).

The p2.8xlarge sports 8 GPUs. While the p2.16xlarge has 16 GPUs.

I have included the pricing (at the time of this writing) for each of the instances below:

  • c4.xlarge: $0.199/hour
  • p2.xlarge: $0.90/hour
  • p2.8xlarge: $7.20/hour
  • p2.16xlarge: $14.40/hour

As you can see, the GPU instances are much more expensive; however, you are able to train networks in a fraction of the cost, making them a more economically viable option. Because of this I recommend using the p2.xlarge instance if this is your first time using a GPU for deep learning.

In the example screenshot below you can see that I have chosen the p2.xlarge instance:

Figure 7: Selecting the p2.xlarge instance for deep learning using the GPU.

Next, I can click “Review and Launch” followed by “Launch” to boot my instance.

After clicking “Launch” you’ll be prompted to select your key pair or create a new key pair:

Figure 8: Selecting a key pair for our Elastic Cloud Compute instance.

If you have an existing key pair you can select “Choose an existing key pair” from the drop down. Otherwise you’ll need to select the “Create a new key pair” and then download the pair. The key pair is used to login to your AWS instance.

After acknowledging and accepting login note from Amazon your instance will start to boot. Scroll down to the bottom of the page and click “View Instances”. It will take a minute or so for your instance to boot.

Once the instance is online you’ll see the “Instance State” column be changed to “running” for the instance.

Select it and you’ll be able to view information on the instance, including the IP address:

Figure 9: Examining the IP address of my deep learning AWS instance.

Here you can see that my IP address is 52.88.146.157 . Your IP address will be different.

Fire up a terminal and you can SSH into your AWS instance:

You’ll want to update the command above to:

  1. Use the filename you created for the key pair.
  2. Use the IP address of your instance.

Step #3: (GPU only) Re-install NVIDIA deep learning driver

If you selected a GPU instance you will need to:

  1. Reboot your AMI via the command line
  2. Reinstall the NVIDIA driver

The reason for these two steps is because instances launched from a pre-configured AMI can potentially restart with a slightly different kernel, therefore causing the Nouveau (default) driver to be loaded instead of the NVIDIA driver.

To avoid this situation you can either:

  1. Reboot your system now, essentially “locking in” the current kernel and then reinstalling the NVIDA driver once.
  2. Reinstall the NVIDIA driver each time you launch/reboot your instance from the AWS admin.

Both methods have their pros and cons, but I would recommend the first one.

To start, reboot your instance via the command line:

Your SSH connection will terminate during the reboot process.

Once the instance has rebooted, re-SSH into the instance, and reinstall the NVIDIA kernel drivers. Luckily this is easy as I have included the driver file in the home directory of the instance.

If you list the contents of the installers  directory you’ll see three files:

Change directory into installers  and then execute the following command:

Follow the prompts on screen (including overwriting any existing NVIDIA driver files) and your NVIDIA deep learning driver will be installed.

You can validate the NVIDIA driver installed successfully by running the nvidia-smi  command:

Step #4: Access deep learning Python virtual environment on AWS

You can access our deep learning and computer vision libraries by using the workon dl4cv  command to access the Python virtual virtual environment:

Figure 10: Accessing the dl4cv Python virtual environment for deep learning.

Notice that my prompt now has the text (dl4cv)  preceding it, implying that I am inside the dl4cv  Python virtual environment.

You can run pip freeze  to see all the Python libraries installed.

I have included a screenshot below demonstrating how to import Keras, TensorFlow, mxnet, and OpenCV from a Python shell:

Figure 11: Importing Keras, TensorFlow, mxnet, and OpenCV into our deep learning Python virtual environment.

If you run into an error importing mxnet, simply recompile it:

This due to the NVIDIA kernel driver issue I mentioned in Step #3. You only need to recompile mxnet once and only if you receive an error at import.

The code + datasets to Deep Learning for Computer Vision with Python are not included on the pre-configured AMI by default (as the AMI is publicly available and can be used for tasks other than reading through Deep Learning for Computer Vision with Python).

To upload the code from the book on your local system to the AMI I would recommend using the scp  command:

Here I am specifying:

  • The path to the .zip  file of the Deep Learning for Computer Vision with Python code + datasets.
  • The IP address of my Amazon instance.

From there the .zip  file is uploaded to my home directory.

You can then unzip the archive and execute the code:

Step #5: Stop your deep learning AWS instance

Once you are finished working with your AMI head back to the “Instances” menu item on your EC2 dashboard and select your instance.

With your instance selected click “Actions => Instance State => Stop”:

Figure 12: Stopping my deep learning AWS instance.

This process will shutdown your deep learning instance (and you will no longer be billed hourly for it).

If you wanted to instead delete the instance you would select “Terminate”.

Troubleshooting and FAQ

In this section I detail answers to frequently asked questions and problems regarding the pre-configured deep learning AMI.

How do I execute code from Deep Learning for Computer Vision with Python from the deep learning AMI?

Please see the “Access deep learning Python virtual environment on AWS” section above. The gist is that you will upload a .zip  of the code to your AMI via the scp  command. An example command can be seen below:

Can I use a GUI/window manager with my deep learning AMI?

No, the AMI is terminal only. I would suggest using the deep learning AMI if you are:

  1. Comfortable with Unix environments.
  2. Have experience using the terminal.

Otherwise I would recommend the deep learning virtual machine part of Deep Learning for Computer Vision with Python instead.

How can I use a GPU instance for deep learning?

Please see the “Step #2: Select and launch your deep learning AWS instance” section above. When selecting your Amazon EC2 instance choose a p2.* (i.e., “GPU compute”) instance. These instances have one, eight, and sixteen GPUs, respectively.

Summary

In today’s blog post you learned how to use my pre-configured AMI for deep learning in the Amazon Web Services ecosystem.

The benefit of using my AMI over the pre-configured virtual machine is that:

  • Amazon Web Services and the Elastic Cloud Compute ecosystem give you a huge range of systems to choose from, including CPU-only, single GPU, and multi-GPU.
  • You can scale your deep learning environment to multiple machines.
  • You retain the ability to use pre-configured deep learning environments but still get the benefit of added speed via dedicated hardware.

The downside is that AWS:

  • Costs money (typically an hourly rate).
  • Can be daunting for those who are new to Unix environments.

After you have gotten your feet wet with deep learning using my virtual machine I would highly recommend that you try AWS out as well — you’ll find that the added speed improvements are worth the extra cost.

To learn more, take a look at my new book, Deep Learning for Computer Vision with Python.

, , , , ,

18 Responses to Pre-configured Amazon AWS deep learning AMI with Python

  1. Sayan September 22, 2017 at 3:05 am #

    Thank you for this article, as it was very informative. I was initially planning to build a desktop PC for deep learning. But you convinced me to try out AWS first. If I calculate the initial cost of building a PC In India, then it would roughly translate to running an AWS p2.xlarge instance 3 hrs. daily for around 2.5-3 yrs. On top of that I don’t have to worry about the maintenance, and the electricity bills.

    • Adrian Rosebrock September 22, 2017 at 8:53 am #

      Not having to worry about maintenance is a big reason why I like cloud-based solutions for deep learning. Even if you botch your instance you can always start fresh with a new one. And when new hardware becomes available you can simply move your code/data to a new instance. It’s great to hear that you decided to go with the AMI!

  2. Anthony The Koala September 24, 2017 at 5:52 pm #

    Dear Dr Adrian,
    Thank you for your information regarding the use of the Amazon “cloud service”. Please excuse my naivety but wish to ask a practical question on implementing deep learning locally.

    Could I achieve the same thing if I had a very large disk drive dedicated to deep learning say 1TB drive or say a 250GB solid state drive and do my deep learning ‘locally’. Perhaps having another RPi acting as a server to the very large storage device?

    Thank you,
    Anthony of Sydney Australia

    • Adrian Rosebrock September 26, 2017 at 8:34 am #

      Your hard drive space isn’t the only concern. The issue here is your CPU and/or GPU. The Amazon cloud allows you to use GPUs for deep learning. If your local system has a GPU, yes, I would recommend using it. If it doesn’t, then the Amazon cloud would be an option.

      Again, you do not have to use GPUs for deep learning but they will tremendously speed up the training process. Some very deep networks that are trained on large datasets can only be accomplished by using the GPU.

  3. Matias Figueroa September 30, 2017 at 3:08 pm #

    Adrian Rosebroc, what kind of video card do you recommend to create this type of project, would a GTX1080 11gb suffice ??,
    or some more economical model like gtx980 . thank’s a lot for sharing your knowledge

    • Matias Figueroa September 30, 2017 at 3:10 pm #

      sorry for writed wrong your last name

    • Adrian Rosebrock October 2, 2017 at 9:50 am #

      The GTX1080 is perfectly acceptable. I also recommend the Titan X 12GB. As long as you have more than 6GB (ideally 8GB or more) you’ll be able to run the vast majority of examples inside Deep Learning for Computer Vision with Python.

  4. santa October 15, 2017 at 3:24 pm #

    I imagine combining Aws with OpenCV and a Rest Web API for an MVC model. You had another tutorial on web interfaces (https://www.pyimagesearch.com/2015/05/11/creating-a-face-detection-api-with-python-and-opencv-in-just-5-minutes/). In that tutorial you have processed images.

    I am now wondering if this combination would work with a video stream (say a RTSP video stream), instead of posting pictures.

    • Adrian Rosebrock October 16, 2017 at 12:22 pm #

      I don’t have any tutorials on working with RTSP streams, but it’s a bit more complicated to setup the client/server relationship. I’ll try to cover this in a future blog post.

  5. Rob Jones October 20, 2017 at 1:34 pm #

    I’ve started running the examples from the DL4CV book on a p2.xlarge instance – works great – getting 6s per epoch on a p2.xlarge which has a Tesla K80.

    Some of the examples produce a training loss graph – in order to view these you need to use X forwarding.

    That’s easily done – just add -X when you ssh

    $ ssh -X -i ubuntu@

  6. Rob Jones October 20, 2017 at 4:55 pm #

    That last comment had some text removed – should have read

    $ ssh -X -i your_key ubuntu@your_ip

    In fact the -X flag allows X forwarding but with a timeout – 10 mins maybe

    Instead you want to use -Y which does the same but without a timeout

    $ ssh -Y -i your_key ubuntu@your_ip

    • Adrian Rosebrock October 22, 2017 at 8:39 am #

      Thanks for sharing, Rob!

      If you’re using the AMI I would also suggest using plt.save rather than plt.imshow. This will allow the figure to be saved to disk, then you can download it and view it.

  7. kaisar khatak October 28, 2017 at 8:22 pm #

    How does your AMI (deep-learning-for-computer-vision-with-python) compare to Amazon Deep Learning AMI CUDA 8 Ubuntu Version AMI and NVIDIA CUDA Toolkit 7.5 on Amazon Linux AMI?

    Thanks.

    • Adrian Rosebrock October 31, 2017 at 8:04 am #

      My AMI focuses on deep learning for computer vision. Additional image processing/computer vision libraries are installed such as scikit-image, scikit-learn, etc. General purpose deep learning libraries (such as ones for NLP, audio processing, text processing, etc.) are not installed. This AMI is also geared towards readers who are working through Deep Learning for Computer Vision with Python.

  8. kaisar khatak October 28, 2017 at 9:55 pm #

    Who owns the deep-learning-for-computer-vision-with-python AMI in the East (N. Virginia) region?

    • Adrian Rosebrock October 31, 2017 at 8:01 am #

      I have not created an AMI in the N. Virginia region, only the Oregon region. I assume a PyImageSearch reader replicated the AMI; however, I would suggest you use the OFFICIAL release only.

  9. Rob Jones November 24, 2017 at 4:57 pm #

    I have an instance of this AMI that has been working fine – stop it, start it with no problem – but every once in a while it seems to lose the nvidia driver when I start it.

    tensorflow/stream_executor/cuda/cuda_diagnostics.cc:145] kernel driver does not appear to be running on this host (ip-172-31-34-37): /proc/driver/nvidia/version does not exist

    I can get it back with ‘cd installers; sudo ./NVIDIA-Linux-x86_64-375.26.run –silent’ as you showed above.

    Not a big deal…just odd…

    • Adrian Rosebrock November 25, 2017 at 12:20 pm #

      It happens eery now due to how Amazon handles the kernels on the AMIs. I’m not sure entirely how it works to be totally honest. I discuss it in more detail over in this blog post.

Leave a Reply