Remote development on the Raspberry Pi (or Amazon EC2)

In this blog post, you will learn how to perform remote development on a Raspberry Pi (or Amazon EC2 server). You will discover how to configure your host machine to connect to these systems and remotely perform Python development.

Today’s tutorial is actually a chapter for my upcoming book, Raspberry Pi for Computer Vision.

As I was writing the chapter I realized that the guide wasn’t specific to the RPi and could actually be used for anyone who needed to configure a remote system for development (and in particular, using the Python programming language).

Therefore, I decided to take the chapter and convert it into a blog post.

Using this tutorial you will learn how to configure:

  1. Your Raspberry Pi for remote computer vision development
  2. An Amazon AWS server for remote deep learning
  3. …or any other system that you can connect to remotely!

To learn how to configure your Raspberry Pi, Amazon EC2 instance, or server for remote Python development, just keep reading!

Remote development on the Raspberry Pi (or Amazon EC2)

In the first part of this tutorial, we’ll discuss the concept of remote development and why it’s not only convenient for us as programmers, but also why you should be doing it when writing code on your non-host machine.

From there I’ll provide you with my three favorite methods to connecting to a Raspberry Pi, AWS instance, or remote server.

We’ll then discuss two easy to use methods for transferring files from your host machine to a remote system.

Finally, we’ll wrap up the post with a discussion of Python IDEs that can be used to write code on a remote system.

What is remote development?

Figure 1: Remote Python development flowchart.

Remote development assumes that you have two computers:

  1. Your host machine (also called your local machine) that has your monitor, keyboard, and mouse attached to.
  2. The remote machine where your code is executing.

Or, more simply put:

  1. Your host machine is your laptop/desktop sitting on your desk where you normally work.
  2. Your remote machine is the machine you login to via your host machine.

To perform development you first connect to the remote machine from your host.

There are a number of ways to connect to the remote machine, including SSH, VNC, Jupyter Notebooks, and others. We’ll be covering these methods later in this guide.

Once you’re connected to the remote machine you can then:

  1. Write code on the remote machine
  2. Execute code on the remote machine

Figure 1 illustrates the concept of using your host machine to login to a device, writing code on the remote system, and then executing it.

While you are still typing using the keyboard on your local system, the commands/code itself are actually being executed on the remote machine.

Why would I want to write my code remotely?

If you’re working with a cloud-based instance such as Amazon EC2, Microsoft Azure, etc., then you do not have physical access to the machine (meaning there is no keyboard/mouse to use).

In that situation you need to login to the server and perform remote development — and you’ll want to ensure that your local host is configured properly for remote development.

If you’re using a Raspberry Pi then you certainly could connect a keyboard, monitor, and mouse, but that’s not always necessary!

First, you’ll have the hassle of setting up the additional hardware — it’s clunky to set up, you’ll lose space on your desk, and worst of all, you’ll constantly be switching back and forth between your RPi keyboard and your laptop/desktop keyboard.

Secondly, even with the release of the Raspberry Pi 4, the RPi itself is still very underpowered compared to your laptop/desktop — you won’t (and shouldn’t) be able to utilize powerful programming IDEs on the Pi itself.

When it comes to Python development, PyCharm is quite literally the standard.

However, IDEs like PyCharm, while super powerful, are resource hogs, consuming quite a bit of RAM and CPU cycles — on a resource-constrained device such as the RPi, running such an IDE is computationally wasteful.

Instead, you should:

  • Configure your local machine to connect to the Pi
  • Code on your laptop/desktop
  • And then have those files transferred (potentially automatically) to the Pi so you can run the code itself on the remote machine.

In the remainder of this tutorial, I’ll show you how to configure your host machine to create a seamless transition, making it feel like you’re writing code on your host but you’re actually writing code on the remote system.

Connecting to a remote machine or Raspberry Pi

There are a number of methods, protocols, and procedures to connect to a remote machine.

The methods discussed in this section are certainly not exhaustive, but they are my favorites and ones you will absolutely encounter when performing remote development.

Secure Shell (SSH)

Figure 2: Secure Shell (SSH) is essential to remote development with Unix systems such as Ubuntu, macOS, EC2, and Raspbian.

The first method, and arguably the most simple, is to utilize Secure Shell (SSH), which is the ssh  command on Unix machines.

To use SSH you need:

  • An SSH server installed and running on your remote machine.
  • An SSH client installed on your local machine.
  • The IP address, username, and password of the remote system.

Most systems will have an SSH server. On Ubuntu, if you need to install an SSH server, simply:

On macOS, you only need to enable “Remote Login”:

See below for two options to enable an SSH server on the Raspberry Pi.

If you’re working with a Unix-based machine (such as Linux or macOS), the ssh  client is installed by default. If you are using Windows you may need to install it

Note: I haven’t used Windows in over 10+ years and do not officially support Windows on the PyImageSearch blog. The last I checked PuTTY was still a good option but I’ll leave SSH install on Windows up to you.

Provided you have SSH installed, first open a terminal/command line on your host machine.

You’ll then need the following:

  1. The IP address of the machine you are connecting to.
  2. The username of the user you’ll be logging into on the remote machine.
  3. The password of the respective user.

Once you have that information, you can use the following command as a template:

If you are using Amazon EC2, you may need to supply a private key instead of a password:

For example, if the IP address of my remote machine is 10.0.0.90 and my username is adrian , the ssh command becomes:

I can then supply my password and login to the remote machine. Once logged in, I can now execute any command I wish (provided I have the proper permissions, of course) on the remote machine.

X11 forwarding

Figure 3: X11 forwarding allows a remote system to render a GUI window on your local system. X forwarding is useful for remote development over high-speed connections and when the GUI is relatively static. It does not perform well for animations and video.

Did you know that you can use SSH to see basic GUI windows from your Python scripts and programs?

First, you need an X-Window manager:

  • If your host is Ubuntu (or another Linux distro), it is built-in.
  • If your host is macOS, you should download + install xQuartz.
  • If your host is Windows, I think you need Xming these days still (It has been over 10 years since I’ve used Windows so you should do your own research).

All you need to do is enable X11 forwarding via the -X switch:

Figure 4: A remote X11 window displayed on my Mac using the Xquartz app and an SSH connection.

Here I’ve launched a Python script named example.py. This script loads an image from disk and displays it to my local host’s screen via cv2.imshow call.

Since I have X11 forwarding enabled, the window opened via OpenCV is displayed on my host machine. Remember, the code is executed remotely and the image is rendered remotely but displayed locally.

You might be wondering:

Is X11 forwarding useful for video streaming?

In short, the answer is “no, don’t bother.” The X11 forwarding protocol doesn’t compress the images at all or effectively to make video streaming possible. Do not attempt to use X11 forwarding for apps where the content changes often or there are video feeds.

A quick note on SSH on the Raspberry Pi

It’s important to note that SSH is turned off by default on the Raspberry Pi, meaning that you cannot connect to your RPi via SSH without first enabling it.

There are a few methods to enable SSH on the Raspberry Pi.

Option 1: You may enable it via the Desktop user interface:

Figure 5: The Raspberry Pi Configuration desktop GUI is accessed from the main menu.

From there, find the “Interfaces” tab and enable SSH:

Figure 6: Enabling SSH on the Raspberry Pi via the Desktop. SSH is essential for remote development on the Raspberry Pi.

Option 2: The second method is to use the following command to enable it:

The problem with this method is that requires you to have physical access to the RPi to type in the command.

The second issue is that if you reboot the Raspberry Pi, SSH will not automatically start (i.e., starting SSH will not persist across reboots).

Option 3: Similar to Option 1, you may enter the Raspberry Pi Configuration via the command line:

Figure 7: The raspi-config command allows you to change system settings.

From there, you’ll be presented with a terminal menu:

Figure 8: Select “5 Interfacing Options” to find SSH and other interface settings for the Raspberry Pi. Once you turn on SSH you can begin remote development with your Raspberry Pi.

Then you’ll be presented with the interfaces, one of which is SSH:

Figure 9: SSH for Raspberry Pi can be turned on from the “P2 SSH” menu item.

A reboot will be required. From there you can use the Raspberry Pi’s IP address to connect to it remotely. To find your RPi’s IP address, simply use the ifconfig  command:

Figure 10: To find the IP address of your Raspberry Pi, use the ifconfig command. Here my RPi is connected to my wlan0 interface on the 192.168.1.150 IP.

Another way to find a Raspberry Pi on your network is to go to our router’s client page. This is useful if you don’t have a screen and keyboard/mouse connected to your RPi at the time.

Virtual Network Computing (VNC)

Figure 11: VNC is a graphical way to connect to remote systems such as the Raspberry Pi or another computer on your network that has a desktop environment.

If you prefer a graphical user interface (GUI) instead of the command line when working on your remote machine, then you can use VNC.

To use VNC you need:

  • A VNC server installed and running on your remote machine.
  • A VNC client installed on your local machine.
  • The IP address, username, and password of the remote system.

However, keep in mind that VNC assumes you have GUI/desktop manager running on your remote system.

For the Raspberry Pi, that’s a perfectly fine assumption to make — the RPi launches a window manager on boot (provided you are using the standard install of Raspbian, of course).

However, for a remote server, such as Amazon EC2 or Microsoft Azure, you likely don’t have a window manager.

In those situations, you cannot use VNC and should instead use SSH.

How to install a VNC server is dependent on your operating system so you’ll want to do your own research on that (as I cannot provide instructions for every single operating system).

That said, I have provided VNC server install instructions for Raspbian and Ubuntu, two of the most popular operating systems you’ll want to install VNC server on.

Installing VNC Server on Raspbian

Luckily for us, it is easy to get VNC going on the Raspbian OS.

You have two options:

Option 1: Use the Desktop configuration to change the setting:

Refer to Figure 5 to open the “Raspberry Pi Configuration” user interface. Then enable VNC via the radio button:

Figure 12: Enabling VNC on the Raspberry Pi via the desktop user interface.

This method requires a reboot.

Option 2: Enable VNC Server with the command line:

At the command line, run sudo raspi-config  (refer to Figure 7).

Then, navigate to the “5 Interfacing Options” menu item (refer to Figure 8).

Finally, select “P3 VNC”:

Figure 13: Enabling VNC via the command line on the Raspberry Pi. We used the raspi-config command to open the configuration settings.

Enable VNC and you’ll be good to go after a reboot.

Installing VNC Server on Ubuntu

There are multiple methods to install a VNC Server on Ubuntu.

Option 1: Given that my preference is Real VNC (for both server and client), you could start by installing their .deb file for the easiest method:

Figure 14: Installing the RealVNC branded VNC Server on Linux can be accomplished with their .deb package file.

To install a .deb file in Ubuntu, you can double click the icon in the file browser/desktop. Alternatively, you may run .deb files from the command line:

Option 2: Alternatively, there is plenty of information online on how to manually install a VNC server from the command line.

Rather than regurgitating someone else’s installation guide here, I’m providing links to promising tutorials:

If you find a tutorial that you particularly like for installing VNC via the command line, please share it in the comments section of this post.

Real VNC Client

My preferred VNC client is Real VNC which is available for macOS, Linux, Windows, and other operating systems.

You can download the Real VNC client appropriate for your system using this link.

Once you have both (1) VNC server running on your remote machine and (2) the VNC client running on your local machine, follow the steps in Figure 11:

  1. Enter the IP address of the remote machine.
  2. Enter the username and password for the account on the remote system.
  3. Enjoy your VNC connection!

From there you have remote access to the desktop and GUI manager on the remote machine.

So, what’s the catch?

The problem with VNC is that it’s slow due to network latency. Even on local networks, VNC can appear to be a bit “laggy” at times.

I personally only use VNC when:

  1. I need to access my Raspberry Pi on a local network.
  2. I need to run Python scripts that utilize cv2.imshow so I can visualize the output.

If your network connection is strong, you can use VNC and cv2.imshow calls to debug real-time Python and OpenCV scripts; however, if your connection is weak and you consistently need to use cv2.imshow, you should simply use a physical monitor attached to your “remote” machine.

Jupyter Notebooks

Figure 15: Jupyter Notebooks are great environments if you can stay contained in your web browser. The community loves Jupyter Notebooks for sharing code demos. There are times when you’ll find that Jupyter limits your development, however.

Jupyter Notebooks have become super popular for Python development.

I encourage you to use them, provided you understand that Jupyter Notebooks are not a replacement for the command line — they are just a different tool in your tool belt.

Just because you can execute commands and code via a Jupyter Notebook does not mean you can ignore learning the command line (and vice versa).

To be a well-rounded developer you need to learn how to use both.

The benefit of Jupyter Notebooks is that you have an interactive IDLE environment that you access via your web browser — you simply point your web browser at the IP address/port of the Jupyter Notebook running on your remote machine and access it. To utilize Jupyter Notebooks you first need to install it on your remote machine:

If you are using Python virtual environments (recommended) you will need to install the jupyter package into whichever Python environment you use to use Jupyter Notebooks inside of.

As a quick sanity check, you can launch a jupyter notebook via:

Your web browser will launch automatically if your notebook is running on the local machine and you are also on your local machine.

Figure 16: To create a new Python Jupyter Notebook file, simply use the “New” menu in the corner. You may also launch a web-based terminal from this menu.

If your notebook is running remotely, you need to launch a web browser and enter the server IP address and port ( http://10.0.0.90:8888 ) into the URL box.

From there, you can create a new notebook by pressing “New” and “Python 3” as shown in Figure 16.

You can insert Python code or system commands into a Notebook in “Cells”. Go ahead and press “Insert”“Insert cell below” a few times. From there you can enter Python code normally. System commands are preceded with !  (refer to cell “In [3]” of Figure 15).

What about displaying images in Jupyter?

Jupyter does not have access to your system GUI.

Therefore, cv2.imshow  and any other GUI command will not work.

If you would like to display images inside your Jupyter notebook, I recommend using matplotlib.

The only caveat is that matplotlib’s default channel ordering for display is RGB. As you likely know, OpenCV’s default is BGR. So you must swap the Blue and Red channels ( cv2.cvtColor ) prior to displaying with matplotlib in a Jupyter notebook. Here’s a quick example:

Figure 17: Displaying images in Jupyter notebooks for remote computer vision development or developing tutorials.

For further details, be sure to refer to my previous post.

Transferring files to your remote machine or Raspberry Pi

In this section, you will learn how to transfer files from your host machine to your remote machine.

There are many methods to accomplish this task, but my two favorite methods include using SCP/SFTP or using a remotely mounted drive.

Manual SCP or SFTP

Figure 18: Transmit is a macOS app for uploading/downloading files to/from remote machines using FTP/SFTP.

If you need to transfer files to your remote machine (such as moving the code associated with your purchase of Raspberry Pi for Computer Vision from your laptop to your RPi), I highly recommend using SCP/SFTP.

If you are using a Unix-based machine you can use the SCP command:

Here I am moving a file named foo.py  from local machine to my remote machine in the ~/foobar_project  folder.

Note how I am supplying the name of the file along with the IP address and username of the remote machine (the password is supplied after I execute the command).

Similarly, I can transfer files back from my remote machine to my local machine:

This command transfers a file named bar.py  back to my local machine in my local machine’s ~/foobar_project  folder.

If you prefer to use a GUI-based program for SCP/SFTP I would recommend using FileZilla (all OSes) or Transmit for macOS (Figure 18).

Using these programs you can simply “drag and drop” files back and forth.

Of course, the biggest problem with SCP and SFTP is that they are manualyou cannot automatically transfer files back and forth (at least not without additional commands).

The “Developing Python scripts remotely via an IDE” section of this tutorial will show you how to automatically transfer files from your host machine to your remote system, but for now, let’s take a look at remote mounted drives.

Remote mounted drive

Figure 19: Using the SSH file system (SSHFS) you can remotely mount a remote system and access its file system, just like you would on your host machine (i.e., via macOS Finder, the Ubuntu/Raspbian GUI, Windows Explorer, etc.). (image source)

I’m not the biggest fan of remote mounted machines or drives, but they are worth noting here.

Using the SSH file system (SSHFS) you can remote mount a remote system and access its file system, just like you would on your host machine (i.e., via macOS Finder, the Ubuntu/Raspbian GUI, Windows Explorer, etc.).

The problem I’ve always encountered with remote mounted devices is that:

  1. Connections mysteriously drop.
  2. Files don’t always transfer.

Instead, if I want a file to sync automatically (such as when I save/update a file), I use a dedicated IDE, which is exactly what we’ll be covering in the next section.

Developing Python scripts remotely via an IDE

In this section, you will learn about two software development IDEs that will make working with Python on remote machines a breeze.

Sublime Text with SFTP plugin

Sublime Text is a popular code editor.

The editor is free but you can purchase an optional license (which I recommend you do if you find yourself using Sublime Text — always be sure to support those who build software you regularly use!).

One of the benefits of Sublime Text is that it allows for extendability via plugins/packages.

For remote developing you should use the Sublime Text SFTP plugin. This plugin is also free but has an optional paid version (which again, you should consider purchasing if you use the SFTP plugin often). The installation instructions for the plugin are listed here.

Once installed, you can launch the Sublime Text editor:

Figure 20: Sublime Text, a popular text/code editor can work with remote files using a plugin.

Open the command palette via “shift + command + p”.

From there, select “SFTP: Setup Server…”:

Figure 21: Selecting the SFTP setup via Sublime Text’s command palette.

You’ll then be prompted with a template JSON file with options you can fill out.

Figure 22: Sublime Text’s SFTP configuration file needs to be updated before you can connect to a remote system for development.

You’ll want to edit the host and user to be the IP address and username of your remote machine. You can also specify the password if you want to avoid entering it each time.

You should also specify the remote_path  which is the default path of where you’ll be placed when logging in — I typically recommend setting the remote_path  to be the path of the home directory for the user.

After editing, my SFTP configuration looks like the following:

Figure 23: A custom Sublime Text SFTP configuration for a Raspberry Pi on my network.

Next, save the file:

Figure 24: Saving the SFTP configuration with a name that I’ll recognize later if I want to work on this remote system using Sublime Text again.

Notice how I am prompted with a dialog box asking me to name the file. You should give the file a descriptive name. Here I am naming my file rpi_office to indicate that this is the Raspberry Pi in my office.

After saving the file I can then select “SFTP: Browser Server…” (via the command palette again):

Figure 25: From the Sublime Text command palette, select “SFTP: Browse Server…” so that you can log in to a remote system and list files.

And then select my “rpi_office”:

Figure 26: Selecting my “rpi_office” remote SFTP configuration in Sublime Text.

I then enter my password and am dropped into the /home/pi  directory.

From there I select “Folder Actions” followed by “New File” to create a new file named test.py :

Figure 27: Creating plus naming a new file called test.py on a remote system (Raspberry Pi) via the Sublime Text SFTP plugin.

I can then save the file, the contents of which are automatically updated on the Raspberry Pi.

Executing the script from an SSH connection, you can see the text “Hello, PyImageSearch Reader!” is displayed to my terminal (again, noting that the code itself was executed on the Raspberry Pi and not my local machine).

Figure 28: As you can see, I’m SSH’ed into my remote Raspberry Pi. I’ve executed the Python script that was just saved to the remote system automatically via Sublime Text’s SFTP plugin.

A quick note to Amazon EC2 users

If you are using Amazon EC2 or any other authentication method that requires an SSH key, you need to edit the sftp_flags  option during server setup.

Specifically, you’ll want to edit to look like the following:

Figure 29: Using Sublime Text’s SFTP plugin with Amazon EC2 systems requires that you insert the path to your encryption key (.pem file) in your configuration.

Notice how I have updated sftp_flags to include the -i option, followed by the path to my SSH key. The SFTP plugin then knows to perform the proper authentication using this file.

Note: While you edit remote code on your local machine, your system will automatically generate/download temporary files that you see on your screen. When you “Save” your code, it is automatically updated to the remote machine. If you ever also need these files on your local system, I would recommend using SCP/SFTP to transfer the files. You may also use the “Save as” command in Sublime to save a file locally.

PyCharm with automatic upload deployment

If you are looking for a more full-fledged IDE, I recommend using PyCharm which has essentially become the de facto standard for Python IDEs.

I’m a huge fan of PyCharm, and if my Python code ends up spanning more than 1-2 files, I tend to use PyCharm over Sublime Text. That said, PyCharm is a big IDE — it’s not small and minimal like Sublime Text is.

Similar to Sublime Text, we can also configure PyCharm to:

  1. Use a remote Python interpreter (including a Python virtual environment on a remote machine).
  2. Automatically transfer edited files, ensuring that code is updated on your remote machine.

This feature requires that you have PyCharm Professional installed (click here to see the features and benefits of PyCharm Professional).

Assuming you have PyCharm installed, let’s create a project that performs automatic upload deployment.

Start by creating a new project:

Figure 30: Creating a new project in PyCharm locally prior to setting it up for remote development.

I have named this project “RPiTest”.

You’ll then want to expand the “Project Interpreter” option to expand the Python interpreter options:

Figure 31: In PyCharm, select “Python Interpreter” to configure the Python environment for the project.

Select the “Existing interpreter” radio button and then click the “…” which will open a new window similar to the following:

Figure 32: Select the “Existing Interpreter” option in PyCharm.

Select the “SSH Interpreter” option and then “New server configuration”:

Figure 33: Select “New server configuration” and enter the IP address, and username of the remote system.

Here I have entered the IP address and username my Raspberry Pi.

Click the “Next” button and then enter the password for your server:

Figure 34: Enter your remote system’s password and ensure that you save the password (checkbox) so that you don’t need to enter it upon each save.

Make sure you check the “Save password” checkbox to ensure your password is saved, otherwise you will have to re-enter it each time you want to save/modify a file (which will become quite frustrating and tedious).

The next window will prompt you to select a Python interpreter, which will by default be /usr/bin/python:

Figure 35: Be sure to select the virtual environment on the remote system that you’d like to use as your Python interpreter.

Since we’re using Python virtual environments we should instead click the “…” and navigate to the python binary inside the /home/pi/.virtualenvs/py3cv3/bin/python directory:

Figure 36: Navigate to the Python executable within the appropriate .virtualenvs folder to select your remote interpreter.

Select “Ok” to confirm the Python virtual environment choice. Notice how the Python interpreter path has now been updated to to be /home/pi/.virtualenvs/py3cv3/bin/python. Click the “Finish” button to finish creating the Python virtual environment.

PyCharm will now provide us with details on our “Existing interpreter” along with the “Remote project location”:

Figure 37: Be sure to edit the PyCharm “Remote project location” under the “Existing Interpreter” section so that your project files are stored in a known location.

However, we don’t want our project to live in /tmp/pycharm_project_409 on the Raspberry Pi. Instead, we want this project to live in /home/pi/RPiTest. To make this update, you can either use the “…” button to navigate or simply manually type in the path into the field.

Click the “Create” button and then the PyCharm IDE will then load:

Figure 38: PyCharm Professional has now been configured for remote development over an SSH/SFTP connection.

It may take 5-10 minutes for PyCharm to index the Python virtual environment and project files on your Raspberry Pi, so be patient.

Let’s add a new file to the project.

Right click on the RPiTest directory in the project and then select “New” and “File”:

Figure 39: Adding a new Python file to our PyCharm remote project.

We’ll name this file pycharm_test.py:

Figure 40: Saving a Python script on a remote machine via PyCharm called pycharm_test.py.

I’ve updated the file to include the following code:

Figure 41: A test script was developed and saved to a remote system via PyCharm.

Notice how PyCharm is indicating that the file has been automatically uploaded to the Raspberry Pi for us. You can validate that the file has been automatically uploaded by logging into the Raspberry Pi via a terminal and checking the file contents:

Figure 42: Over an SSH connection in my terminal, I’ve verified that after saving a file via PyCharm to my remote Raspberry Pi, that the file is, in fact, updated.

When it comes to remote Python development it’s hard to beat PyCharm with automatic upload.

Yes, there are more steps involved (at least when compared to configuring Sublime Text), but once you have it working, it’s well worth it!

Note: While you edit remote code on your local machine, your system will automatically generate/download temporary files that you see on your screen. When you “Save” your code, it is automatically updated to the remote machine.

Executing code via the command line

When performing remote Python development, my personal recommendation is to:

  1. Use either Sublime Text or PyCharm with automatic SFTP enabled.
  2. Execute my Python scripts via command line using SSH.

You are free to use whichever option you prefer. Perhaps you like to use Jupyter Notebooks. Or maybe you feel like you have more control using manual SFTP upload. All options are perfectly okay, it’s just a matter of your own preference.

That said, I’m going to give you a quick demonstration of executing a script via the command line over SSH.

First, I am going to login to my Raspberry Pi (or AWS server) using SSH:

I have listed the contents of my home directory and see I have an existing file named simple_example.py.

I’d like to edit this file so I’m also going to login via Sublime Text and open it:

Figure 43: I’ve opened a file on my remote system using Sublime Text’s SFTP plugin. Now we’re going to edit the file an execute it.

Here you can see simple_example.py  is now open in Sublime Text.

I’m going to take the opportunity to edit this file and include a command line argument — the code listing now looks like:

I save the file via Sublime Text and then switch back to my terminal. I can then execute the Python script via:

Note how I’ve supplied the command line argument to the script — we use command line arguments quite often on PyImageSearch so make sure you take the time now to understand how they work.

Also, note how the script has been executed not on my local machine, but on the remote Raspberry Pi.

Summary

In this tutorial, you discovered my personal recommendations when performing remote Python development on the Raspberry Pi, Amazon EC2, or any other remote server instance.

When you’re connecting to the machine I recommend using:

  • SSH
  • VNC
  • Jupyter Notebooks

When editing code remotely I suggest you use either:

  • Sublime Text with SFTP plugin
  • PyCharm with automatic upload deployment

My personal preference is to edit code via either Sublime Text or PyCharm and then execute via SSH + command line. That said, you should take the time to play with each option. One method is not always better than the others and you may find you prefer one method over the other in the majority of situations — that’s perfectly okay!

Keep in mind that these are all tools in your tool belt, learn how to use them all and you’ll have more tools available to you.

I hope you enjoyed this tutorial!

To be notified when future tutorials are published here on PyImageSearch (and obtain my free computer vision and deep learning resource guide), just enter your email address in the form below!

, , , , , ,

37 Responses to Remote development on the Raspberry Pi (or Amazon EC2)

  1. Denis Brion July 1, 2019 at 11:29 am #

    With Windows (at least <=7), eaasiest way I found to use ssh was with cygwin (cygwin-x) .

    Both ssh and ssh -Y (forwarding) and scp work very easily with a RPi (or a nanoPi) with Windows giving a(graphical) screen, keyboard…..

    Other advantage of cygwin (though it might disappear with virtual box -10 years old; eats more RAM- , or with M$ mixing Windows and GNUlinux parts -in some future-) is that you can write simple programsM/scripts, test them on something like GNUlinux, and then port them with very few changes….

    BTW: vim (installed on RPi though apt-get install cream …) works fine on a remote connection (is purely text: no need to foroward X) …. and eats few resources.

    Both ways of enabling ssh on RPi are the same (raspi-config and a graphical wrapper); people with exotic keymaps (azeri, albanese, german, French, Spanish) need raspi-config to have a comfortable keymap… and I hope they atre able to cope wwith other options such as ssh enabling…..)

    • Adrian Rosebrock July 4, 2019 at 10:34 am #

      Thanks for sharing, Denis!

  2. Dexter July 1, 2019 at 11:31 am #

    Thanks for the cool tutorial, Dr. Adrian! Just wanted to ask if you’ve tested the binarized AI models of XNor.AI in a Raspberry Pi Zero as their AI models are optimized for these low power, low compute devices? Or if you’re interested by any chance to write a tutorial as a blog post here and include it in your upcoming book? Here’s the link to XNor.AI’s binarized AI models: https://ai2go.xnor.ai/home/

    • Adrian Rosebrock July 4, 2019 at 10:34 am #

      Thanks for sharing, but no, I have not used them before.

  3. Ryan July 1, 2019 at 11:35 am #

    I don’t know for sure that it works with a raspberry pi, but VS Code’s “Remote – SSH” extension is really impressive for doing python/opencv remotely.

  4. Andrew Baker July 1, 2019 at 12:45 pm #

    Great tutorial. What if you are not using your home network? I’m at the coffee shop and want to access my machine at home? Would the steps be the same or do I need to make some additional changes to my home router?

    • Adrian Rosebrock July 4, 2019 at 10:33 am #

      You’ll want to configure your home routers “port forwarding” rules. That way you SSH your home IP address with a specific port supplied and your router will know to route that to the RPi.

  5. David Bonn July 1, 2019 at 1:08 pm #

    Great post, Adrian.

    One of the things I found challenging about using AWS was how to transfer large-ish training datasets to my instance.

    My first approach used ssh + rdist, which was slow and clunky but basically worked.

    Later on I switched to using FileZilla to transfer to an S3 bucket and used s3fs to mount the bucket and build an HDF5 file from the dataset. That approach worked better but there are a lot more moving parts and some of them (s3fs) are kind of quirky.

    Oh, and PyCharm is awesome. Your previous posts led to my rediscovering it.

    • Adrian Rosebrock July 4, 2019 at 10:33 am #

      What I normally end up doing is splitting my .zip file into 1GB chunks, then uploading each individually. That helps prevent issues with large file transfers.

  6. wally July 1, 2019 at 1:36 pm #

    The only thing here new to me was the Jupyter notebooks thing, I’d encountered them but never really “got it”, so this was very helpful. This confirms its not my style — I prefer to stay out of web browsers if possible.

    But you forgot to mention the #1 way to improve your ssh workflow, especially if you use scp:

    ssh-copy-id user@remote.

    After entering the password for this connection and transfer, you’ll never be asked again from this host to that remote.

    The gist, done on the local host:
    First if you haven’t already done so create a public/private key pair (you must set a passphrase or it’ll nag for password to login defeating the purpose!):
    ssh-keygen -t dsa

    Follow the prompts, and when its complete copy the key with:
    ssh-copy-id username@remotehost

    The Coral development board requires this, and they make it a real PITA to copy over the key since password logins are disabled by default. They supply a utility you can use via the serial port connection, I cussed a lot getting it to work.

    Maybe I’m too old school minimalist, but I find Idle over an ssh -X connection works great, and if you have Python you have Idle with minimal effort.

    • Adrian Rosebrock July 4, 2019 at 10:32 am #

      I’m in agreement — Jupyter can be nice, but not for long running experiments. Secondly, sometimes people think that Jupyter is an excuse to NOT learn the command line. They often fall behind pretty quickly. If you’re reading this and use Jupyter consistently, take the next week and use exclusively the command line. You’ll learn a lot and thank me in the long run.

  7. Rammohan July 1, 2019 at 1:40 pm #

    It’s Good about how to handle raspberry on first time

    • Adrian Rosebrock July 4, 2019 at 10:31 am #

      I’m glad you enjoyed it!

  8. Steve Leighton July 1, 2019 at 3:47 pm #

    That’s a really helpful summary Adrian, I’ve bookmarked it for future reference!

    Quick question – every time I have to set up a new Pi or reinstall Raspbian I have to go and find a TV and keyboard etc to configure it before using it headless. It’s a bit of a pain. Is there any way to edit the Raspbian files installed on the SD card on a desktop machine before installing it in the Pi that would allow me to activate SSH and VNC? Thanks again!

    • Adrian Rosebrock July 4, 2019 at 10:31 am #

      Yes, you just need to add a blank “ssh” file (no file extension) to the boot partition of the SD card. Full guide here.

      • Steve Leighton July 4, 2019 at 11:03 am #

        Thanks. Would never have guessed it was such a trivial change!

  9. Bob July 1, 2019 at 3:53 pm #

    As a longtime Sublime Text user that has migrated to Visual Studio Code, I was happy to see that they recently added remote development support via Docker, ssh and WSL as well: https://devblogs.microsoft.com/python/remote-python-development-in-visual-studio-code/

    • Adrian Rosebrock July 4, 2019 at 10:30 am #

      Thanks for sharing, Bob!

  10. Leonard Bogdonoff July 1, 2019 at 6:42 pm #

    I had no idea about the X-Window manager. These little tips that I come across are why I love this publication! Thank you!

    • Adrian Rosebrock July 4, 2019 at 10:28 am #

      You are welcome, Leonard!

  11. Abhinav July 1, 2019 at 8:58 pm #

    Considering the fact that most of the people would not want to go for paid version of sublime and pycharm, the only option left is the eclipse with pydev environment installed in it. I have been using eclipse for python remote development and debugging for raspberry pi, for more than 2years now. Its completely free and supports remote development

    • Adrian Rosebrock July 4, 2019 at 10:28 am #

      I’m not here to tell you what IDE to use, just the IDEs that I prefer. If you like Eclipse (I don’t) the more power to you.

  12. Nathan Ton July 1, 2019 at 11:17 pm #

    Really nice post! Thanks for this.

    There’s just some small things I want to add. For using SFTP plugin with remote server like EC2, the property ssh_key_file works the same as the flag ‘-i [your-key-path]’. Also, VS Code has this same plugin so you have one more choice of IDE.

    • Adrian Rosebrock July 4, 2019 at 10:25 am #

      Thanks Nathan!

  13. Alang July 1, 2019 at 11:41 pm #

    Thanks for the great sharing.

    If some one prefer VS Code, refer to the following link to set up remote development.

    https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack&WT.mc_id=-blog-scottha

    • Adrian Rosebrock July 4, 2019 at 10:25 am #

      Thanks for sharing, Alang!

  14. Tuan Anh July 2, 2019 at 2:09 am #

    This is a very useful for LAN remote server. But how about remote over internet?
    for example my server put in my offices and my local in my home?

    • Adrian Rosebrock July 4, 2019 at 10:24 am #

      All methods covered here will work for remote “over internet” machines as well, provided you can access the IP address of the remote machine.

  15. Marcus Jaiclin July 2, 2019 at 7:37 pm #

    I find this very useful and consistent with my own experiences.

    One small suggestion: when I was reading your tutorial, I came across this question: “Is X11 forwarding useful for video streaming?” and your answer, which is “No.” I’ve been trying various solutions and am currently using X11 forwarding because it worked better than all the other not very good options I’d tried.

    Then, a couple of hours later, I found your blog post on ImageZMQ! Maybe add a quick sentence and link to that after your negative answer?

    Thanks for all your work!

    • Adrian Rosebrock July 4, 2019 at 10:21 am #

      X11 forwarding is very different than ImageZMQ though so perhaps I’m not understanding your question? Are you suggesting to use ImageZMQ to debug video streams remotely?

  16. Nico July 3, 2019 at 1:27 am #

    Also consider VSCode in the browser: https://github.com/cdr/code-server

  17. Gordon July 3, 2019 at 5:45 pm #

    Thanks for the great article. Althought I have known about many of these remote access methods, I still learned a few things. 1) Ditch NOOBS and just burn the desired OS to the SD card. Then, if you are not using a Windows system, (good for you!) the SD card can be mounted on the local host. This allows you to modify the setup and config files before installing it in the Raspberry Pi. Amoung the things you can do is to enable SSH, set a static IP address, and rename the Pi.
    I use these remote access methods at work to run Pi’s as controllers and data loggers and mini servers. More powerful machines are used as wi-fi sniffers. When the experiment is over, I use FTP, Samba, or NFS to gather all data, logs and traces to my laptop. It really is a great way to work without having to worry about managing a butt load of keyboards, monitors, and mice.

    • Adrian Rosebrock July 4, 2019 at 10:13 am #

      I definitely agree, don’t use NOOBS. Just use the default Raspbian OS, it will make life much easier.

  18. John Sheehan July 4, 2019 at 1:12 pm #

    Personally I’ve found that RDP has much smoother operation than VNC (or Xming) for remote desktop access. IIRC, I believe xrdp is also included in the default Rasbian distro.

    I’m also a big fan of PyCharm, but didn’t know about the remote option for general use, even though I’ve been doing just that with the MicroPython plugin ever since it was released. Thanks for the tip!

  19. Pranav Lal July 4, 2019 at 9:34 pm #

    Hi Adrian and all,
    This is an excellent primer on setting up remote development. Jupitor notebooks are not for me because they are still a pain when it comes to screen reader accessibility. The screen reader cannot detect the text entered into the cells of the notebook. I have to select the text, paste it into note[pad and, do my edits, ensure that I am on the right cell, select all the text and then paste the edited text. I am sticking to the command line. I will be trying vs code.
    Finally, for those who are using windows 10, there is indeed a built-in ssh client which is not bad. I have not found a way to enable logging and copying and pasting require more keystrokes than I like but it is one more option. You need to install the client from within “programs and features”.

  20. Yiyi July 9, 2019 at 12:28 pm #

    Thanks for the great articles. It is very useful. One quick question, when I try to connect my mac laptop to the host machine by using VNC. The VNC server in the host machine is set up, and the VNC viewer is also installed on my mac. However, connecting is not successful at all. I use the university internet. I am not sure whether it is because of this.

    • Adrian Rosebrock July 10, 2019 at 9:35 am #

      Make sure you can access the IP address of your server machine. It sounds like it may be behind a firewall or port forwarding is not configured on your router.

Leave a Reply

[email]
[email]