Montages with OpenCV

Today’s blog post is inspired by an email I received from PyImageSearch reader, Brian.

Brian asks:

Hi Adrian,

I’m really enjoying the PyImageSearch blog. I found your site a few days ago and I’ve been hooked on your tutorials ever since.

I followed your tutorial on building an image search engine, but instead of displaying the result images one-by-one (like you did), I want to display the top-20 results in a montage.

Is there a way to do with OpenCV?

Thanks,

Brian

Great question Brian, thanks for asking.

One of my favorite aspects of running the PyImageSearch blog is being able to chat with you, the reader, and discover the projects you’re working on.

It’s especially exciting when I can take questions or comments and turn them into blog posts — that way the entire PyImageSearch community is able to benefit from the answer.

Today we will learn how to build montages of images using OpenCV and the imutils package. A big thank you to Kyle Hounslow who contributed the build_montages  function to imutils.

To learn more about building an image montage with OpenCV, just keep reading.

Looking for the source code to this post?
Jump right to the downloads section.

Montages with OpenCV

There are four primary pieces to today’s blog post.

In the first part, we’ll learn how to build a list of image paths from an image dataset residing on disk.

From there, we’ll use the build_montages  function to take this list of images and create the actual montage.

Next, we’ll display the montage to our screen.

Finally, I’ll provide an example of using montages to display images with OpenCV.

To download the source code + example images to this blog post, be sure to use the “Downloads” section below.

Creating a montage with OpenCV

To get started, open up a new file, name it montage_example.py , and insert the following code:

Lines 2-6 import our required Python packages. Notice how build_montages  is imported from the imutils package.

If you do not have imutils  installed on your system (v0.4.3 as of this writing), then make sure you install/upgrade it via pip :

Note: If you are using Python virtual environments (as all of my OpenCV install tutorials do), make sure you use the workon  command to access your virtual environment first and then install/upgrade imutils .

From there, we can parse our command line arguments:

Our script requires one command line argument, followed by a second optional one, each detailed below:

  • --images : The path to your directory containing the images you want to build a montage out of.
  • --samples : An optional command line argument that specifies the number of images to sample (we default this value to 21  total images).

Next, we can use the --images  path to randomly select some input images:

To obtain a listing of all image paths inside the --images  directory, we make a call to the list_images  function (Line 18).

For the purpose of this exercise we randomly shuffle the image paths on Line 19, followed by taking a sample of these images to display to our screen (Line 20). The set of imagePaths  returned by this sampling will be used to build our montage.

For your own applications you likely will not have to bother with randomly shuffling and selecting a set of image paths — you will already have your image paths.

In the context of Brian’s original question, he is looking to display the results of his image search engine.

The results therefore contain his image paths.

Again, keep in mind that we are simply demonstrating how to build a montage with OpenCV — how you actually use this example is entirely up to you.

Given our imagePaths , we are ready to build the montage:

On Line 23 we initialize our list of images .

We then loop through the imagePaths  on Lines 26-29, loading each image  from disk, and then appending the image  to our images  list.

To actually construct the montage, we make a call to the build_montages  function on Line 32 — this is where all of the heavy lifting is done. If you’re curious about the internals of the build_montages  method and what is going on under the hood, be sure to check out the source code implementation on GitHub.

The build_montages  function requires three arguments:

  • image_list : This parameter is a list of images loaded via OpenCV. In our case, we supply the images  list built on Lines 26-29.
  • image_shape : A tuple containing the width and height of each image in the montage. Here we indicate that all images in the montage will be resized to 129 x 196. Resizing every image in the montage to a fixed size is a requirement so we can properly allocate memory in the resulting NumPy array. Note: Empty space in the montage will be filled with black pixels.
  • montage_shape : A second tuple, this one specifying the number of columns and rows in the montage. Here we indicate that our montage will have 7 columns (7 images wide) and 3 rows (3 images tall).

The build_montages  method returns a list of montage images in NumPy array format.

If there are more images in the images  list than the montage_shape  can hold, a new montage is created for the extra images . This process is repeated until all images  have been added to a montage. This process is identical to displaying search results over multiple pages.

Our final code block handles displaying the montages  to our screen:

On Line 35 we loop over each of the montages  (again, similar to displaying N number of (faux) “search results” on a page).

Lines 36 and 37 then display the current montage  to our screen. The cv2.waitKey call pauses execution of our script until we select the currently active window and press any key on our keyboard. This will cause the for  loop to advance.

Once we reach the end of the montages  list, the script exits.

Displaying the Montage

Approximately two years ago I was involved in a computer vision project that required me to build a simple image fashion search engine. To accomplish this, I built a simple web crawler to spider Nordstrom.com and download all the product images and associated meta data.

We are are going to use a tiny sample of this data today when demoing the build_montages  function.

Once you’ve used the “Downloads” section below to download the source code + example images, you can execute the following command to see the results:

After executing the script you should see output similar to the following:

Figure 1: Building a montage with OpenCV and Python.

Note: The exact images that you see in the montage will vary from mine since we are randomly sampling from the input directory.

As we can see in Figure 1 above, we have three rows, each row containing seven images. Each image in the montage has been resized to a fixed size of 128 x 196 pixels.

In the context of Brian’s question at the top of this blog post, this montage could be search results from his image search engine algorithm.

As a second example, let’s increase the --sample  such that we create multiple montages since all images will not fit in a three row, seven column format:

Since 3 x 7 = 21, we know that sampling 33 images cannot possibly fit into a 21 image montage.

Luckily for us, the build_montages  function realizes that there are too many images to fit into a single montage and thus creates two montages.

The first montage can be seen below with all 21 spaces in the montage occupied:

Figure 2: The first montage generated with OpenCV is completely filled.

The second montage holds the remaining 12 images that could not fit in the first montage:

Figure 3: The second montage displays images that could not fit in the first montage.

Notice how empty spaces in the montage are filled with black pixels.

Summary

In today’s blog post I demonstrated how to build a montage with OpenCV and Python to visualize a collection of images. This is a handy tool you can use in your own image processing projects, such as in Brian’s image search engine project detailed at the top of this blog post.

I would also like to take a second and give a big shoutout to Kyle Hounslow who contributed the build_montages  function to the imutils package — thanks again Kyle!

In next week’s blog post I’ll demonstrate how to use this montage functionality in an actual application where we sort images in a dataset according to how “colorful” they are.

To be notified when this next blog post goes live, be sure to enter your email address in the form below.

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 11-page Resource Guide on Computer Vision and Image Search Engines, including exclusive techniques that I don’t post on this blog! Sound good? If so, enter your email address and I’ll send you the code immediately!

, ,

7 Responses to Montages with OpenCV

  1. Ricardo May 29, 2017 at 2:18 pm #

    Awesome man, thanks for your work!

    • Adrian Rosebrock May 31, 2017 at 1:19 pm #

      Glad to hear it Ricardo! 🙂

  2. Chuck Untulis May 30, 2017 at 9:23 pm #

    In the early days of the web, I implemented by hand, a montage of images into a composite image. The last image placed was on top and partially occluded the images under it. This allowed me to conserve screen real estate but still allowed mysual cortex to recognize and select the parts of the original images and then to select them. Has anyone done anything like that?

    I also thought about, but never implemented a hyperbolic tree to again reduce real estate but still add structure to images and allow easy recognition and retrieval. Has anyone done anything along those lines.

    • Adrian Rosebrock May 31, 2017 at 1:05 pm #

      Hi Chuck — I haven’t seen an implementation specific to what you are referring to.

  3. Parth Kothare June 30, 2017 at 2:03 am #

    This is great ! But what if we need to set a image in a fixed pattern like two rows and three coloumns etc.. Would really like if someone could help me in this direction !
    Thanks in advance

    • Adrian Rosebrock June 30, 2017 at 8:01 am #

      Hi Parth — can you elaborate more on what you mean by a “fixed pattern”?

Trackbacks/Pingbacks

  1. Computing image “colorfulness” with OpenCV and Python - PyImageSearch - June 5, 2017

    […] metric, we’ll sort a given dataset according to color and display the results using the image montage tool that we created last […]

Leave a Reply