One of the main benefits of the bat-country Python package for deep dreaming and visualization is its ease of use, extensibility, and customization.
And let me tell you, that customization really came in handy last Friday when the Google Research team released an update to their deep dream work, demonstrating a method to “guide” your input images to visualize the features of a target image.
The results were quite impressive — so I decided to port the functionality to bat-country.
Truth be told, it only took ~20 minutes from start-to-finish to get the code together. I honestly spent more time running the Python scripts to gather example images and updating the documentation than I did updating the codebase.
The secret to this quick turnaround is the extensibility of the
BatCountry class where nearly every function and every method can be overridden and extended.
Want to change how each image is pre-processed or post-processed? No problem. Define your own custom processor and pass it in. Want to change the objective function? Again, just define your own objective and you’re good to go.
In fact, defining your own custom objective function is the the exact route I took when extending
bat-country . I simply defined a new objective function, allowing the step function to be further customized, and we’re done!
In the remainder of this blog post we’ll play around with the new bat-country update to perform guided dreaming — and even use it to generate our own art using guided deep dreaming!
Guided deep dreaming
Last Friday the Google Research team posted an update to their deep dream working demonstrating it was possible to guide your dreaming process by supplying a seed image. This method passes your input image through the network in a similar manner, but this time using your seed image to guide and influence the output.
bat-country , it’s just as easy to perform guided dreaming as deep dreaming. Here’s some quick sample code:
# import the necessary packages from batcountry import BatCountry from PIL import Image import numpy as np import argparse # construct the argument parser and parse the arguments ap = argparse.ArgumentParser() ap.add_argument("-b", "--base-model", required=True, help="base model path") ap.add_argument("-l", "--layer", type=str, default="inception_4c/output", help="layer of CNN to use") ap.add_argument("-i", "--image", required=True, help="path to base image") ap.add_argument("-g", "--guide-image", required=True, help="path to guide image") ap.add_argument("-o", "--output", required=True, help="path to output image") args = ap.parse_args() # we can't stop here... bc = BatCountry(args.base_model) features = bc.prepare_guide(Image.open(args.guide_image), end=args.layer) image = bc.dream(np.float32(Image.open(args.image)), end=args.layer, iter_n=20, objective_fn=BatCountry.guided_objective, objective_features=features) bc.cleanup() # write the output image to file result = Image.fromarray(np.uint8(image)) result.save(args.output)
If you don’t already have
bat-country installed, either pull down the code from the GitHub repo or use pip to install it on your system:
pip install bat-country or
pip install --upgrade bat-country . You’ll also need a working installation of Caffe.
What’s nice about this approach is that we can “guide” what the output image looks like. Here I use a seed image of Vincent van Gogh’s Starry Night and apply to an image of clouds:
As you can see, the output cloud image after applying guided dreaming appears to mimic many of the brush strokes of Van Gogh’s painting.
Which got me thinking — what would happen if I took some well-known paintings from extremely famous artists such as Andy Warhol, MC Escher, Pablo Picasso, Jackson Pollock, and Vincent van Gogh and used them as inputs and guides to each other?
What would the results look like? Would the artistic style of each painting be transferred to the other?
In order to test this out, I collected images of the following pieces of work:
- Andy Warhol – Gun
- Andy Warhol – Marilyn Monroe
- Jackson Pollock – Energy Made Visibile
- MC Escher – Sky and Water I
- MC Escher – Mosaic II
- Pablo Picasso – Guernica
- Vincent van Gogh – Wheat Field with Cypresses
And then I passed each of them through the
demo_guided.py script detailed above.
Generating art with deep dreaming
Overall, the results look really fantastic. I’m especially pleased with how Wheat Field with Cypresses and Guernica turned out when used as input images and the others paintings as guided images.
Below you can find some of my favorite images:
Vincent van Gogh – Wheat Field with Cypresses
Pablo Picasso – Guernica
MC Escher – Mosaic II
Andy Warhol – Marilyn Monroe
In this blog post I reviewed the updates to the bat-country package will allow for guided deep dreaming by supplying two images: an input image that will be passed through the network, and a seed image that the network will use to “guide” the output.
I then took the updated code and used it to generate art from famous works, such as Jackson Pollock’s Energy Made Visible, Pablo Picasso’s Guernica, and Vincent van Gogh’s Wheat Field with Cypresses.
Definitely consider installing the
bat-country package on your system and giving deep dreaming a try! It’s strangely addictive (and not to mention, a lot of fun) to play around and generate your own images.
Finally, if you’re interested in deep learning, deep dreaming, and computer vision, think about signing up for the PyImageSearch Newsletter by entering your email address in the form below. I send out regular updates on the PyImageSearch blog, each filled with actionable, real-world computer vision projects.
See you next time!