Detecting Natural Disasters with Keras and Deep Learning

In this tutorial, you will learn how to automatically detect natural disasters (earthquakes, floods, wildfires, cyclones/hurricanes) with up to 95% accuracy using Keras, Computer Vision, and Deep Learning.

I remember the first time I ever experienced a natural disaster — I was just a kid in kindergarten, no more than 6-7 years old.

We were outside for recess, playing on the jungle gym, running around like the wild animals that young children are.

Rain was in the forecast. It was cloudy. And very humid.

My mother had given me a coat to wear outside, but I was hot and uncomfortable — the humidity made the cotton/polyester blend stick to my skin. The coat, just like the air around me, was suffocating.

All of a sudden the sky changed from “normal rain clouds” to an ominous green.

The recess monitor reached into her pocket, grabbed her whistle, and blew it, indicating it was time for us to settle our wild animal antics and come inside for schooling.

After recess we would typically sit in a circle around the teacher’s desk for show-and-tell.

But not this time.

We were immediately rushed into the hallway and were told to cover our heads with our hands — a tornado had just touched down near our school.

Just the thought of a tornado is enough to scare a kid.

But to actually experience one?

That’s something else entirely.

The wind picked up dramatically, an angry tempest howling and berating our school with tree branches, rocks, and whatever loose debris was not tied down.

The entire ordeal couldn’t have lasted more than 5-10 minutes — but it felt like a terrifying eternity.

It turned out that we were safe the entire time. After the tornado had touched down it started carving a path through the cornfields away from our school, not toward it.

We were lucky.

It’s interesting how experiences as a young kid, especially the ones that scare you, shape you and mold you after you grow up.

A few days after the event my mom took me to the local library. I picked out every book on tornados and hurricanes that I could find. Even though I only had a basic reading level at the time, I devoured them, studying the pictures intently until I could recreate them in my mind — imagining what it would be like to be inside one of those storms.

Later, in graduate school, I experienced the historic June 29th, 2012 derecho that delivered 60+ MPH sustained winds and gusts of over 100 MPH, knocking down power lines and toppling large trees.

That storm killed 29 people, injured hundreds of others, and caused loss of electricity and power in parts of the United States east coast for over 6 days, an unprecedented amount of time in the modern-day United States.

Natural disasters cannot be prevented — but they can be detected, giving people precious time to get to safety.

In this tutorial, you’ll learn how we can use Computer Vision and Deep Learning to help detect natural disasters.

To learn how to detect natural disasters with Keras, Computer Vision, and Deep Learning, just keep reading!

Looking for the source code to this post?
Jump right to the downloads section.

Detecting Natural Disasters with Keras and Deep Learning

In the first part of this tutorial, we’ll discuss how computer vision and deep learning algorithms can be used to automatically detect natural disasters in images and video streams.

From there we’ll review our natural disaster dataset which consists of four classes:

  • Cyclone/hurricane
  • Earthquake
  • Flood
  • Wildfire

We’ll then design a set of experiments that will:

  • Help us fine-tune VGG16 (pre-trained on ImageNet) on our dataset.
  • Find optimal learning rates.
  • Train our model and obtain > 95% accuracy!

Let’s get started!

How can computer vision and deep learning detect natural disasters?

Figure 1: We can detect natural disasters with Keras and Deep Learning using a dataset of natural disaster images. (image source)

Natural disasters cannot be prevented — but they can be detected.

All around the world we use sensors to monitor for natural disasters:

  • Seismic sensors (seismometers) and vibration sensors (seismoscopes) are used to monitor for earthquakes (and downstream tsunamis).
  • Radar maps are used to detect the signature “hook echo” of a tornado (i.e., a hook that extends from the radar echo).
  • Flood sensors are used to measure moisture levels while water level sensors monitor the height of water along a river, stream, etc.
  • Wildfire sensors are still in their infancy but hopefully will be able to detect trace amounts of smoke and fire.

Each of these sensors is highly specialized to the task at hand — detect a natural disaster early, alert people, and allow them to get to safety.

Using computer vision we can augment existing sensors, thereby increasing the accuracy of natural disaster detectors, and most importantly, allow people to take precautions, stay safe, and prevent/reduce the number of deaths and injuries that happen due to these disasters.

Our natural disasters image dataset

Figure 2: A dataset of natural disaster images. We’ll use this dataset to train a natural disaster detector with Keras and Deep Learning.

The dataset we are using here today was curated by PyImageSearch reader, Gautam Kumar.

Gautam used Google Images to gather a total of 4,428 images belonging to four separate classes:

  • Cyclone/Hurricane: 928 images
  • Earthquake: 1,350
  • Flood: 1,073
  • Wildfire: 1,077

He then trained a Convolutional Neural Network to recognize each of the natural disaster cases.

Gautam shared his work on his LinkedIn profile, gathering the attention of many deep learning practitioners (myself included). I asked him if he would be willing to (1) share his dataset with the PyImageSearch community and (2) allow me to write a tutorial using the dataset. Gautam agreed, and here we are today!

I again want to give a big, heartfelt thank you to Gautam for his hard work and contribution — be sure to thank him if you have the chance!

Downloading the natural disasters dataset

Figure 3: Gautam Kumar’s dataset for detecting natural disasters with Keras and deep learning.

You can use this link to download the original natural disasters dataset via Google Drive.

After you download the archive you should unzip it and inspect the contents:

Here you can see that each of the natural disasters has its own directory with examples of each class residing inside its respective parent directory.

Project structure

Using the tree  command, let’s review today’s project available via the “Downloads” section of this tutorial:

Our project contains:

  • The natural disaster dataset. Refer to the previous two sections.
  • An output/  directory where our model and plots will be stored. The results from my experiment are included.
  • Our pyimagesearch  module containing our Cyclical Learning Rate Keras callback, a configuration file, and Keras Learning Rate Finder.
  • A selection of videos/  for testing the video classification prediction script.
  • Our training script, train.py . This script will perform fine-tuning on a VGG16 model pre-trained on the ImageNet dataset.
  • Our video classification prediction script, predict.py , which performs a rolling average prediction to classify the video in real-time.

Our configuration file

Our project is going to span multiple Python files, so to keep our code tidy and organized (and ensure that we don’t have a multitude of command line arguments), let’s instead create a configuration file to store all important paths and variables.

Open up the config.py file inside the pyimagesearch module and insert the following code:

The os  module import allows us to build OS-agnostic paths directly in this config file (Line 2).

Line 6 specifies the root path to our natural disaster dataset.

Line 7 provides the names of class labels (i.e. the names of the subdirectories in the dataset).

Let’s define our dataset splits:

Lines 13-15 house our training, testing, and validation split sizes. Take note that the validation split is 10% of the training split (not 10% of all the data).

Next, we’ll define our training parameters:

Lines 19 and 20 contain the minimum and maximum learning rate for Cyclical Learning Rates (CLR).We’ll learn how to set these learning rate values in the “Finding our initial learning rate” section below.

Lines 21-24 define the batch size, step size, CLR method, and the number of training epochs.

From there we’ll define the output paths:

Lines 27-33 define the following output paths:

  • Serialized model after training
  • Learning rate finder plot
  • Training history plot
  • CLR plot

Implementing our training script with Keras

Our training procedure will consist of two steps:

  1. Step #1: Use our learning rate finder to find optimal learning rates to fine-tune our VGG16 CNN on our dataset.
  2. Step #2: Use our optimal learning rates in conjunction with Cyclical Learning Rates (CLR) to obtain a high accuracy model.

Our train.py file will handle both of these steps.

Go ahead and open up train.py in your favorite code editor and insert the following code:

Lines 2-27 import necessary packages including:

  • matplotlib : For plotting (using the "Agg"  backend so plot images can be saved to disk).
  • tensorflow : Imports including our VGG16  CNN, data augmentation, layer types, and SGD  optimizer.
  • scikit-learn : Imports including a label binarizer, dataset splitting function, and an evaluation reporting tool.
  • LearningRateFinder : Our Keras Learning Rate Finder class.
  • CyclicLR : A Keras callback that oscillates learning rates, known as Cyclical Learning Rates. CLRs lead to faster convergence and typically require fewer experiments for hyperparameter updates.
  • config : The custom configuration settings we reviewed in the previous section.
  • paths : Includes a function for listing the image paths in a directory tree.
  • cv2 : OpenCV for preprocessing and display.

Let’s parse command line arguments and grab our image paths:

Recall that most of our settings are in config.py . There is one exception. The --lr-find  command line argument tells our script whether or not to find the optimal learning rate (Lines 30-33).

Line 38 grabs paths to all images in our dataset.

We then initialize two synchronized lists to hold our image  data  and labels  (Lines 39 and 40).

Let’s populate the data  and labels  lists now:

Lines 43-55 loop over imagePaths, while:

  • Extracting the class label  from the path (Line 45).
  • Loading and preprocessing the image  (Lines 49-51). Images are converted to RGB channel ordering and resized to 224×224 for VGG16.
  • Adding the preprocessed image  to the data  list (Line 54).
  • Adding the label  to the labels  list (Lines 55).

Line 59 performs a final preprocessing step by converting the data  to a "float32"  datatype NumPy array.

Similarly, Line 60 converts labels  to an array so that Lines 63 and 64 can perform one-hot encoding.

From here, we’ll partition our data and set up data augmentation:

Lines 67-72 construct training, testing, and validation splits.

Lines 75-82 instantiate our data augmentation object. Read more about data augmentation in my previous posts as well as in the Practitioner Bundle of Deep Learning for Computer Vision with Python.

At this point we’ll set up our VGG16 model for fine-tuning:

Lines 86 and 87 load VGG16  using pre-trained ImageNet weights (but without the fully-connected layer head).

Lines 91-95 create a new fully-connected layer head followed by Line 99 which adds the new FC layer to the body of VGG16.

Lines 103 and 104 mark the body of VGG16 as not trainable — we will be training (i.e. fine-tuning) only the FC layer head.

Lines 109-111 then compile  our model with the Stochastic Gradient Descent ( SGD ) optimizer and our specified minimum learning rate.

The first time you run the script, you should set the --lr-find  command line argument to use the Keras Learning Rate Finder to determine the optimal learning rate. Let’s see how that works:

Line 115 checks to see if we should attempt to find optimal learning rates. Assuming so, we:

  • Initialize LearningRateFinder  (Line 119).
  • Start training with a 1e-10  learning rate and exponentially increase it until we hit 1e+1  (Lines 120-125).
  • Plot the loss vs. learning rate and save the resulting figure (Lines 129 and 130).
  • Gracefully exit  the script after printing a message instructing the user to inspect the learning rate finder plot (Lines 135-137).

After this code executes we now need to:

  1. Step #1: Review the generated plot.
  2. Step #2: Update config.py with our MIN_LR  and MAX_LR, respectively.
  3. Step #3: Train the network on our full dataset.

Assuming we have completed Steps #1 and #2, let’s now handle Step #3 where our minimum and maximum learning rate have already been found and updated in the config.

In this case, it is time to initialize our Cyclical Learning Rate class and commence training:

Lines 142-147 initialize our CyclicLR .

Lines 151-157 then train our model  using .fit_generator with our aug  data augmentation object and our clr  callback.

Upon training completion, we proceed to evaluate and save our model :

Line 161 makes predictions on our test set. Those predictions are passed into Lines 162 and 163 which print a classification report summary.

Line 167 serializes and saves the fine-tuned model to disk.

Finally, let’s plot both our training history and CLR history:

Lines 170-181 generate a plot of our training history and save the plot to disk.

Note: In TensorFlow 2.0, the history dictionary keys have changed from acc  to accuracy  and val_acc  to val_accuracy . It is especially confusing since “accuracy” is spelled out now, but “validation” is not. Take special care with this nuance depending on your TensorFlow version.

Lines 184-190 plot our Cyclical Learning Rate history and save the figure to disk.

Finding our initial learning rate

Before we attempt to fine-tune our model to recognize natural disasters, let’s first use our learning rate finder to find an optimal set of learning rate ranges. Using this optimal learning rate range we’ll then be able to apply Cyclical Learning Rates to improve our model accuracy.

Make sure you have both:

  1. Used the “Downloads” section of this tutorial to download the source code.
  2. Downloaded the dataset using the “Downloading the natural disasters dataset” section above.

From there, open up a terminal and execute the following command:

Provided the train.py script exited without error, you should now have a file named lrfind_plot.png in your output directory.

Take a second now to inspect this image:

Figure 4: Using a Keras Learning Rate Finder to find the optimal learning rates to fine tune our CNN on our natural disaster dataset. We will use the dataset to train a model for detecting natural disasters with the Keras deep learning framework.

Examining the plot you can see that our model initially starts to learn and gain traction around 1e-6 .

Our loss continues to drop until approximately 1e-4  where it starts to rise again, a sure sign of overfitting.

Our optimal learning rate range is, therefore, 1e-6  to 1e-4 .

Update our learning rates

Now that we know our optimal learning rates, let’s go back to our config.py file and update them accordingly:

Notice on Lines 19 and 20 (highlighted) of our configuration file that the MIN_LR  and MAX_LR  learning rate values are freshly updated. These values were found by inspecting our Keras Learning Rate Finder plot in the section above.

Training the natural disaster detection model with Keras

We can now fine-tune our model to recognize natural disasters!

Execute the following command which will train our network over the full set of epochs:

Here you can see that we are obtaining 95% accuracy when recognizing natural disasters in the testing set!

Examining our training plot we can see that our validation loss follows our training loss, implying there is little overfitting within our dataset itself:

Figure 5: Training history accuracy/loss curves for creating a natural disaster classifier using Keras and deep learning.

Finally, we have our learning rate plot which shows our our CLR callback oscillates the learning rate between our MIN_LR and MAX_LR, respectively:

Figure 6: Cyclical learning rates are used with Keras and deep learning for detecting natural disasters.

Implementing our natural disaster prediction script

Now that our model has been trained, let’s see how we can use it to make predictions on images/video it has never seen before — and thereby pave the way for an automatic natural disaster detection system.

To create this script we’ll take advantage of the temporal nature of videos, specifically the assumption that subsequent frames in a video will have similar semantic contents.

By performing rolling prediction accuracy we’ll be able to “smooth out” the predictions and avoid “prediction flickering”.

I have already covered this near-identical script in-depth in my Video Classification with Keras and Deep Learning article. Be sure to refer to that article for the full background and more-detailed code explanations.

To accomplish natural disaster video classification let’s inspect predict.py:

Lines 2-7 load necessary packages and modules. In particular, we’ll be using deque  from Python’s collections  module to assist with our rolling average algorithm.

Lines 10-19 parse command line arguments including the path to our input/output videos, size of our rolling average queue, and whether we will display the output frame to our screen while the video is being generated.

Let’s go ahead and load our natural disaster classification model and initialize our queue + video stream:

With our model , Q , and vs  ready to go, we’ll begin looping over frames:

Lines 38-47 grab a frame  and store its dimensions.

Lines 51-54 duplicate our frame  for output  purposes and then preprocess it for classification. The preprocessing steps are, and must be, the same as those that we performed for training.

Now let’s make a natural disaster prediction on the frame:

Lines 58 and 59 perform inference and add the predictions to our queue.

Line 63 performs a rolling average prediction of the predictions available in the Q .

Lines 64 and 65 then extract the highest probability class label so that we can annotate our frame:

Lines 68-70 annotate the natural disaster activity in the corner of the output  frame.

Lines 73-80 handle writing the output  frame to a video file.

If the --display  flag is set, Lines 84-91 display the frame to the screen and capture keypresses.

Otherwise, processing continues until completion at which point the loop is finished and we perform cleanup (Lines 95 and 96).

Predicting natural disasters with Keras

For the purposes of this tutorial, I downloaded example natural disaster videos via YouTube — the exact videos are listed in the “Credits” section below. You can either use your own example videos or download the videos via the credits list.

Either way, make sure you have used the “Downloads” section of this tutorial to download the source code and pre-trained natural disaster prediction model.

Once downloaded you can use the following command to launch the predict.py script:

Here you can see a sample result of our model correctly classifying this video clip as “flood”:

Figure 7: Natural disaster “flood” classification with Keras and Deep Learning.

The following example comes from the 2016 Fort McMurray wildfire:

Figure 8: Detecting “wildfires” and other natural disasters with Keras, deep learning, and computer vision.

For fun, I then tried applying the natural disaster detector to the movie San Andreas (2015):

Figure 9: Detecting “earthquake” damage with Keras, deep learning, and Python.

Notice how our model was able to correctly label the video clip as an (overly dramatized) earthquake.

You can find a full demo video below:

Where to next?

Figure 10: My deep learning book is the go-to resource for deep learning students, developers, researchers, and hobbyists, alike. Use the book to build your skillset from the bottom up, or read it to gain a deeper understanding. Don’t be left in the dust as the fast paced AI revolution continues to accelerate.

Today’s tutorial helped us solve a real-world classification problem for classifying natural disaster videos.

Such an application could be:

  • Deployed along riverbeds and streams to monitor water levels and detect floods early.
  • Utilized by park rangers to monitor for wildfires.
  • Employed by meteorologists to automatically detect hurricanes/cyclones.
  • Used by television news companies to sort their archives of video footage.

We created our natural disaster detector by utilizing a number of important deep learning techniques:

  • Fine-tuning a Convolutional Neural Network that was trained on ImageNet
  • Using a Keras Learning Rate Finder
  • Implementing the Cyclical Learning Rate callback into our training process to improve model accuracy
  • Performing video classification with a rolling frame classification average approach

Admittedly, these are advanced concepts in the realm of deep learning and computer vision. If you have your own real-world project you’re trying to solve, you need a strong deep learning foundation in addition to familiarity with advanced concepts.

To jumpstart your education, including discovering my tips, suggestions, and best practices when training deep neural networks, be sure to refer to my book, Deep Learning for Computer Vision with Python.

Inside the book I cover:

  1. Deep learning fundamentals and theory without unnecessary mathematical fluff. I present the basic equations and back them up with code walkthroughs that you can implement and easily understand. You don’t need a degree in advanced mathematics to understand this book.
  2. More details on learning rates, tuning them, and how a solid understanding of the concept dramatically impacts the accuracy of your model.
  3. How to spot underfitting and overfitting on-the-fly, saving you days of training time.
  4. How to perform fine-tuning on pre-trained models, which is often the place I start to obtain a baseline result to beat.
  5. My tips/tricks, suggestions, and best practices for training CNNs.

To learn more about the book, and grab the table of contents + free sample chapters, just click here!

Credits

Dataset curator:

Video sources for the demo:

Audio for the demo video:

Summary

In this tutorial, you learned how to use computer vision and the Keras deep learning library to automatically detect natural disasters from images.

To create our natural disaster detector we fine-tuned VGG16 (pre-trained on ImageNet) on a dataset of 4,428 images belonging to four classes:

  • Cyclone/hurricane
  • Earthquake
  • Flood
  • Wildfire

After our model was trained we evaluated it on the testing set, finding that it obtained 95% classification accuracy.

Using this model you can continue to perform research in natural disaster detection, ultimately helping save lives and reduce injury.

I hope you enjoyed this post!

To download the source code to the post (and be notified when future tutorials are published on PyImageSearch), just enter your email address in the form below.

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL! Sound good? If so, enter your email address and I’ll send you the code immediately!

, , , ,

18 Responses to Detecting Natural Disasters with Keras and Deep Learning

  1. Gary November 11, 2019 at 3:41 pm #

    Very nice work!
    My understanding is here you classify every scene as one of the four natural disaster classes. However, I think it’d be very interesting to include another class, i.e. none of these classes happens using some negative examples. This way, we could use it to detect whether a natural disaster really happens.

    • Adrian Rosebrock November 11, 2019 at 4:06 pm #

      You’re absolutely right. Next week I’ll be discussing that idea through an example. Stay tuned!

      • hamze November 12, 2019 at 8:13 am #

        Thanks,
        I am also interested in Gary’s question. What if the image is none of them? a non-disaster image?

        • Adrian Rosebrock November 14, 2019 at 9:03 am #

          Hey Hamze — did you see my response? I’ll be discussing that in next week’s tutorial. Basically you create an additional “ignore” class that consists of non-disaster images.

  2. Mohanad November 11, 2019 at 4:30 pm #

    You are AMAZING Adrian, and I wish to catch up with your blogs soon. Thank you very much

    • Adrian Rosebrock November 11, 2019 at 4:48 pm #

      Thanks Mohanad!

  3. sujan November 11, 2019 at 7:42 pm #

    Just came to show my gratitude.
    #Respect

    • Adrian Rosebrock November 12, 2019 at 7:50 am #

      Thanks Sujan!

  4. Itai Druker November 12, 2019 at 2:20 am #

    Great work Adrian, really loved the idea of learning rate finder and the cyclic LR

    • Adrian Rosebrock November 12, 2019 at 7:49 am #

      Thanks Itai!

  5. elon November 12, 2019 at 8:28 am #

    Nice work !

    • Adrian Rosebrock November 14, 2019 at 9:02 am #

      Thanks!

  6. Yann Yeung November 14, 2019 at 6:57 am #

    nice work, Adrian, i am your big fan! but i still have a little suggestion is that you can set your data by a fixed known length first in line 39 instead of using “append” because when i use “append” it went to be very slow. set its length first will be better

    • Adrian Rosebrock November 14, 2019 at 9:02 am #

      Hey Yann — do you mean allocating a NumPy array with the fixed dimensions and then updating it? If so, yes, that would work.

  7. khan November 17, 2019 at 3:20 pm #

    Thank you adrian for this amazing article, please tell is how to load very large dataset, when I try your code on large datasets your code is unable to load large datasets

  8. ahmed mohammed November 19, 2019 at 1:53 am #

    This is beautiful. Thank you

    • Adrian Rosebrock November 21, 2019 at 8:55 am #

      Thanks Ahmed!

Before you leave a comment...

Hey, Adrian here, author of the PyImageSearch blog. I'd love to hear from you, but before you submit a comment, please follow these guidelines:

  1. If you have a question, read the comments first. You should also search this page (i.e., ctrl + f) for keywords related to your question. It's likely that I have already addressed your question in the comments.
  2. If you are copying and pasting code/terminal output, please don't. Reviewing another programmers’ code is a very time consuming and tedious task, and due to the volume of emails and contact requests I receive, I simply cannot do it.
  3. Be respectful of the space. I put a lot of my own personal time into creating these free weekly tutorials. On average, each tutorial takes me 15-20 hours to put together. I love offering these guides to you and I take pride in the content I create. Therefore, I will not approve comments that include large code blocks/terminal output as it destroys the formatting of the page. Kindly be respectful of this space.
  4. Be patient. I receive 200+ comments and emails per day. Due to spam, and my desire to personally answer as many questions as I can, I hand moderate all new comments (typically once per week). I try to answer as many questions as I can, but I'm only one person. Please don't be offended if I cannot get to your question
  5. Do you need priority support? Consider purchasing one of my books and courses. I place customer questions and emails in a separate, special priority queue and answer them first. If you are a customer of mine you will receive a guaranteed response from me. If there's any time left over, I focus on the community at large and attempt to answer as many of those questions as I possibly can.

Thank you for keeping these guidelines in mind before submitting your comment.

Leave a Reply

[email]
[email]