Clouds are a problem for all optical imagery.  As you can see from the image below, there are two main problems. The first is that clouds themselves obscure the land surface, but at least they are distinctive and should be easy to find and remove.  The second is the cloud shadow. The shadow is more problematic because you can still see the ground surface below, but the solar illumination is different to the non-shadow areas, so we really want to get rid of them too.  In this article, we will explore three different methods to get rid of the clouds and produce a continuous cloud-free image.

TABLE OF CONTENTS

• The basic principles of cloud removal 
  • Principle 1: Optical sensors can't see through clouds.
  • Principle 2: We can remove all images (from our analysis) that have too many clouds
  • Principle 3: There are trade-offs
• Method 1: A simple cloud-free image
  • Step 1: Set up the blocks
  • Step 2: Choose the composite method to maximise cloud-free pixels
• Method 2: Using Landsat 
  • Step 1: Simple cloud score + mask
  • Step 2: Interpret the output
• Method 3: Using Sentinel 2 
  • Step 1: QA + S2 cloudscore algorithm.
  • Step 2: S2 Cloudless
• A final note on NDVI 
  • Remember: There isn't a cloud filter which is perfect for all time periods and all climatic regions. Experiment to find out which one works best for your site, and your application. 

The basic principles of cloud removal 

Principle 1: Optical sensors can't see through clouds.

• It sounds obvious, but it is important to recognise that there is no way to remove clouds and see the ground below.  
• All we can do is look at multiple images, collect as many cloud-free pixels as possible, and use them to make the final image. 
• This means that "cloud removal" is really about removing those pixels that have the influence of cloud in them and keeping those that don't. That's why we call it "cloud filtering". 
• While big bright fluffy clouds are relatively easy to detect in an image, the thinner wispy clouds are much harder to filter out. 

Principle 2: We can remove all images (from our analysis) that have too many clouds

• The image output you see in Earth Blox is a result of stitching together (mosaicking) lots of different scenes.
• The image suppliers tag each scene to let you know how much of the image is covered in cloud.
• This means we can remove any scenes that have too much cloud cover (since there is no point doing time-consuming processing to filter out clouds on a scene that will only give you a few cloud-free pixels).
• This is the metric that the CLOUD FILTER block will ask you for. The Cloud cover percentage (or the Cloud score) is asking you, "What % of cloud cover in each scene are you willing to consider?"  If you say "20" then only scenes with less than 20% cover will be used in the analysis.  If you say "100" then every scene will be used.
• Depending upon the season, the geographic location and the time period you choose, it is possible that you will come across a situation where there are no scenes available that meet your % cloud criteria.  If this is the case, you need to increase the % cloud cover metric or extend the time period (to include more scenes and increase the likelihood of having a low-cloud image).

Principle 3: There are trade-offs

• You will have to make some compromises when using optical data.
• (If you really can't make these compromises, then you might want to look at how to use the radar data, as they have no clouds.)
• For most parts of the world, if you wait long enough, you will find a cloud-free pixel for every part of your area of interest. 
• But the longer the time period, the more change can be happening on the ground surface. 
• This means you have to think carefully about what you are looking for in the imagery. 
• If you are studying agriculture, then using 12 months of data to create your cloud-free image may result in crop-related data for many different stages in the growing cycle.  
• If you are looking at tropical deforestation, then a 12-month summary of forest change may be sufficient.
• This is when your choice of Median, Mean, Max, Min or Mosaic, becomes an important consideration.
  • Median: Returns the median value for all of the cloud-free pixels collected.
  • Mean: Returns the mean value for all of the cloud-free pixels collected.
  • Max (Min): Returns the maximum (minimum) value for all of the cloud-free pixels collected.
  • Mosaic: Returns the latest pixel value all of the cloud-free pixels collected.
• Note that Mean returns a composite value, whereas the others are all a single value from a single scene. (The median is a single value from a single scene if there are an odd number of scenes. If there is an even number of scenes, then it is the average of two values.)

Method 1: A simple cloud-free image

Step 1: Set up the blocks

• This will work for a relatively cloud-free area.
• It will work for both Landsat and Sentinel 2.
• Use only the Satellite block and the Bands to visualise block.  You don't need a Cloud filtering block.
• Choose a time period that is many months.

Step 2: Choose the composite method to maximise cloud-free pixels

• Choose Median for the Composite method. 
• Since the median is the middle value, you will get a cloud-free value as long as more than 50% of the scenes didn't have cloud in that pixel.
• This only works for Median because:
  • If you choose Max, you will selectively pick out the (high) values that are cloud.
  • If you choose Min, you will selectively pick out the (low) values that are cloud-shadow.
  • If you choose Mosaic, you will get the last value, whether it was cloudy or not.
• For areas that have frequent cloud cover, this will not work. 
• The example result below is for Barbados. Note how this tropical island has had enough cloud cover inland during the year that there are still clouds in the image.

Method 2: Using Landsat 

Step 1: Simple cloud score + mask

• This is the only cloud filter currently available for Landsat data.
• It removes all scenes with a percentage of cloud cover greater than the Cloud score and then uses the Landsat cloud mask to remove pixels with clouds.
• The Landsat cloud mask is at the pixel level but works best when there are fewer clouds. 
• Here is Barbados again, but this time removing any scenes that have >40% cloud cover. 

Step 2: Interpret the output

• Note that pixels that are never cloud free will not be assigned a value. 
• These pixels will not appear on the map so you will see the background through these pixels. For example:

• These pixels are white because that is the ROADMAP in the background. As a general rule it is better to have no data than bad data.
• For your particular area, experiment with timescales and Cloud score values to see which is best for your application. 
• Remember: there isn't a cloud filter which is perfect for all time periods and all locations. 

Method 3: Using Sentinel 2 

There are two algorithms available for Sentinel 2.

Step 1: QA + S2 cloudscore algorithm.

• This is like the Landsat method above but uses a Sentinel 2 cloud and shadow mask.
• It removes all scenes with a percentage of cloud cover greater than the Max cloud cover and then uses the S2 cloud mask to remove pixels with clouds.
• The S2 cloud mask is at the pixel level, based on spectral properties, but works best when there are fewer clouds.  

Step 2: S2 Cloudless

• As with the other methods, this first removes all scenes with a percentage of cloud cover greater than the Cloud score value.
• It then uses the S2cloudless cloud mask to remove pixels with clouds.
• S2cloudless is a machine learning algorithm used to find clouds and create an image mask.
• Earth Blox uses this information to mask out the clouds.
• It then calculates where the shadows will be, and masks them too.
• Because of the extra calculations involved, this method may take longer than the default.  

A final note on NDVI 

You might be wondering what happens if you use the NDVI index block without a cloud filter block.   The good news is that clouds have a very low NDVI, so if you are looking at vegetation trends (which have a very high NDVI) simply ensure you select Maximum or 9th Percentile on the NDVI block.

This will work fine in areas of low cloud cover, or long time periods.  But it is always worthwhile adding a cloud removal block first, just in case. 

Remember: There isn't a cloud filter which is perfect for all time periods and all climatic regions. Experiment to find out which one works best for your site, and your application. 

We always track the feedback you give us so that we can improve how we help you. 

Please let us know if you found this one helpful.