Our process for maintaining an open geospatial data resource.
At Element 84, our interest in open source technology runs deep. We’re committed to making geospatial data more readily available and usable to end users because we believe that accessibility is key to fostering innovation throughout the community. Our work supports this commitment, and Element 84 maintains a catalog of Sentinel-2 and Landsat data available at the Earth Search endpoint. Within the last few months we’ve expanded the number of open collections on Earth Search, a catalog of data available on AWS, to further extend the API’s impact.
Earth Search as a community resource
Our work to make Earth Search as accessible and beneficial to the community as possible is ongoing. A few years ago, Element 84 deployed the original Earth Search API. The API was a catalog of the Sentinel-2 Cloud-Optimized GeoTIFF data we converted from the original JP2K files that were already available on AWS. We created STAC metadata for the Sentinel-2 L1C collection, as well as Sentinel-2 L2A, then created a third collection of the Sentinel-2 L2A, but converted to Cloud-Optimized GeoTIFFs.
The original Earth Search catalog was created before the final 1.0 version of STAC was released, and also before several useful STAC extensions were published. In order to maintain Earth Search as a resource and keep it compatible with the STAC ecosystem, we regenerated the STAC metadata using the latest version of STAC, relevant extensions, and with the latest best practices in mind.
The new version (“v1”) of Earth Search with these updates is available at earth-search.aws.element84.com/v1. The old version (“v0”) of Earth Search is now considered deprecated and will soon stop receiving updated STAC Items before it is deleted entirely.
Changes from Earth Search v0 to v1
- The `sentinel-s2-l1c` collection is now named `sentinel-2-l1c`
- The `sentinel-s2-l2a` and `sentinel-s2-l2a-cogs` collections have been combined into a single collection called `sentinel-2-l2a`.
- In the `sentinel-2-l2a` Collection, for each spectral band there are two assets, the Cloud-Optimized GeoTIFF file in the AWS `us-west-2` region, and the source JP2K file in the `eu-central-1` region. The JP2K assets are the assets named with use “jp2” in the asset key (e.g., “red-jp2”, “nir09-jp2”).
- In addition to the EO, Projection, and View STAC extensions, Earth-Search v1 uses the Processing, MGRS, Grid, and Raster STAC extensions
- The STAC metadata in Earth-Search is created by using the appropriate stac tools package wrapped in a stac-task task.
- Specific per-Collection notes can all be found in the Earth Search GitHub repository, which is also used for issue tracking.
New Collections
Along with the STAC updates, we’ve added several new datasets: Landsat Collection 2, Sentinel-1 GRD, National Agriculture Imagery Program (NAIP), Copernicus Digital Elevation Model (DEM).
- A Landsat collection was added which represents Landsat “Collection 2” – a single dataset consisting of Landsats 4-8, processed, registered and atmospherically corrected to surface reflectance. All of the spectral bands, from optical through thermal, are available in the same Item. The assets are in a requester pays bucket.
- NAIP – The National Agricultural Imagery Program provides high resolution 4-band (RGB + Near Infrared) aerial imagery for CONUS. Images are captured at least once every three years for each state. This Collection includes NAIP data 2010-present, in Cloud-Optimized GeoTIFF format.
- Sentinel-1 GRD – The Sentinel Ground-Range Detected amplitude-only data is available as Cloud-Optimized GeoTIFFs. New more accurate footprints over the original are generated during ingest .
- Copernicus DEM – The Copernicus DEM data is made available in two different Collections that only vary by the resolution (30 meter vs 90 meter).
Earth Search Collections
| AWS Public Dataset | Collection | stactools Package | # of Items |
| Sentinel-2 | sentinel-2-l1c | sentinel2 | ~21.7 million |
| Sentinel-2 and Sentinel-2 COGs | sentinel-2-l2a | sentinel2 | ~21.7 million |
| USGS Landsat | landsat-c2-l2 | landsat | ~8.5 million |
| NAIP | naip | naip | ~1.2 million |
| Sentinel-1 GRD | sentinel-1 | sentinel1 | ~2.7 million |
| Copernicus DEM | cop-dem-glo-30 | cop-dem | ~26,000 |
| Copernicus DEM | cop-dem-glo-90 | cop-dem | ~26,000 |
The future of Earth Search
Moving forward, we are excited to continue growing Earth Search as a resource for the community of public data users. The STAC catalog will inevitably continue to grow, and we look forward to continuing to sponsor additional datasets while maintaining the API more broadly to align with those updates.
If you’re new to Earth Search, we’d love to get you started with the API. You can view example use cases here. In addition to our base level Earth Search API, we also offer Earth Search private which allows users to run the API in a private account and keep up-to-date with public datasets while simultaneously indexing private datasets.
To stay up to date with changes or additions to Earth Search please sign up for the Earth Search user mailing list. To report issues, visit the Earth Search GitHub repository.
Earth search is provided for free, but does not come with any guaranteed service. If you are using Earth Search in production and are interested in a private data catalog that can incorporate both public and private data sources, please drop us a line. Earth Search is powered by FilmDrop, a collection of open-source applications and libraries, which you can demo for free right now.