EarthServer initiative is establishing Agile Analytics on Petabyte data cubes as a commodity.
Pushing the boundaries of Big Earth Data services, the intercontinental Earthserver initiative enables researchers to browse, access, and analyze massive multi-dimensional data sets from a wide range of sources. Big Earth Data at your fingertips - this is the vision of EarthServer for unleashing the potential of Big Data through a disruptive paradigm shift in technology.
The marine data service will be developed by PML and will initially make use of the European Space Agency Ocean Colour – Climate Change Initiative data which comprises of a multi-sensor 15 year time series of remote sensed ocean colour data. This dataset represents several years’ worth of effort and provides a unique opportunity to researchers. The dataset comprises of several variables including Remote Sensed Reflectance of multiple light frequencies, the derived Chlorophyll concentration and water classes. Each of the variables is also accompanied by two uncertainty variables, Root Mean Square and Bias, which will be queryable. The dataset created is in the region of 100TB which presents various challenges when users wish to interact with it. These data, made available in EarthServer-1, will form the initial offering from the marine data service. The data holding will be increased significantly up to PB volumes by adding the ESA Sentinel data. Data storage techniques developed in EarthServer-1 will be utilised to expose the ESA Sentinel ocean colour data as it becomes available. These data will represent an estimated 1TB of data per day for the ocean colour product alone and therefore will present the same issues as the OC-CCI data when users try to interact. The Sentinel data provide a good target for use within the project as they represent an ongoing commitment for production and storage that represents many Petabytes of data. The first Sentinel-3 satellite is currently scheduled for launch in mid-2015, followed by a second, so they work together to provide maximum coverage.
The data service will form part of PML’s core data offering for the wider marine research community and will remain operational after the project as part of PML’s big data strategy.