This article is an overview of how more powerful and efficient space data processing is enabling companies and organizations to improve environmental applications and develop more sustainable services.
It discusses the uses of big data collected in space, particularly for sustainability-related applications, and explores how Cloudflight, a participant of the satsearch membership program, helps organizations to maximize the value and efficiency of such solutions through advanced data processing.
The article was developed in collaboration with Cloudflight.
Using big data from space
Although the demand for big data from space is growing rapidly across different industries, only a limited number of businesses have the capability to efficiently process it in order to ease this burden and create more profitable and sustainable services.
Substantial technical expertise and domain knowledge, combined with well-established tools and processes, are required to process and extract actionable insights from space data effectively. This challenge is particularly acute at scale.
For example, if the user would like to analyze historical data, there are challenges in the varying temporal and spatial resolutions recorded, and significant variability in coverage over time due to rapid technological advancements. This means that all sources of historical data cannot be treated in the same way and delivery platforms need to be adept at combining disparate streams.
Another technical challenge is to process different types of data from varying payloads and satellite sources, such as applications employing both optical and radar sensors. In particular, processing and interpreting radar data can require a high level of technical expertise due to the complexities involved in parsing accurate signals.
Although higher resolution Earth Observation (EO) satellite data are more readily available in today’s marketplace, they can be expensive to utilize when large areas of land or sea need to be analyzed, requiring higher processing capacity. In addition, the supporting infrastructure for processing operations, such as secure data management systems, storage, encryption, and back-up facilities, add additional complexity, risks, and, ultimately, costs, to any service.
Processing of big data from space can also help in reducing the burden on ground-level monitoring infrastructure (for example, high-quality and readily available maritime tracking data could reduce reliance on coastal radar). It can also be used to improve theoretical modeling and data assimilation processes in order to develop more accurate assessment and prediction models.
These developments mean that there is an increasing need for easy-to-use integrated platforms that allow users with limited technical capabilities to utilize high volume data from space efficiently, and to derive actionable insights with minimal support. Such platforms can also enable the combination of large EO datasets with other data, such as economic indicators, agricultural statistics, health information, or other available socioeconomic records.
In such processes, data providers play an important role in maximizing the benefits of information collected in space. Some of the key features and functions that data providers and processors add to the value chain are:
- User-friendly integrated platforms,
- Fast delivery of the required data,
- Frequency and recency of the available data in requested locations,
- High-resolution images at an affordable cost,
- Accessibility to different types of data (optical, SAR etc.),
- Established processing procedures for different types of data,
- Handling the varied spatial and temporal resolutions,
- Technical support,
- Management of shared and fused data from different satellite sources,
- The extraction of actionable insights,
- Simple and understandable visualization of such insights,
- Ready-to-use products and services, designed for specific applications, integrating different complementary data sources, and
- Ready-to-use state-of-the-art data processing approaches.
Next let’s take a closer look at how such data collection and management solutions are being employed in environmental projects.
Space data in sustainability applications
Satellite-based EO data, which is also often generically referred to as space data, has proven to provide an array of benefits to humanity. From weather pattern predictions to emergency management, space data is helping communities, organizations, companies, and governments to improve decision-making processes and implement better actionable measures.
Traditionally, passive satellite imagery has been popular since the inception of EO satellites. But with rapid technology developments in both upstream and downstream markets, active satellite imagery collection has become more popular, particularly due to the growing demand in sustainability applications.
The Global Climate Observing System (GCOS) – a consortium of organizations from around the world – has identified 54 Essential Climate Variables (ECVs) that relate to the health of the environment. The ECVs consist of individual variables, or linked groups of variables, and cover the land, sea, and air.
Many of these variables can be tracked, at least in part, by space-based assets, and such data streams are playing increasingly important roles in climate change modeling and mitigation strategies. This is also adding to the data processing burden that satellite operators are facing.
In such applications resolution is one of the most important metrics and it is commonly categorized into radiometric, spatial, spectral, and temporal. Spatial resolution is primarily used for assessing the tracking and monitoring of objects, while spectral resolution is important in environmental surveillance applications such as measuring temperatures or gathering certain forms of agricultural information.
Similarly, temporal resolution refers to the quality of observational insights of the same area captured at different time intervals, while radiometric resolution mainly refers to differentiating scales of a color – e.g. comparing pixels in grey-scale images.
All four types of resolution are important in a wide range of sustainability applications such as electric grid monitoring, climate change, asset monitoring, pollution mapping, land management, and measuring sea surface temperatures.
In particular, spatial resolution relates directly to image value in the market. For example, EO imagery with 15 centimeter resolution will be more valuable than 10 meter imagery for asset monitoring. Low resolution imagery, e.g. between 10 and 30 meters, can be used for more local applications such as land management or providing community insights on conservation of the environment.
High resolution imagery is in demand in both commercial and government markets, and as more organizations and nations are aligning with the United Nations (UN) Sustainable Development Goals (SDGs), it is increasingly being used in the green energy market and related sustainability applications.
The growing availability of high-resolution data is also driving the development of entirely new products and tools that can monitor, track, and provide appropriate insights in environmental applications. These solutions are being further enhanced through the use of advanced technologies such as artificial intelligence (AI) to create powerful, next generation systems.
All of this growth in demand for EO data, particularly for sustainability-related applications, is leading to a corresponding increase in demand for higher volume processing capabilities.
The demand for more efficient space data processing
We have previously written about the growing data burden on-board satellites, in particular highlighting four drivers that are affecting operations;
- Limitations due to on-board storage capacity,
- Computing requirements driven by more powerful and complex applications,
- The need for better in-orbit decision-making, and
- The growing downstream data demand.
The fourth point is, arguably, the most important factor when considering the data processing burden across the entire satellite value chain.
Satellite and payload manufacturers have been successfully designing and building a variety of more advanced payloads every year and proving their functions and profitability in space.
For example, multi-functional and agile satellites are becoming more common in the industry, such as versatile systems capable of re-positioning an EO camera to take multiple images in a single pass. This results in the collection of more data, more often, and with higher volumes of usable imagery, as opposed to inaccurate or obscured pictures.
In addition, on-board computers (OBCs) and satellite on-board data processors are becoming more sophisticated. Modern satellite OBCs are now able to manage, process, and store more data than older models, and the latest on-board data processing (OBDP) systems are capable of getting rid of useless or corrupted files far more easily, so there is a higher volume of quality data to downlink and process. On-board processors are also able to compress that data so a greater volume can be downlinked in each package.
To find out more about the wider opportunities that digital technologies are bringing to the NewSpace industry, take a look at Cloudflight’s white paper on the topic here.
Manufacturers’ continued successes in advancing the design, manufacture, testing, and flying of cutting-edge satellites in space indicates that there will be a continued increase in the volume of quality data generated. Satellite communications systems and associated ground stations have also become more advanced and efficient, increasing the volume of quality data that can be transferred to the ground for processing.
In addition, the business use cases of satellites are also evolving, with emerging models such as satellite-as-a-service and data-as-a service gaining traction. These can also result in an increase in the volume of useful data generated.
As mentioned, satellite data used in industry includes imagery across different spectral wavelengths and with variations in resolution, fidelity, accessibility, and size. And as the availability of space data has increased, applications have also expanded, and business models have emerged in monitoring areas such as land, marine, atmosphere, climate change, disaster management, security, and more.
For example, satellite data approaches that were only used for weather forecasting in previous years are now also employed for Earth surface monitoring applications such as identifying invasive plants, measuring vegetation yields, and tracking naval vessels.
Users also have access to open satellite data from providers such as Landsat and Copernicus. These can be integrated and utilized with data from other sources to create new use cases. While satellites and their launches are a common focus in the space industry, particularly from the outside, there is often less public awareness of the value of exploiting such existing space data in this way.
In summary, due to growing demand, a wide range of applications can be developed if companies have the capacity to efficiently exploit the information held in the terabytes of new and historical data that can be accessed today.
And in order to do so, cloud- and on-site-based data processing services have been evolving and scaling up to meet the growing needs of the space industry. In the next section we take a look at the benefits that such solutions can bring to space applications.
Accelerated data processing in space
The ability to more efficiently process large amounts of data enhances the sustainability of any company or process. Simply put, the less time and energy that is required to crunch the data and extract the insights or value needed, the fewer emissions are ultimately generated.
This also has an obvious financial impact. The unit economics of any space service include the cost of processing across the data pipeline, so reducing such costs improves the profitability of each business case.
For example, Cloudflight worked with well-known satellite operator Spire on projects that exploit data collected from the company’s constellation of over 100 multi-purposes satellites. Cloudflight has helped to scale Spire’s data processing resources and implement new processes in order to lower ongoing operating costs. Similarly, Cloudflight had accelerated the GRASP algorithm for ESA to make it applicable globally and for full mission timeframes.
Alongside lowering operational costs and enabling new products and services, advanced data processing also speeds up time-to-market. An experienced provider is able to help satellite operators create a market-ready service faster by leaning on experience to develop commercial-level data, and scaling the operation when needed.
Cloudflight has developed deep expertise in the processing of space-borne data in a variety of market segments. Before we review some of this work, first let’s introduce the company more fully.
About Cloudflight
Cloudflight is one of the most well-established full-service providers for digital innovation and solutions in Europe, operating across a number of industrial sectors, including the space industry. Cloudflight has over 20 years’ of experience in digital technologies and has successfully carried out more than 1,000 projects around the world.
In recent years AI, machine learning (ML), and cloud-based services have become an integral part of several industries. While a significant number of companies are already providing digital solutions and services in these areas for terrestrial applications, the space industry has also started integrating more digital technologies; both in operational and processing capabilities.
The space industry’s growing demand for ease in operations, high volume data management, and operating efficiencies for network management systems, has resulted in this gradual adoption of AI, cloud computing, and advanced data processing capabilities.
Leveraging such emerging technologies is also enhancing product output in verticals such as Earth Observation (EO); where data processing, management, and end product development processes can be simplified and improved. In addition, the evolution of new AI capabilities has helped some space businesses to scale operations and develop innovative new products.
In all of these processes, Cloudflight is helping companies explore new territory (sometimes literally!) in the space sector, using cloud-based digital solutions to enhance processing capabilities.
EO in particular is one of the space verticals in which higher volume data processing, management, and distribution are still yet to meet their full potential; particularly in terms of consumer affordability. And as NewSpace technologies have progressed, EO applications are increasingly exposed to the consumer market, leading to more companies participating in downstream product development processes.
Next let’s take a close look at Cloudflight’s work in space.
Cloudflight’s work in the space sector: data platforms, data processing, and data acceleration
Cloudflight has a long history of using space data and providing services around it, whether it is active and passive Earth Observation data (optical, radar, lidar), satellite communication, or the collection of Automatic Dependent Surveillance–Broadcast (ADS-B) data discussed earlier.
One example of scientific cooperation is the development of the GRASP algorithm for the detection of aerosol particles and surface reflectance, together with the University of Lille and GRASP SAS, to have a better understanding of the effects of aerosols on global climate development. The teams continuously collaborate on optimizing the algorithm as well as on new projects with newer and more accurate data, often even with new satellites or instruments.
Data collection and insight generation have increasingly gained momentum in the EO field. Rapid growth in the software industry and the incoming wave of innovative technologies, such as ML and other AI techniques, are also driving cross-industrial adoption, and stakeholders in the space industry have also started utilizing these technologies on a larger scale.
Cloudflight’s work in the fields of space science and technology is expanding, and downstream industries are increasingly utilizing its services to strengthen their product lines and services. The company also carries out platform development to provide satellite operators with a commercially-viable system to deliver end-user data that is designed to be secure, robust and efficient.
For new teams, this process typically begins with an assessment of a service concept using historical data, discussed next.
Building space data business models
Today’s space industry is moving at a faster pace than ever. And although the barriers for entry have significantly lowered in recent years, particularly with respect to launch and component costs, customer expectations have also grown, meaning the value of any commercial service needs to be proven as quickly as possible.
One established route that new entrants to the market are taking is to model business cases and service concepts with historical data. Publicly available data is often a good starting point, but there can be limits on this in terms of quality, completeness, availability, data definition information, and other factors.
Cloudflight can help operators with access to additional archives of data with which new ideas can be tested or validated and establish contacts to a strong R&D network. This information can significantly de-risk and reduce time-to-market for a team developing a new product or service.
Alongside facilitating access to such data, Cloudflight also has broad expertise in supporting companies to develop their algorithms and make them production-ready. These services include algorithm optimization, benchmarking, the generation of data models, High Performance Computing (HPC), and other activities required to help their customers develop products.
It is important for any business, whether new or established, to be versatile in today’s market. Next-generation software, payloads, OBCs, and other satellite sub-systems are enabling a new wave of multi-purpose systems in space.
It is now possible to repurpose and reposition even some hardware to meet new business requirements, and utilize software-defined satellite concepts to adapt to changing conditions on-orbit.
Such versatile functions require agile data processing capabilities that can change and upgrade rapidly. This is a core aspect of Cloudflight’s services, which the company has developed over a long history of different industry projects.
Finally, one of the key drivers of a successful space-based data service is the strength of the operator’s partnership with the processing provider. Next, let’s take a look at how satellite operators can work closely with space data processing service providers, such as Cloudflight, in order to create more sustainable and profitable services.
Working with a space data processing provider
Selecting and acquiring the right processing solution for a satellite data service is a different procurement task to simply purchasing off-the-shelf hardware components. In contrast to spending an endless amount on the adaptation of standard tools and libraries, Cloudflight follows an agile approach, together with prospective clients, to design and develop bespoke solutions tailored to their specific needs, resources, end-users, and strategic goals.
This begins with an initial meeting to flesh out the challenges and opportunities, which can be held under a non-disclosure agreement (NDA) if required, and determine the most applicable use cases where a new custom solution is required or more efficient data processing can play a key role.
When working on the data processing pipeline, there are several things that satellite operators can prepare as entry points, which will also be beneficial to the entire operation, such as:
- Develop easily readable code according to good practices,
- Ensure that an overview of the entire data pipeline can be made available to the processing solutions provider,
- Ensure the availability of relevant data (subject to any business sensitivity, commercial confidentiality, or other privacy restrictions), and
- Provide some historical reference data if more recent data is unavailable.
Cloudflight’s established and proven client on-boarding processes are designed to get a project up and running quickly, often within a week. The company is organized in virtual teams across Europe to provide high levels of flexibility and have shown in many projects that remote work is as good as on-site collaboration.
This way, they can provide their partners with the best resources needed to solve a challenge.
Working closely together with the satellite operator’s teams results in efficient communication and a mutual exchange of know-how so that even if the cooperation ends, the knowledge about how things work – and importantly what didn’t work – is also available at the operator’s side and not lost.
Ultimately, in the modern market the value of a satellite service is determined by the value of the data the company can provide to end-users. The speed, quality, and efficiency of the processing of that data is a key driver of this value and improving it can lead to a consistently more profitable and more sustainable service.
Cloudflight has invested heavily in developing a team that they believe has the skills and experience to meet any challenge in the space domain.
As a larger data processing company they have the stability and processes to provide value in services at any scale while remaining versatile enough to adapt to changing market conditions.
To find out more about how Cloudflight can help your company build a more sustainable and successful space service, please visit their satsearch supplier hub here.