First AI image from space with HyperScout

For the first time in history an image was processed in space using artificial intelligence. The image was processed by the tailored artificial intelligence hardware of HyperScout 2, a miniaturized Earth observation instrument that is developed under the leadership of cosine. The deep neural network algorithm identified the clouds in an image of part of the Earth’s surface. The capability to process images using artificial intelligence on a satellite opens up possibilities for a large number of applications.

The premier on artificial intelligence in space was announced today by Josef Aschbacher, director of the Earth Observation program of the European Space Agency, at the opening of the Phi-Week 2020 symposium. The HyperScout 2 instrument, carrying the ?-sat-1 artificial intelligence experiment, was launched into space on 3 September 2020 from the Guiana Space Center in Kourou, on one of the 53 satellites on board Arianespace Vega flight VV16. The HyperScout 2 instrument is on board one of the two nanosatellites of the FSSCat mission that monitors sea ice and soil moisture in support of the Copernicus Land and Marine Environment Services, and was made possible by ESA’s InCubed program.

Integrated into the HyperScout 2 instrument is a Myriad 2 Vision Processing Unit (VPU) from Intel. This allows the instrument to process the spectral images with machine learning algorithms, without requiring more power than available on a nanosatellite. The identification of clouds in an image from Earth is a first demonstration of the capabilities of an AI system on board an Earth observation satellite. The AI algorithm is trained on ground using machine learning on synthetic as well as HyperScout data, which includes properties of the image that are invisible to the human eye due to the 45 different color bands in the visible and infrared spectrum. The capability to process spectral images in space using artificial intelligence makes a large range of applications possible.

Applied to Earth

One of the potential applications is wildfire management. High risk fire zones can be mapped, alerts can be triggered and the development and spreading of an eventual wildfire can be monitored. There are also many possible benefits for agriculture, such as crop yield prediction, indicating how much crop could be harvested and when to harvest. This makes it possible to take action, such as optimizing irrigation, adjusting levels of fertilization and targeted use of minimal levels of pesticides.

Using a constellation of multiple HyperScout instruments, observations can be made several times per week or even per day. This can also be of great benefit for environmental monitoring, water quality, air quality, deforestation and change detection.

Next steps

“Data that is acquired by the HyperScout 2 instrument can be combined with data from the HyperScout 1 instrument, which has been in orbit for almost 3 years on the GOMX-4B satellite. In addition a version of the AI algorithms developed for the HyperScout 2 VPU can be uploaded to the GPU, the graphic processing unit, of HyperScout 1. This presents a unique opportunity for partners to work with cosine to develop applications using distributed AI on both HyperScout

1 and HyperScout 2″, explains Marco Esposito, business unit manager Remote Sensing at cosine. “Beyond that, we are also looking into the use of AI in the other instrument lines of cosine. This includes infrared imaging for agricultural monitoring, imaging polarimetry for cloud and aerosol assessment, spectroscopic imaging for air quality and LIDAR for height mapping and bathymetry.