A satellite photo from Planet shows military aircraft arriving at Luninets Air Base in Belarus about 60 kilometers (40 miles) from the Ukrainian border, Feb 28, 2022

A satellite photo from Planet shows military aircraft arriving at Luninets Air Base in Belarus about 60 kilometers (40 miles) from the Ukrainian border, Feb 28, 2022 PLANET

A ‘ChatGPT’ For Satellite Photos Already Exists

Using advanced generative AI and a massive dataset of Earth images, it’s possible to discover objects almost anywhere in just hours.

Scene: A U.S. adversary is at work on a new type of drone, ship, or aircraft and it’s your job to find it, wherever it is. 

Not long ago, that task would take a massive effort of human, signals, and open-source intelligence collection. But a researcher from AI company Synthetaic has created a tool that will allow users to find virtually any large object that exists in any satellite photo of the Earth within just one day. It’s also the sort of capability the National Geospatial-Intelligence Agency is also looking to develop, and it could radically shift strategic advantage on the battlefield. 

Corey Jaskolski, founder and CEO of Synthetaic, dubbed his satellite image scanning tool Rapid Automatic Image Categorization, or RAIC. After the Chinese weather balloon incident caught the nation’s attention in January, Jaskolski applied RAIC to satellite photos of the Earth’s surface, as collected by geospatial satellite imaging company Planet. He was able to trace the balloon’s origins to China in just a matter of days. 

Now, Jaskolski says, the company is using those lessons to further reduce the time. “Our goal is to be able to ingest the entire Planet daily take [of Earth images] and be able to process that all in less than 24 hours. So if you wanted to literally look for balloon launches around the entire world, we could give you a daily update of that every day. Let you know if there was a balloon launched anywhere.”

Interest around new publicly available AI tools has been spiking, thanks to new generative pretrained transformer—or GPT—tools that allow users to write essays, build business plans, and perform complex tasks with a simple prompt. The national security community has a similar need, but for AI applications for the vast expanse of satellite, surveillance, and other data that could help uncover adversary activities and new capabilities. 

But it’s not necessarily a straightforward task, as Jaskolski learned when he attempted to find the origin of that Chinese balloon—a thing that had never been photographed in the open, much less labeled and inserted into a dataset readable by a machine-learning algorithm.

“Normally with an AI, you have to have a bunch of labeled examples for the AI to learn, so, and it's not a small amount of data. Like when Facebook and Google train an AI, they commonly train on a billion labeled images, not even, you know, thousands or millions, but literally a billion labeled images," Jaskolski said. "The thing that would normally stop an AI from finding this balloon is we don't have any data. We don't have any labels. We don't know what it looks like from space.” 

The RAIC is part of a new class of AI tools that don’t require a massive, labeled dataset to generate what Jaskolski describes as an understanding of what to look for. He was able to teach it to look for the balloon based only on a single hand-made drawing. 

“We started out with technologies that are used for generative AI transformers and GaNS. [But] instead of using that technology to generate images, we use that technology in order to basically understand the data domain,” he said. 

In essence, by continuously looking at satellite images, the RAIC tool develops a familiarity that comes close to expertise. So when it scans satellite imagery, it has a rudimentary understanding of what’s unusual, and can look for specific unusual objects. And the input doesn’t have to be precise. Jaskolski says his drawing depicted what a balloon might look like in satellite data, and RAIC was able to find it. Then, once they found the actual balloon in one of the satellite datasets, RAIC was able to look for that in other images.

“After a couple days really searching for it in Alaska in Canada, we decided to just bite the bullet and ingest that massive amount of Earth across China, Japan, South Korea, North Korea, and the ocean, open ocean and Aleutian islands,” he said.  They also used wind modeling to narrow down where the balloon may have started its flight.

That brought them to islands 300 miles off the coast of China. “At that point we got really excited…And so from there, we find it five or six more times, all the way back to the hidden island.”

At last week’s Planet conference in Washington, Microsoft President Brad Smith described a future in which people could ask image-based search tools to find objects, just as we ask search engines for recommendations today. Microsoft is a major investor in OpenAI, the best-known GPT platform.

“I do believe that this next era of AI, you know, with GPT based-technology, is a query-able Earth,” Smith said. 

NGA has already taken control of Project Maven, the Pentagon’s flagship AI program for image analysis. At the Planet conference, NGA head Vice Adm. Frank Whitworth said the agency is trying to turn Maven from an experimental effort into a program of record, “which means we will need to be very clear on the efficacy of every dollar.”

The agency is “experimenting with [geographical intelligence] AI programs that integrate large language models to allow analysts to ask and answer specific intelligence questions,” an NGA spokesperson told Defense One. “We see a future where these models can be trained with big spatial data to answer questions in space and time.”

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.