National Reconnaissance Office in Chantilly, Virginia

National Reconnaissance Office in Chantilly, Virginia Trevor Paglen

Daring Deal

Amazon is building a cloud for the intelligence community under a plan to upend the status quo.

The intelligence community is about to get the equivalent of an adrenaline shot to the chest. This summer, a $600 million computing cloud developed by Amazon Web Services for the Central Intelligence Agency over the past year will begin servicing all 17 agencies that make up the intelligence community. If the technology plays out as officials envision, it will usher in a new era of cooperation and coordination, allowing agencies to share information and services much more easily and avoid the kind of intelligence gaps that preceded the Sept. 11, 2001, terrorist attacks.

For the first time, agencies within the IC will be able to order a variety of on-demand computing and analytic services from the CIA and National Security Agency. What’s more, they’ll only pay for what they use. 

The vision was first outlined in the IC Information Technology Enterprise plan championed by Director of National Intelligence James Clapper and IC Chief Information Officer Al Tarasiuk almost three years ago. Cloud computing is one of the core components of the strategy to help the IC discover, access and share critical information in an era of seemingly infinite data. 

For the risk-averse intelligence community, the decision to go with a commercial cloud vendor is a radical departure from business as usual.

In 2011, while private companies were consolidating data centers in favor of the cloud and some civilian agencies began flirting with cloud variants like email as a service, a sometimes contentious debate among the intelligence community’s leadership took place. 

As one former intelligence official with knowledge of the Amazon deal told Government Executive, “It took a lot of wrangling, but it was easy to see the vision if you laid it all out.” 

The critical question was would the IC, led by the CIA, attempt to do cloud computing from within, or would it buy innovation? 

Money was a factor, according to the intelligence official, but not the leading one.  The government was spending more money on information technology within the IC than ever before. IT spending reached $8 billion in 2013, according to budget documents leaked by former NSA contractor Edward Snowden. The CIA and other agencies feasibly could have spent billions of dollars standing up their own cloud infrastructure without raising many eyebrows in Congress, but the decision to purchase a single commercial solution came down primarily to two factors.

“What we were really looking at was time to mission and innovation,” the former intelligence official said. “The goal was, ‘Can we act like a large enterprise in the corporate world and buy the thing that we don’t have, can we catch up to the commercial cycle? Anybody can build a data center, but could we purchase something more?

 “We decided we needed to buy innovation,” the former intelligence official said.

A Groundbreaking Deal 

The CIA’s first request for proposals from industry in mid-2012 was met with bid protests to the Government Accountability Office from Microsoft and AT&T, two early contenders for the contract. Those protests focused on the narrow specifications called for by the RFP. GAO did not issue a decision in either protest because the CIA reworked its request to address the companies’ complaint.

In early 2013, after weighing bids from Amazon Web Services, IBM and an unnamed third vendor, the CIA awarded a contract to AWS worth up to $600 million over a period of up to 10 years. The deal, handled in secret, was first reported by FCW in March 2013, sending ripples through the tech industry.

A month after the deal became public, IBM filed a bid protest with GAO that the watchdog eventually upheld in June, forcing the CIA to reopen bids to both companies for the contract. A legal struggle between Amazon and Big Blue ensued, and AWS filed a lawsuit against the federal government in July 2013, claiming the GAO sustainment was a “flawed” decision. 

In October, U.S. Court of Federal Claims Judge Thomas Wheeler sided with Amazon and overturned GAO’s decision to force the CIA to rebid the contract. Big Blue went home, AWS claimed victory under the deal’s original financial specs, and nearly 18 months after the procurement was first released, the CIA and Amazon went to work.

It is difficult to underestimate the cloud contract’s importance. In a recent public appearance, CIA Chief Information Officer Douglas Wolfe called it “one of the most important technology procurements in recent history,” with ramifications far outside the realm of technology. 

“It’s going to take a few months to bring this online in a robust way, but it’s coming,” Wolfe said.  “And I think it’s going to make a big difference for national security.” 

Securing New Capabilities 

The Amazon-built cloud will operate behind the IC’s firewall, or more simply: It’s a public cloud built on private premises. 

Intelligence agencies will be able to host applications or order a variety of on-demand services like storage, computing and analytics. True to the National Institute of Standards and Technology definition of cloud computing, the IC cloud scales up or down to meet the need. 

In that regard, customers will pay only for services they actually use, which is expected to generate massive savings for the IC.

“We see this as a tremendous opportunity to sharpen our focus and to be very efficient,” Wolfe told an audience at AWS’ annual nonprofit and government symposium in Washington. “We hope to get speed and scale out of the cloud, and a tremendous amount of efficiency in terms of folks traditionally using IT now using it in a cost-recovery way.”

Many agencies within the IC already have identified applications to move to the cloud. In a recent report, National Reconnaissance Office Chief Information Officer Donna Hansen said her agency had picked five applications, including its enterprise resource planning software, to migrate to the IC cloud. 

As with public clouds, the IC cloud will maximize automation and require standardized information, which will be shared through application programming interfaces, known as APIs. Amazon engineers will oversee the hardware because AWS owns the hardware and is responsible for maintaining it just as they do in the company’s public data centers. 

Whenever Amazon introduces a new innovation or improvement in cloud services, the IC cloud will evolve. Company officials say AWS made more than 200 such incremental improvements last year, ensuring a sort of built-in innovation to the IC cloud that will help the intelligence community keep pace with commercial advances. Wolfe said AWS’ capacity to bring commercial innovation from places like Silicon Valley to the IC is one of the contract’s greatest benefits. Whenever AWS introduces new products, the CIA will be able to implement them.  

“The biggest thing we were trying to do—the visionary folks a couple years ago—was answer the question, ‘How do we keep up?’” Wolfe said. “The mission we have is important. The pace and complexity is really not [diminishing], in fact, it may be increasing. We feel it is very important to deliver the best IT and best products and services we can to our customers in the IC.”

What of the data, though? Intelligence agencies are drowning in it, collecting and analyzing an amalgamation of information from sensors, satellites, surveillance efforts, open data repositories and human intelligence, among other sources. Is that data really secure in the cloud? 

The CIA is convinced it is.

The IC cloud “will be accredited and compliant with IC standards,” says a senior CIA official familiar with the IC cloud. It will, for example, be able to handle Sensitive Compartmented Information, a type of classified information. “Security in the IC cloud will be as safe as or safer than security on our current data centers,” the senior CIA official says.

Because the IC cloud will serve multiple tenants—the 17 agencies that comprise the IC—administrators will be able to restrict access to information based on the identity of the individual seeking it. The idea is to foster collaboration without compromising security. Visually, the IC cloud can be thought of as a workspace hanging off the IC’s shared network—a place where data can be loaded for a variety of tasks like computing or sharing. The IC cloud gives agencies additional means to share information in an environment where automated security isn’t a barrier to the sharing itself. This could prove vital in situations reminiscent of 9/11, in which national security is an immediate concern.  

Cloud vendors, including Amazon, have argued that cloud infrastructures can be more secure than traditional data centers because there are fewer points of entry, but the leaks by Snowden illustrate the potential threat from inside an organization. Snowden was able to access and download classified information intelligence officials said he shouldn’t have been able to access. 

To access information within the IC cloud, analysts must have the proper permissions. In addition, the standardized environment and automation means all activity within the cloud is logged and can be analyzed in near real-time. 

Some government officials view cloud computing as inherently less secure than computing on locally controlled servers, but the CIA’s acceptance of commercially developed cloud technology “has been a wake-up call” to those who balk at it, according to John Pirc, a former CIA cybersecurity researcher who is now chief technology officer at NSS Labs, a security research firm. 

“You hear so many people on the fence about cloud, and then to see the CIA gobble it up and do something so highly disruptive, it’s kind of cool,” says Pirc. “To me, this removes the clouded judgment that cloud isn’t secure. Their moving forward with this should send a message to the rest of the industry that cloud is something you shouldn’t be afraid of.”

Pirc is no stranger to disruptive technologies. At the CIA’s research labs in the early 2000s, he recalls virtualization—a technology that allows multiple operating systems to run simultaneously on the same servers, allowing for far more efficient computing—before it became an integral component of many IT enterprises.  Intelligence agencies use commercial off-the-shelf technology all the time, but to Pirc, the importance of the cloud capabilities the CIA gets through leveraging Amazon Web Services’ horsepower is best exemplified in computing intelligence data. Scalable computing is critical for fostering shared services and enhanced collaboration between disparate intelligence agencies.  

“What it allows them to do is spin up servers and add more [computing power] fast, and when you’re computing intelligence data, the more compute power you have, the faster you can react,” Pirc says. “In the private sector, compute is all about money and profit, but from my viewpoint when I worked for the agency, you’re working with extremely time-sensitive information. Being able to have that compute power, something that might have taken a couple of hours might instead take a few seconds. Profits aren’t lost when you make mistakes in the intelligence community—people die when you make mistakes.” 

A test scenario described by GAO in its June 2013 bid protest opinion suggests the CIA sought to compare how the solutions presented by IBM and Amazon Web Services could crunch massive data sets, commonly referred to as big data. 

Solutions had to provide a “hosting environment for applications which process vast amounts of information in parallel on large clusters (thousands of nodes) of commodity hardware” using a platform called MapReduce. Through MapReduce, clusters were provisioned for computation and segmentation. Test runs assumed clusters were large enough to process 100 terabytes of raw input data. AWS’ solution received superior marks from CIA procurement officials, according to GAO documentation, and was one of the chief reasons the agency selected Amazon. 

Limited Details

The CIA declined to comment when Government Executive asked about the extent of the IC cloud’s capabilities or that of the National Security Agency’s cloud. Amazon also declined to describe the IC cloud’s technical capabilities. 

It is a good bet, though, that the AWS-built cloud for the IC will have capabilities at least equal to existing capabilities Amazon has already implemented across government.

For example, the company provides the cloud bandwidth for the Securities and Exchange Commission’s collection of more than 1 billion trade records and more than a terabyte of new data per day through its Market Information Data Analytics System. This example may be prescient given that now-public surveillance efforts indicate the IC collects billions and perhaps trillions of pieces of metadata, phone and Internet records, and other various bits of information on an annual basis. The potential exists for the CIA to become one of AWS’ largest customers.  

Within the intelligence community, examples abound where the cloud’s capabilities could significantly boost the mission. 

As the geospatial hub of the community, the National Geospatial-Intelligence Agency ingests, analyzes, metatags and reports all geo-intelligence and multisource content in its flagship program called Map of the World. Geospatial data’s importance to the IC has increased in recent years, as evidenced by NGA’s nearly $5 billion budget and its staff nearly doubling in size since 2004. For intensive applications like ingesting or analyzing geospatial data, scalable computing could have a significant impact on mission performance. The cloud also could improve the way the agency shares its large data sets. 

What the IC has done with cloud is not easily replicable, according to American Council for Technology President Rick Holgate, but it is worth paying attention to. 

 “The IC has a model other agencies should look to and aspire to in terms of transforming the way they think about delivering services across a large enterprise,” Holgate says. “They are looking to common platforms and service delivery models across an entire enterprise, and not just gaining cost efficiencies, but to provide foundational capabilities to really allow it to operate.”

Whether or not the IC cloud serves as an example for the rest of government, the CIA’s quest to buy innovation will loom large for years to come.

----------

Time to Share

The Intelligence Community Information Technology Enterprise lays out a vision for the IC’s 17 agencies to securely discover, access and share information. The goal: greater mission success.

In essence, ICITE (pronounced EYEsite) changes the business model for intelligence agencies by requiring that  they share services. 

Cloud computing—behind the IC’s firewall—is one element of that vision. Think of it as the IC sharing information behind a walled castle apart from the rest of the world operating on the Internet.  

The Central Intelligence Agency and the National Security Agency are leading IC cloud development; the NSA’s privately hosted cloud was launched in 2013. Importantly, it was made available to users on new and legacy systems so that personnel on any of its systems could use it. The cloud built by Amazon Web Services at the behest of the CIA will be a shared resource for services on an as-needed basis. Intelligence agencies can order a variety of services like storage, computing, analytics, database services or application hosting. Both clouds will work in complementary fashion, according to senior CIA officials. 

Other ICITE components include a common desktop, an IC-wide applications mall, and network requirement and engineering services.

The Defense Intelligence Agency and National-Geospatial Intelligence Agency have piloted shared desktop capabilities across their agencies for several thousand users. Those capabilities eventually will spread throughout the IC. 

In essence, ICITE changes the business model for intelligence agencies, mandating shared services. Instead of each agency building out its own systems, select agencies—either one or two of those with larger budgets—are responsible for governing its major components. 

The Elephant in the Room

In cloud computing, there’s Amazon Web Services and then there’s everybody else. 

IT research firm Gartner had to rescale its famed infrastructure-as-a- service “Magic Quadrant” study in 2013 to accommodate Amazon Web Services’ enormous competitive lead. 

The quadrant ranks cloud providers based on their ability to execute operations and the comprehensiveness of their vision. For several years there hasn’t been even a close challenger to AWS. Gartner’s 2014 quadrant shows that AWS captures 83 percent of the cloud computing infrastructure market. 

In the combined cloud markets for infrastructure and platform services, hybrid and private clouds—worth a collective $131 billion at the end of 2013—Amazon’s revenue grew 67 percent in the first quarter of 2014, according to Gartner. 

While the public sector hasn’t been as quick to capitalize on cloud computing as the private sector, government spending on cloud technologies is beginning to jump. 

Researchers at IDC estimate federal private cloud spending will reach $1.7 billion in 2014, and $7.7 billion by 2017. In other industries, software services are considered the leading cloud technology, but in the government that honor goes to infrastructure services, which IDC expects to reach $5.4 billion in 2017. 

In addition to its $600 million deal with the CIA, Amazon Web Services also does business with NASA, the Food and Drug Administration and the Centers for Disease Control and Prevention. Most recently, the Obama Administration tapped AWS to host portions of HealthCare.gov. 

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.