Federal Use of Cloud-Based Big Data Solutions, FY 2019-2021

Procuring technology solutions used to be relatively straightforward. Need a new capability for financial management, buy it on the General Service Administration’s Information Technology Schedule 70. Need some new servers, buy them on the National Aeronautics and Space Administration’s Solutions for Enterprise-Wide Procurement V Government-Wide Acquisition Vehicle. Nowadays, however, IT solutions don’t fit neatly into established market segments and nowhere is this more obvious than in the cloud computing market. Today, if customers require a capability they can buy it as a standalone piece of software or they can buy it as a cloud-based Software-as-a-Service. Federal agencies get it and they are gradually accommodating the new, more complex reality by buying solutions that can fall into multiple market segments.

One area where this trend is particularly visible is in the convergence of the cloud and “big data” markets. Big data, broadly speaking, includes more than just advanced analytics and visualization tools. It can also include high performance computing (HPC) and modeling and simulation work. The question is whether these capabilities are delivered as-a-Service or not.

Procuring big data solutions can also be daunting to contracting officials with little technical background. In this case leveraging cloud-based solutions can make things a little simpler and the procurement data shown below illustrates the point.

In fiscal 2021, federal agencies obligated (i.e., spent) $350M on big data solutions delivered via the cloud. This represented a slight decrease from the year before, but a 45% increase vs. spending on similar solutions in Fiscal Year (FY) 2019.

What constitutes a big data solution in this case? The data includes spending on advanced analytics, artificial intelligence/machine learning (a form of advanced analytics), analysis support services, and HPC.

The total value of contracts (TCV) awarded for these types of capabilities/work saw an even higher increase due to the award of a contract by the National Security Agency for HPC-as-a-Service valued at $2.0B.

Additional observations on the federal use of cloud-based big data solutions include the following:

• In FY 2021, civilian agencies spent $271M vs. $78M spent by the Department of Defense.
• Business Intelligence solutions led advanced analytics spending with agencies obligating $284M in FY 2021.
• Spending on cloud-based AI/ML crawled higher. Agencies spent $3.6M in FY 2021 vs. $619K in FY 2019.
• The top 3 advanced analytics solutions by spending (3-year totals) were Tableau ($452M), Palantir ($73M), and UFED Cloud Analyzer ($28M).
• The top 5 agencies by analytics spending (3-year totals) were Education ($451M), Health and Human Services ($50M), Homeland Security ($50M), the Office of the Secretary of Defense ($39M), and the Treasury ($29M).

As a sub-segment of the federal cloud market the procurement and use of big data solutions is growing nicely. This is a trend that ought to continue in the years to come.