A Breakdown of the $21B in Historical Big Data Contract Obligations

Federal big data spending grew 10% from FY 2020 to 2023 and totaled nearly $21B, continuing a trend of expansion in this specific IT marketplace.

Each year, the Federal Market Analysis team looks at three full fiscal years of contract spending in a number of IT markets. In a recent analysis of big data obligations, the team identified nearly $21B in big data-related goods and services from FY 2020 to 2022, almost a 10% increase over that three-year period.

The expansion of the federal big data market is driven by the government’s continued focus on evidence-based decision making, particularly in areas of Biden Administration priorities such as customer experience metrics, workforce environment assessments, and environmental data regulation. Open data practices and the push for AI adoption are also contributing factors to the growing big data landscape.

Though the three-year period includes $1.2B in Covid-related obligations, the pandemic had little impact on the overarching growth of the big data market. Rather, it influenced specific spending peaks at a few agencies in FY 2020 and 2021.

In May, preliminary analysis was conducted on big data obligations, however, after extensive search of over 4 million lines of spending data, Deltek identified the following big data spending among top departments from FY 2020-2022:

Sources: FPDS, Deltek

Observations:

• Within the defense sector, Army and Air Force experienced significant growth, 75% and 73% respectively, in big data obligations from FY 2020-2023, driven by additional spending on Analysis Support at Army, and AI/ML and High Performance Computing at Air Force,
• Under the civilian market, NASA and Treasury grew 145% and 63%, respectively, in the three-year period, driven by additional investments in Data Management and Integration.
• Defense Agencies decreased in spending, from $1.5B in FY 2020 to $1.3B in FY 2022, due to a downturn in AI/ML spending at the Defense Finance and Accounting Service and Defense Logistics Agency.
• Agencies in the defense and civilian sectors relied most on the GSA Multiple Award Schedule ($3.4B) and SEWP V ($1.5B) vehicle for purchase of big data products and services from FY 2020-2022.
• Small businesses were awarded 46%, or $9.5B, of total big data spend from FY 2020-2022.

For deeper analysis, Deltek also categorizes identified big data obligations into eight “pockets”, or solution types, of big data goods and services (see here for description of each). Growth trends among the solution types over the three-year period include:

Observations:

• Storage and servers continue to see a decrease in the big data market due to the increased use of cloud computing storage and services.
• The increased High Performance Computing trend is driven by additional investment under Army’s USACE HPC Modernization Program Technology Insertion initiative.
• Analysis Support experienced a spending peak in FY 2020 due to increased spending at the SBA for Data Analysis and Loan Recommendation Services for COVID-19.
• Data Management and Integration spending is driven by civilian agency investment specifically in big data services, which totaled $739M in FY 2020 and grew to $1.2B in FY 2023.
• Of the analytics types identified, Business Analytics led spending under the Analytics solution type and totaled $847M, followed by Security Analytics ($474M) and Health Analytics ($373M).
• Big Data Systems and Platforms peaked in FY 2020 due to COVID-19 related task orders at HHS and VA.  
• The Biden Administration’s push for evidence-based analysis is likely a contributing factor in the increase for Visualization-related products and services.

Deltek’s upcoming report, Federal Big Data Market, 2023-2027, due for publish at the end of October 2023, will contain additional insight to big data contract spending trends, as well as the policies, technologies and other drivers propelling this expanding federal IT environment.