Preliminary Spending Data Shows Big Data Software Strength

Big data can be tricky technology market to analyze because its solutions are often spread across multiple segments. Transport bandwidth is absolutely necessary for big data, but it can be classified as either communications/network spending or hardware. Similarly, storage can be hardware, software, or services. Thankfully, spending on software is the easiest to isolate due to the prominence of certain brand names in the market, so this post provides a few numbers on big data software spending based on an analysis of the available obligations data from fiscal years 2014 to 2016. Readers are advised to keep in mind that this is only a preliminary analysis. A final and complete analysis of big data software spending from FY 2014-2016 will be published in FMA’s Federal Progress Update report in October.

Total Big Data Software Spending by Fiscal Year

The numbers presented here show spending on big data-related software by government agencies, including analytics, databases, visualization, high performance computing, and other types of applications. This spending shows strong growth of 44.2% over the three years from FY 2014 to FY 2016.

Growth in agency spending on big data-related software is largely attributable to two factors: relative stability in agency budgets over the three-year period and growing comfort among agencies with using big data-related technologies.

Most Popular Analytics Solutions

In terms of the most popular analytics solutions purchased by agencies, the following chart shows Palantir in the lead, with SAS and Splunk following. Keep in mind that these numbers are for analytics solutions only so software frameworks like Hadoop are not included.

Spending by Agency

With Palantir at the top of the list it will probably come as no surprise that law enforcement and military departments are heavily represented in the top 11 agency buyers of big data-related software.

These agencies appear to be the most experienced users of big data analytics. Similarly, they are the ones with the most pressing mission need for analytics solutions. This points to other factors influencing the growth of big data solution use across government. Namely, in addition to pressing national security and law enforcement concerns, the appearance of agencies like HHS and VA on the list also points to the growing tidal wave of data-driven medical care and disease tracking (i.e., IoT!). Lastly, those agencies with massive transactional responsibilities – like Treasury and SSA – also use analytics to meet their mission objectives. Add other agencies like NASA and DOE with scientific objectives to the mix and you have a hierarchy of the mission areas most likely to use big data analytics.

Finally, a word about budget constraints is necessary. The growth of spending on big data-related software is encouraging, but it is also fragile. Data from several years ago showed that agency spending on big data across all segments slumped in fiscal 2013 thanks to the impact of sequestration. Should President Trump’s first budget in fiscal 2018 be signed it could also signal another decline in the market.