Three Takeaways from NITRDs New Big Data Research and Development Strategy

Published: June 01, 2016

Big DataCloud ComputingSupercomputing

Investment will be required for federal agencies to leverage big data to the extent they desire. NITRD’s Big Data Strategic Steering Group lays out where those investments could be made.

Toward the end of May, the Big Data Steering Group of the Subcommittee on Networking and Information Technology Research and Development, or NITRD, posted a strategic plan for Federal Big Data Research and Development. NITRD sits at the center of advanced technology development and investment in the U.S. federal government, so the release of a strategic plan for big data can help industry understand where agencies are going when it comes to R&D spending.
The NITRD plan is organized around seven strategies that represent key areas of importance for Big Data R&D, with the priorities listed under each strategy highlighting intended outcomes that can be addressed by the missions and research funding of NITRD agencies. For those counting, the agencies and organizations coordinating their actions through NITRD are NIST, NOAA, DARPA, NSA, OSD, Army, Navy, Air Force, DOE, HHS, DHS, EPA, NASA, NARA, NRO, NSF, and OMB.
The seven strategies outlined in the plan are:
  1. Agencies should create next-generation capabilities by leveraging emerging Big Data foundations, techniques, and technologies.
  2. Agencies must support R&D to explore and understand the trustworthiness of data and resulting knowledge, to make better decisions, enable breakthrough discoveries, and take confident action.
  3. Agencies should build and enhance research cyberinfrastructure that enables Big Data innovation in support of agency missions.
  4. Agencies should increase the value of data through policies that promote its sharing and management.
  5. Efforts must be made to understand Big Data collection, sharing, and use with regard to privacy, security, and ethics.
  6. Agencies should work to improve the national landscape for Big Data education and training to fulfill increasing demand for both deep analytical talent and analytical capacity for the broader workforce.
  7. Agencies must create and enhance connections in the national Big Data innovation ecosystem.
Given how nebulous these strategic goals are, what are the key takeaways for vendors offering big data goods and services?
First, to keep pace with the size, speed, and complexity of data, NITRD is recommending that its member agencies scale up their computing systems. Here the determination to scale up runs into the fiscal reality of tightening budgets. Only a handful of courses are available. Agencies can a) invest in their own computing infrastructure, b) they can rely more on commercial cloud providers, or c) they can share computing infrastructure, particularly high-performance computers in the DoD and at civilian agencies like DOE and NOAA. Because sharing HPC infrastructure is done on a timed basis and may not be convenient or provide capabilities as often or as long as are required, the most cost-effective solution will be to work more closely with commercial cloud providers.
Takeaway – Increasing the use of cloud is inevitable. Some agencies may make targeted investments in computing power for specific purposes, but agency demand for computing power will invariably lead to higher spending on cloud.
Second, the Steering Group believes it likely that in the future an increasing number of decisions will be informed by big data processes. The key here, given the exponential growth of data vs. finite fiscal resources, is artificial intelligence. Agencies interested in more than just R&D will therefore become increasingly reliant on artificial intelligence to make data actionable.
Takeaway - AI is starting out slowly, but evolving rapidly. Vendors with the resources should begin positioning for increased demand by agencies to make AI systems less expensive, more readily available, and easier to employ. Combine AI with a cloud environment and you have the system of the future.
Third, big data professionals have increasingly recognized the importance of keeping a “human-in-the-loop” when it comes to making use of processed data. Part of this is addressing the critical shortage of data scientists. In part, this challenge will solve itself as more IT professionals are attracted by the potential for a lucrative career, but for industry the benefit will be in making these people available to agency customers.
Takeaway – Develop a big data workforce, even if this means fostering its long-term growth by partnering with educational institutions to bring current students into your stable of data science practitioners. Federal agencies will always be pressed to find the people they require, meaning they will turn to vendors to provide the human expertise they need.
For a copy of the NITRD’s Federal Big Data Research and Development Strategic Plan, click here.