NIST Finalizes Multiyear Effort for its Big Data Interoperability Framework

The NIST Big Data Public Working Group (NBD-PWG) has released the final version of the NIST Big Data Interoperability Framework (NBDIF) after years of effort to develop a framework to advance the progress of big data. The NBDIF primarily provides the common requirements to help agencies develop and deliver interoperable and adaptable big data analytics software based on an open reference architecture.

Specifically, the framework targets developers to create and deploy analytical software tools to work in any computing platform and transfer seamlessly to any other platform. “If software vendors use the framework’s guidelines when developing analytical tools, then analysts’ results can flow uninterruptedly, even as their goals change and technology advances,” according to NIST’s announcement.

Released in three versions, each version marks the stages of NBD-PWG work on the framework. The first version identified high-level, key components of the big data reference, while the second iteration defined the general interfaces of those components. In its final version, the NBDIF builds general applications through general interfaces to validate the framework.

The result is a series of nine volumes that make up the NBDIF:

1. Definitions
2. Taxonomies
3. Use Case and Requirements
4. Security and Privacy
5. Architectures White Paper Survey
6. Reference Architecture
7. Standards Roadmap
8. Reference Architecture Interface
9. Modernization and Adoption

Major Objectives:

Goals for the NBDIF include achieving streamlined standards and understanding for big data components, as well as:

• A common language for big data solutions, processes and systems
• Federal agency understanding of vendor’s big data software to categorize and compare solutions
• Consistent methods to implement technology to solve similar problem sets
• A roadmap of standards for interoperability, probability, reusability and extendibility of big data solutions
• Maturity elements and barriers for secure adoption of big data systems

The NBDIF announcement provides several applicable examples in the use of the NBDIF to solve common, contemporary data problems throughout federal agencies. One such example includes weather forecasting. Meteorologists must deal with analytical tools that track multiple variables of data that change simultaneously. By implementing the measures within the NBDIF, forecasting can become flexible with solutions that allow meteorologists to update improvements to existing models effortlessly.

Implications

For contractors, the common definitions, specifications and patterns within the NBDIF will help provide clarity and standardization in big data contract requirements.

Moreover, the NBDIF paves the way for vendors to develop software solutions that are interoperable and adaptable across multiple computing platforms, including cloud-based environments.

Lastly, the NBDIF provides federal government awareness in the adoption of big data software, allowing agencies to better discern analytic solutions that fit into their data environment.