NIST Releases Plan to Set AI Standards in Motion

The importance of artificial intelligence (AI) in the U.S. is increasing rapidly. Recognizing the significant impact AI could have on national security, the economy and quality of life, U.S. leaders are striving to pave a robust path for the technology. In February 2019, the President issued Executive Order (EO) 13859, “Maintaining American Leadership in Artificial Intelligence” to enhance the federal government’s role in, among others, “facilitating AI R&D and promoting the trust of the American people in the development and deployment of AI-related technologies.”

Within its requirements, the EO calls for NIST to issue a plan for federal engagement in the development of technical standards and the required tools to support trustworthy systems that use AI. The plan is to identify federal priority needs for standardization of AI systems, the entities which federal agencies should seek technical leadership roles in AI, and the opportunities and challenges facing leadership in the standardization of AI technologies.

Development of the plan calls for outreach into the government and private communities. In response, NIST issued an RFI in the Federal Register, held discussions with federal agencies, and provided a draft of the standards plan for public review and comment. Additionally, NIST held a workshop in May with private industry and academia, leading discussions in setting the national/international stage to defining AI standards, both technically and as a policy tool.

Earlier this month, NIST released the anticipated plan, which “provides guidance for bolstering the engagement of Federal agencies in AI technical standards to promote continued U.S. leadership in AI.” The plan addresses each angle of the EO’s requirements including, providing a strategy to define AI standards, establishing priorities for federal government involvement in AI standards, and recommending federal government actions to advance in AI leadership.

STANDARDS AND ARTIFICIAL INTELLIGENCE

The plan emphasizes the need for two types of AI standards – those specific to AI application as well as standards for parts of an AI-driven system. Citing technical and non-technical AI standards that already exist or being developed, NIST acknowledges that future AI standards creation must be consistent with these policies and principles, set by both public and private sectors. Moreover, existing IT governance standards should also be taken into consideration as they may be equally be applicable to AI as they are to other technologies.

To advance the development and adoption of AI, standards must complement the following set of related tools:

• Data sets in standardized formats
• Tools for capturing and representing knowledge and reasoning in AI systems
• Fully documented use cases
• Benchmarks, evaluations and problems in the evolution of AI
• Testing methodologies
• Metrics to measure and characterize AI technologies
• AI testbeds
• Tools for accountability and auditing of AI system outputs

Development of AI standards must begin with reaching a consensus on a core set of AI principles, NIST advises. Moreover, AI standards must include societal and ethical considerations, though how to accomplish this is not yet clear. Leaders must also find a delicate balance for when to identify AI standards – premature standards could get in the way of innovation while delayed standards could prove difficult in achieving sector consensus.

Overall, standards tend to guide technology advancement and innovation, and promote a larger, more interoperable marketplace. Specifically, standards for AI  “that articulate requirements, specifications, guidelines, or characteristics can help to ensure that AI technologies and systems meet critical objectives for functionality, interoperability, and trustworthiness—and that they perform accurately, reliably, and safely.”

PRIORITIES AND RECOMMENDATIONS FOR U.S. GOVERNMENT’S ROLE IN AI STANDARDS

According to NIST, the federal government must pursue AI standards that are, among others, consensus-based, inclusive, open and transparent. Other priorities outlined for the federal government in AI include clearly stated scopes, management of AI systems and human-centered standards. Additionally, government must take into consideration both horizontal AI standards, which will apply across several AI applications and vertical standards that will focus on specific application areas such as healthcare or transportation.

In addition to these priorities, the plan recommends, “the Federal government should commit to deeper, consistent, long-term engagement in AI standards development activities to help the United States to speed the pace of reliable, robust, and trustworthy AI technology development.” To do this, NIST encourages the federal government to bolster knowledge of AI among federal agencies, focus on research to accelerate understanding of technology trustworthiness and support public-private partnerships to develop and use AI standards. Finally, NIST recommends that the U.S. government engage globally to advance AI standards for purposes of national security and economic needs.