An Accountability Framework for Federal AI Systems

AI technology promises to improve a host of applications and disciplines including federal government operations, however, auditing AI systems is a daunting task. Some systems lack the transparency for auditing, due to either vendor proprietary reasons or difficulty understanding the software. Thus, the GAO issued a June 2021 report establishing a framework to help ensure accountability of the AI lifecycle (design, development, deployment, and continuous monitoring).

Specifically, the GAO states that, “It is necessary to lay down a framework for independent verification of AI systems even as the technology continues to advance. Auditors and the oversight community play a vital role in the trust but verify equation, and they need a toolkit to evaluate this changing technology. More importantly, organizations that build, purchase, and deploy.”

The framework will help to ensure that AI systems remain unbiased, solve the problems identified, uses data sets appropriately, and selects suitable algorithms. Without this type of validation, the GAO argues, AI system outputs may result in unintended consequences.

The GAO developed the framework based on AI-related literature, interviews by subject matter experts, and presentations and comments by a forum convened by the Comptroller General on AI oversight.

The framework depicts four central principles. Under each principle is a set of key practices for federal agencies to consider when implementing AI systems:

Source: Deltek, GAO-21-519SP

The report goes into further detail with descriptions of key practices as well as associated audit questions and procedures for each practice.

The framework targets an audience that includes federal agencies, state and local governments, industry, civil liberties advocacy groups, academia and research institutions.