Agencies Most Adept at Implementing GAO Recommendations for IT
Published: February 25, 2015
In the wake of GAO’s new high risk report which added IT acquisition to its ranks, Deloitte’s Advanced Analytics and Modeling (AAM) practice released a study of the effectiveness of GAO’s recommendations over time which showed that agencies are most adept at implementing GAO recommendations for information technology and IT security.
The report, entitled “Accountability Quantified: What 26 years of GAO reports can teach us about government management,” set out to determine if GAO recommendations were an effective way to drive targeted change within agencies. Additionally, Deloitte wanted to “use GAO as an example of how agencies can better structure their internal oversight activities to quantify accountability and drive results.” For the report, Deloitte analyzed 1.3 million pages of GAO reports using text analytics and analyzed the 40,000-plus recommendations made by GAO from 1983 through 2014.
Summaries of their findings for seven key questions are listed below:
The study found that agencies show the highest rate of success at implementing GAO recommendations in the areas of information security, information technology, education, and equal opportunity. Information security logged a 94% completion rate and information technology an 87% completion rate. These results may seem counter intuitive due to the bad press and scrutiny that federal IT programs have received in recent months. However, GAO’s recommendations are often tactical and not large-scale enterprise solutions or system changes, making it easier for agencies to comply.
Unfortunately, repeated recommendations to an agency in the same area do not improve an agency’s success rate. The study found, “There is no meaningful relationship between how many recommendations an agency receives in a specific area and how often they succeed in that area.”
Additionally, it’s worth noting that adoption of recommendations was studied over nearly a 30 year period. So, although agencies showed that they had implemented a high number of IT recommendations, this took place over an extended period of time and repeated prodding by GAO did little to hasten fixes.
Deloitte offers that because of GAO’s high success rate in the information technology space, it may have room to increase the number and strength of the specific recommendations it gives around IT security issues. On a broader scale, Deloitte recommends that GAO and agencies apply more standardization to their oversight data in order to analyze and interpret it more easily. Specifically Deloitte recommends the following next steps:
- Keep score by tracking where their recommendations are succeeding or failing.
- Convert reports to a text analytics-friendly electronic format.
- Establish a coding structure for reports.
- Uncover hidden trends. Develop a standard taxonomy for oversight reporting terms.
- Develop real-time accountability scorecards—and make them public.