GAO Takes Aim at How Effectively Agencies Estimate IT Program Costs
The end of the federal fiscal year (FY) for 2012 and Election Day are fast approaching and federal spending is at the forefront of public awareness. Congress continues to grapple with FY 2013 appropriations and the looming dangers of potential budget sequestration come January. With yearly federal spending on information technology (IT) in the $80 billion range, many legislators want to know that our federal IT dollar is being managed effectively. And a recent Government Accountability Office (GAO) report suggests that federal IT planners could do better at effectively estimating the cost of major IT programs.
The end of the federal fiscal year (FY) for 2012 and Election Day are fast approaching and federal spending is at the forefront of public awareness. Congress continues to grapple with FY 2013 appropriations and the looming dangers of potential budget sequestration come January. With yearly federal spending on information technology (IT) in the $80 billion range, many legislators want to know that our federal IT dollar is being managed effectively. And a recent Government Accountability Office (GAO) report suggests that federal IT planners could do better at effectively estimating the cost of major IT programs.
The GAO analysis is a response to a request by Senator Susan M. Collins, the Ranking Member on the Senate Homeland Security and Governmental Affairs Committee, to evaluate the implementation of cost-estimating processes at selected federal government departments and agencies – Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Labor, and Veterans Affairs, and the Environmental Protection Agency. GAO set out to determine which of these had effectively implemented cost-estimating policies and procedures and to determine if the cost estimates of certain related IT investments are reliable enough to support budget and program decisions. They found that there was significant variance in how these departments’ and agencies’ cost-estimating policies and procedures fit with GAO’s 2009 best practices and most of them had significant weaknesses which GAO linked to organizations not making agency-level cost-estimating functions a high enough priority.
GAO defines four characteristics for a reliable cost estimate:
- Comprehensive – Completely defines the program in detail, including the current schedule, and include both government and contractor costs over its full life cycle, from inception to retirement.
- Well-documented – Supported by detailed documentation describing the method and data used to derive the cost and how the expected funding will be spent to meet the program objective.
- Accurate – Error-free, unbiased and not overly conservative or optimistic, adjusted for inflation and updated with program changes, and based on the most likely costs in light of the historical experience on comparable programs.
- Credible – Provides a level of confidence by describing any limitations of the cost analysis due to assumptions, uncertainty or biases surrounding the data as well as providing analyses around risk and sensitivity to changes in assumptions.
Room for Improvement
After looking at the selected federal government departments and agencies, GAO determined that nearly all the programs they assessed did not fully meet their defined characteristics for a reliable cost estimate, meaning that they did not fully reflect all four of the above characteristics. (See table below.) As a result, all but 1 of the 16 major acquisition programs assessed was using cost estimates that GAO deemed as “not fully reliable.”
|
Assessment of Cost-Estimating Practices for Case Study Programs
|
|||||||||||
|
Program
|
Comprehensive
|
Well-documented
|
Accurate
|
Credible
|
||||||
|
Agriculture
|
Public Health Information System
|
◐
|
○
|
○
|
○
|
||||||
|
Web-Based Supply Chain Management
|
○
|
○
|
○
|
○
|
|||||||
|
Commerce
|
Comprehensive Large Array-data Stewardship System
|
◐
|
○
|
○
|
○
|
||||||
|
Patents End-to-End: Software Engineering
|
◐
|
◐
|
◐
|
○
|
|||||||
|
Defense
|
Tactical Mission Command
|
◐
|
◐
|
◐
|
○
|
||||||
|
Consolidated Afloat Networks and Enterprise Services
|
●
|
●
|
●
|
●
|
|||||||
|
Environmental Protection Agency
|
Financial System Modernization Project
|
◐
|
◐
|
◐
|
○
|
||||||
|
Superfund Enterprise Management System
|
○
|
○
|
○
|
◐
|
|||||||
|
Homeland Security
|
Integrated Public Alert and Warning System
|
◐
|
◐
|
●
|
◐
|
||||||
|
Rescue 21
|
◐
|
◐
|
◐
|
◐
|
|||||||
|
Justice
|
Unified Financial Management System
|
◐
|
◐
|
◐
|
○
|
||||||
|
Next Generation Combined DNA Index System
|
◐
|
◐
|
◐
|
◐
|
|||||||
|
Labor
|
OSHA Information System
|
◐
|
◐
|
◐
|
○
|
||||||
|
PBGC Benefit Administration
|
◐
|
◐
|
◐
|
◐
|
|||||||
|
Veterans Affairs
|
Health Data Repository
|
○
|
○
|
○
|
○
|
||||||
|
Veterans Benefits Management System
|
◐
|
◐
|
○
|
○
|
|||||||
|
Key
●=Fully met—the program provided evidence that it fully implemented the cost-estimating practices for this characteristic.
◐=Partially met—the program provided evidence that it implemented about half or a large portion of the cost-estimating practices for this characteristic.
○=Not met—the program did not provide evidence that it implemented the practices or provided evidence that it only minimally implemented the cost-estimating practices for this characteristic.
|
|||||||||||
|
Source: GAO
|
|||||||||||
Implications and Opportunities
Program cost overruns continue to be a major area of scrutiny for large federal IT programs. GAO found that most of the eight agencies they assessed lacked training requirements in cost-estimation as well as central, independent cost-estimating teams. If agencies take GAO’s assessment seriously and seek to update their policies to conform to improve cost estimation then this could produce some short-term opportunities in the areas of training, policy development, and strategic alignment, as well as potential staff augmentation contracts with expertise in cost IT analysis and management.
Until agencies address weaknesses in their cost estimation policies and practices so that these figures are dependable for making decisions they will continue to struggle to make effective IT plans and efficiently manage their IT budgets. And failure in this area will continue to bring scrutiny from OMB and Congress.
There are no comments for this entry.
[Add Comment]