NASA OIG: Final Memorandum on Audit of the Stratospheric Observatory for Infrared Astronomy (SOFIA) Program Management Effectiveness
National Aeronautics and Space Administration Office of Inspector General Washington, DC 20546-0001
March 27, 2009
TO: Associate Administrator, Science Mission Directorate
Assistant Administrator, Office of Infrastructure
Director, Ames Research Center
Director, Dryden Flight Research Center
FROM: Assistant Inspector General for Auditing
SUBJECT: Final Memorandum on Audit of the Stratospheric Observatory for Infrared Astronomy (SOFIA) Program Management Effectiveness (Report No. IG-09-013; Assignment No. A-08-011-00)
NASA’s Stratospheric Observatory for Infrared Astronomy (SOFIA), an airborne observatory within the airframe of a Boeing 747SP, will study the universe in the infrared spectrum. NASA began SOFIA Program formulation in 1991, initiated development in 1996, and the Program has since experienced cost overruns and schedule delays. Costs have exceeded 217 percent of the initial cost estimate and limited scientific operations are approximately 10 years behind the original schedule. As of January 2009, the SOFIA Program’s life-cycle cost estimates were approximately $1.1 billion for development and implementation and approximately $3.4 billion including a 20-year operational lifespan. We initiated this audit in light of the Program’s historical management issues and aircraft maintenance concerns.
The overall objective of this audit was to determine whether NASA was effectively managing the SOFIA Program to accomplish the near- and long-term objectives while meeting established milestones and controlling costs. Specifically, we determined whether SOFIA Program management had effectively planned for the long-term servicing of the aircraft, implemented effective controls to limit further cost growth and schedule delays, and monitored contractor performance. We also reviewed internal controls as they related to the objectives. See Enclosure 1 for details on the audit’s scope and methodology.
Executive Summary
SOFIA Program management had made significant progress in identifying and addressing past problems associated with management structure, schedule, and quality assurance. Program management had established adequate risk assessment and quality assurance processes to oversee contractor performance with respect to the accomplishment of near-term goals. However, we found that Program management had not yet completed actions required to address the long-term servicing needs of the aircraft, had not requested an independent cost estimate (ICE), and lacked an effective cost control process to evaluate the Program’s cost efficiency in meeting schedule milestones. As a result, Program management cannot accurately assess the effects of long-term aircraft servicing and maintenance on the Program’s life-cycle costs, demonstrate cost efficiencies, or provide earned value for completed contractor work.
SOFIA Program management had initiated some actions to address the long-term objectives related to maintaining and providing spare parts for the 30-year-old SOFIA aircraft. Program management drafted logistics and maintenance plans, and the Director, Dryden Flight Research Center, submitted a request to the Assistant Administrator for Infrastructure to obtain the two Shuttle Carrier Aircraft following the retirement of the Space Shuttle Program. However, as of January 30, 2009, Program management had neither finalized the logistics and maintenance plans nor obtained an agreement for the transfer of the Shuttle Carrier Aircraft. Settlement of these actions should provide SOFIA Program management the information needed to better estimate the cost for aircraft spare parts and maintenance requirements. Although the transfer of the Shuttle Carrier Aircraft could save the Program approximately $20 million in spare parts costs, the uncertainty over the exact retirement date of the Space Shuttle Program demands consideration and development of alternative spare parts plans. Similarly, completion of these plans should help in the development of a baseline from which management can monitor long-term cost and schedule performance.
SOFIA Program management did not have an effective cost control process because its monitoring process did not integrate cost with schedule. The Program’s process monitored cost and schedule separately, and the process was ineffective because it measured cost performance against a cost baseline that was unreliable as reflected by an ever-increasing cost trend. Further, there was no evidence of an independent cost estimate being performed to validate the Program’s cost estimates.
SOFIA Program management’s cost and schedule monitoring process did not function in accordance with American National Standards Institute/Electronic Industries Alliance – 748 (ANSI/EIA-748), “Standard for Earned Value Management Systems,” June 1998,1 as required by NASA Procedural Requirements (NPR) 7120.5D, “NASA Space Flight Program and Project Management Requirements,” March 6, 2007. The process also did not comply with the SOFIA Program Plan, July 10, 2007, which states that Earned Value Management (EVM) would be the primary tool employed to control and monitor cost performance of the SOFIA Program. Data for each work package in the process that Program management instituted pertain strictly to schedule and did not include cost estimates; therefore, Program management cannot quantify earned value and cost performance by work package.
SOFIA Program management did not obtain an ICE to validate SOFIA Program cost estimates–for total life-cycle cost or detailed work package cost estimates. An ICE can provide reasonable assurance that the cost baseline is reliable for use in an Earned Value Management System (EVMS) to measure cost performance. NPR 7120.5D requires that an ICE be performed following a program or project’s formulation phase. The SOFIA Program is in the implementation phase and will continue in this phase until reaching Full Operational Capability in 2014. An objective ICE improves reliability of program cost estimates by serving as an independent validation of them. The SOFIA Program has a history of increasing life-cycle cost estimates. An ICE would provide NASA management with a benchmark to assess the reasonableness of the Program’s cost estimates. Reliable cost estimates are also vital to the proper functioning of an EVMS.
The Program has not implemented EVMS nor conducted an ICE because SOFIA Program management has emphasized meeting near-term schedule milestones and has chosen not to expend resources in those other areas. As a result, Program management did not have an adequate EVMS to provide reliable earned value and early identification of performance trends and variances from the management plan.
We also found that cost control measures were lacking at the contract level. SOFIA Program management did not incorporate “cost control” as a performance evaluation factor in two cost-plus-award-fee (CPAF)2 contracts. The NASA supplement to the Federal Acquisition Regulation (FAR), NASA FAR Supplement (NFS) Version 04.0, November 1, 2004, requires that CPAF contracts include a cost control evaluation factor worth no less than 25 percent of the total weighted evaluation factors. The SOFIA Program’s contracting officer stated that cost control factors were included as sub-criteria of the Business Management evaluation factor. We evaluated those sub-criteria in the contracts’ Award Fee Determination Plans and NASA’s award fee cover letters to the contractors. Our evaluation of the cost control sub-criteria contained within the Business Management evaluation factor showed that the sub-criteria related to cost control did not add up to the 25 percent weight requirement. In addition, the award fee cover letters to the contractors did not address cost control performance, as required by NFS.
We concluded that SOFIA Program management deemphasized the importance of cost control, which negated its effectiveness, did not incentivize the contractors to control costs, and emphasized meeting near-term schedule milestones without adequate regard for controlling cost. Consequently, NASA expended $233,600 of Government funds for award fees that did not meet NFS requirements and, therefore, have been made without appropriate basis. Further, if Program management does not change the evaluation criteria in the Award Fee Determination Plans to meet the requirement that cost control accounts for no less than 25 percent of the total weighted evaluation factors, future award fees could potentially be awarded without the appropriate emphasis on cost control.
