NASA OIG: NASA Should Reconsider the Award Evaluation Process and Contract Type for the Operation of the Jet Propulsion Laboratory (Redacted)
The Issue
The Jet Propulsion Laboratory (JPL) is a NASA federally funded research and development center (FFRDC) operated under contract by the California Institute of Technology (Caltech), a private nonprofit educational institution. The NASA Management Office (NMO) at JPL oversees the contract. Since 1993, NASA has awarded three cost-plus-award-fee (CPAF) contracts to Caltech for the operation of JPL. Work under these contracts has included accomplishments in astrophysics, earth sciences, solar system exploration, and technology. Examples since 2003 include “Spirit” and “Opportunity,” the Mars Exploration Rovers; the Spitzer Space Telescope; the Microwave Limb Sounder; and the Tropospheric Emission Spectrometer.
The current 5-year CPAF contract, awarded to Caltech in November 2002, is valued at approximately $7.5 billion 2–$1.5 billion per year, including available award fees of $22 million. The current contract also includes an award term incentive, allowing Caltech to earn term extensions on the contract in increments of 3 or 9 months for up to an additional 5 years. The potential total value of these incentives is $7.5 billion. Award fee and award term determinations for the JPL contract are made by the NASA Associate Administrator based on annual evaluations of the contractor’s performance against criteria established in the NASA “Award Fee Contracting Guide,” June 27, 2001.
We initiated this audit to determine whether NASA appropriately calculated and justified the award fees and term extensions earned by Caltech for the operation of JPL. (See Appendix A for details of the audit’s scope and methodology.)
Results
NASA can improve its management of the JPL contract:
- NASA’s overall assessment of contractor performance may have been overstated because the Agency’s performance evaluations for fiscal year (FY) 2007 were incomplete or did not otherwise comply with its guidance.
- The contractor’s poor performance on a large, significant project was offset in NASA’s assessment by higher performance on smaller projects because the Agency did not use proportional weighting in its evaluations.
- NASA’s award of $16 million in fees and 27 months of contract term extensions, value at $3.375 billion, were unsupported because the Agency’s performance evaluation factors did not include an assessment of required cost control measures.
- NASA does not have assurance that the existing contract still meets its needs or provides the best value for the taxpayer because the Agency did not fully comply with Federal Acquisition Regulation (FAR) requirements for a 5-year comprehensive review of the use and need for the FFRDC.
- NASA’s use of a single CPAF contract for all aspects of the FFRDC creates a significant administrative and management burden for the Agency that is unnecessary given that there is a basis for the contracting officer to establish fair and reasonable prices for routine operations and maintenance of the facility.
The Assistant Administrator for Procurement and the NMO concurred with our recommendations to address these issues and agreed that there are potential monetary benefits associated with their implementation.
We found that performance evaluations and the method used to calculate the award fee score for Caltech’s FY 2007 performance may not accurately reflect the overall performance of the contractor (Finding A). For the period of performance ending September 30, 2007, this resulted in potentially higher ratings, which could have led to NASA inappropriately paying award fees and awarding a term extension. Although the criteria used for evaluating the contractor’s performance were sufficiently specific and measurable, NASA evaluators provided incomplete assessments or assigned inappropriate ratings. For example, the evaluation for one criterion stated that the contractor did not meet the Satisfactory or Excellent standards due to cost; however, the evaluator provided a rating of Good, stating that he did not want to penalize the contractor’s technical performance for poorer cost performance. In addition, there is no documentation to indicate that criteria were proportionately weighted by a project’s overall importance to NASA or by project cost, which allowed exceptional contractor performance on smaller projects to conceal poorer performance on a larger, more significant project.
In addition, the performance evaluation factors did not include cost control measures weighted at no less than 25 percent of the total weighted evaluation factors, as required by NASA guidance. We calculated that cost control measures were weighted between 5 percent and 14 percent of the total (Finding B). The low percentage weighting deemphasized the importance of controlling cost, minimized the effectiveness of cost control, and gave the contractor minimal incentive to control costs. As a result, a portion of Caltech’s award fees were unsupported costs and an ineffective use of Government funds. Specifically, we consider $16 million of the $97 million paid in award fees for Caltech’s performance under the 2003 contract to be unsupported. In addition, we question the 27 months in term extensions, valued at approximately $3.375 billion, awarded to Caltech.
We also found that NASA did not adequately consider alternative sources for the operation of JPL as an FFRDC and could not provide support to justify its decision to continue procuring the services of Caltech (Finding C). The FAR requires a comprehensive review of the use and need for FFRDCs every 5 years and provides five criteria for the review, which is designed to consider alternative sources to meet the sponsor’s needs and to ensure that the contractor is providing the best value for the taxpayer. However, the review NASA conducted in 2002 prior to awarding the most recent contract for operating its FFRDC, in 2003, was less than exhaustive in its search for viable competitors. A review conducted in 2008 addressed four of the five FAR criteria, but did not include consideration of alternative sources. NMO Procurement Office personnel decided to postpone the search for alternative sources until the current contract term, including extensions, was due to expire. Without a comprehensive review or assessment of viable competitors that could provide services for the operation of JPL, NASA cannot have assurance that it is obtaining the best value for the taxpayer. To be compliant with the FAR, NASA should conduct a comprehensive review when the contract is extended by an award term, which is, potentially, every year. However, NASA may be able to eliminate the completion of successive FFRDC reviews by amending the Performance Evaluation Plan to, in effect, escrow all award terms earned or lost during the base years of the contract and conducting a comprehensive use and need review at the end of the base term.
Lastly, we determined that a single CPAF contract is not the best contract vehicle to procure services from Caltech (Finding D). JPL had been managed using multiple cost-plus-fixed-fee (CPFF) contacts for more than 35 years, until NASA consolidated the two CPFF contracts to a single CPAF contract in 1993. We concluded that using a single CPAF contract for the operation of the entire FFRDC is difficult for NASA to effectively manage because the contractor is responsible for multiple, complex, and often unrelated deliverables (for example, products crucial to NASA’s missions in the areas of earth sciences and solar system exploration, as well as normal facility functions such as public affairs and maintenance of the grounds). In addition, a CPAF contract requires significant oversight and documentation to evaluate contractor performance, and NASA did not perform a cost-benefit analysis to ensure that the benefits of a CPAF contract adequately offset the additional costs associated with contract administration. NASA also did not adequately evaluate alternative contract vehicles to support JPL operations. As a result, NASA may not be getting the best value for the taxpayer.
Management Action
To ensure that evaluations appropriately reflect contractor performance, we recommended that NASA provide detailed, explicit direction to contract performance evaluators concerning their responsibilities, monitor the documentation provided by the evaluators for accuracy and completeness, and apply a weighting factor to each criterion or project based on significance and cost. In addition, the NMO Procurement Officer should (1) modify the “Performance Evaluation Plan for Management of the Jet Propulsion Laboratory” to clearly include cost control measures weighted at no less than 25 percent of the performance evaluation factors; (2) perform a comprehensive use and need review of the FFRDC, to include a proactive search for alternative sources; (3) consider amending the Performance Evaluation Plan to “escrow” earned award terms; (4) reevaluate the use of award terms for any future FFRDC contract; and (5) perform a cost-benefit analysis and evaluate alternative contract vehicles for any FFRDC follow-on contracts.
Comments from the Assistant Administrator for Procurement; the NMO Director, JPL; and the NMO Procurement Officer in response to a draft of this report concurred with our recommendations; in subsequent correspondence, the NMO agreed that there are potential monetary benefits to implementing the recommendations. The Assistant Administrator for Procurement, who signed the comments, stated that the NMO will propose additional language to clarify the performance evaluation process; ensure evaluation submissions are complete; request that annual criteria is weighted; capture cost control at no less than 25 percent of the award fee evaluation score; proactively consider alternative sources; reconsider the use of award term incentives in future JPL or FFRDC contracts; conduct a cost-benefit analysis if a CPAF contract is used again; and consider alternative contract vehicles for the next FFRDC contract. See Appendix D for the full text of management’s comments, including an appendix of “Technical Comments” providing clarifying details on certain issues.
We consider management’s proposed actions to be responsive for all recommendations. The recommendations are resolved and will be closed upon completion and verification of management’s corrective actions.
