NASA OIG: Review of NASA’s Space Technology Mission Directorate Portfolio
WHY WE PERFORMED THIS AUDIT
With a fiscal year (FY) 2022 budget of $1.1 billion and approximately 2,500 projects under its management, NASA’s Space Technology Mission Directorate (STMD) oversees a portfolio of space technology investments that support the Agency as it seeks to sustain and extend human activities in space, explore the solar system, search for evidence of past and present life, expand the commercial space economy, and ensure American global leadership in space technology.
In this audit, we examined the extent to which NASA’s management of its STMD portfolio aligns to space technology needs and whether performance measures and outcomes reflect the Directorate’s goals. To complete this work, we interviewed STMD officials to understand the processes that guide technology investment decisions and how the Directorate develops and reports on its performance goals. We also reviewed NASA’s strategic and performance plans and reports as well as STMD’s portfolio plan, strategic framework, and data from the STMD Portfolio Analysis Resource (SPAR) database.
WHAT WE FOUND
In 2020, STMD began the Strategic Technology Architecture Roundtable (STAR) initiative to prioritize technology gaps within the Directorate’s portfolio. This process uses information from stakeholders to identify priority technology gaps that should be addressed in the near term. The first iteration of the STAR process took 2 years and culminated with the 2022 issuance of reports describing how STMD’s portfolio contributes to achieving 17 desired outcomes to close technology gaps related to commercial competitiveness as well as Moon, Mars, and future science missions.
We found the stakeholders we interviewed were generally satisfied with the STAR process because it sufficiently integrated their feedback on priority technology needs. While the STAR process establishes the framework to integrate stakeholder needs and prioritize investment decisions, STMD does not have a reliable way to measure the cost of its efforts to close the gaps. STMD’s SPAR system is designed to offer insights into the Directorate’s portfolio; however, it does not include accurate and complete project costs due to infrequent updates, use of cost estimates, and exclusion of costs deemed sensitive procurement information. Furthermore, we found STMD is still working to map its investments to the 17 desired outcomes established through the STAR process, with only 20 percent of projects active in FY 2021 mapped to 1 or more desired outcomes. Incomplete cost information and mapping of projects limits the visibility STMD officials have over an already complex portfolio. Annually, STMD selects hundreds of new projects from thousands of proposals, and in FY 2022 STMD canceled several projects to fund higher priority needs. Without a complete picture of how STMD’s projects contribute to desired outcomes, NASA and other stakeholders do not have the information necessary to inform these strategic investment and divestment decisions.
As part of its performance management process, NASA develops strategic objectives and performance goals that allow the Agency to measure and track progress towards achieving its goals. We found that in FYs 2020 and 2021, STMD achieved satisfactory performance against its strategic objective—to innovate and advance transformational space technologies—and for some goals it significantly exceeded its targets. While STMD’s performance goals historically focused on a variety of quantifiable outputs that demonstrate effective management of project cost, schedule, and performance, these measures did not fully capture the extent to which STMD-funded technologies help achieve the Directorate’s strategic objective. Acknowledging this shortfall, in FY 2022 STMD altered several performance goals to focus on outcome-based measures. For example, STMD removed the output-based measure of investing in a target number of projects and replaced it with new outcome-based measures such as achievement of knowledge transitions and external partner investments. However, outcome-based performance measures can be difficult to quantify, especially for early-stage projects with future potential but little immediate application. Consequently, STMD will need to continue its efforts to develop outcome-based measures where appropriate and communicate to stakeholders the importance of programs best measured by these difficult-to-quantify measures.
WHAT WE RECOMMENDED
To ensure that STMD has visibility into how its portfolio aligns to priority outcomes to inform future investment and divestment decisions, we recommended the Associate Administrator for STMD: (1) reexamine its SPAR data system to ensure it provides as accurate and complete a picture of project costs as is practicable and (2) update its STARPort data system with complete information on project alignment to STAR desired outcomes for all projects active in FY 2021 and beyond. Additionally, to better illustrate whether STMD-funded technologies help achieve the Directorate’s strategic goals, we recommended the Associate Administrator for STMD: (3) complete efforts to develop additional outcomebased performance measures based on the transition, advancement, and infusion of technologies. We provided a draft of this report to NASA management who concurred with our recommendations and described planned actions to address them. We consider the proposed actions responsive and will close the recommendations upon completion and verification.
