Testimony of Thomas Young, Chairman of the Mars Program Independent Assessment Team before the House Science Committee

Introduction

Mars Climate Orbiter failed to achieve Mars orbit on September 23, 1999. On December 3, 1999, Mars Polar Lander and
two Deep Space 2 microprobes failed. As a result, the NASA Administrator established the Mars Program Independent
Assessment Team (MPIAT) with the following charter:

Review and Analyze Successes and Failures of Recent Mars and Deep Space Missions

• Mars Global Surveyor
  
• Mars Climate Orbiter
  
• Pathfinder
  
• Mars Polar Lander
  
• Deep Space 1
  
• Deep Space 2
  

Examine the Relationship Between and Among

• NASA Jet Propulsion Laboratory (JPL)
  
• California Institute of Technology (Caltech)
  
• NASA Headquarters
  
• Industry Partners
  

Assess Effectiveness of Involvement of Scientists

Identify Lessons Learned From Successes and Failures

Review Revised Mars Surveyor Program to Assure Lessons Learned Are Utilized

Oversee Mars Polar Lander and Deep Space 2 Failure Reviews

Complete by March 15, 2000

In-depth reviews were conducted at NASA Headquarters,
JPL, and Lockheed Martin Astronautics (LMA). Structured
reviews, informal sessions with numerous Mars Program
participants, and extensive debate and discussion within the
MPIAT establish the basis for this report. The review process
began on January 7, 2000, and concluded with a briefing to the
NASA Administrator on March 14, 2000.

This report represents the integrated views of the members of the MPIAT who are identified in the appendix. In total,
three related reports have been produced: this report, a more detailed report titled “Mars Program Independent Assessment
Team Report” (dated March 14, 2000), and the “Report on the Loss of the Mars Polar Lander and Deep Space 2 Missions” (dated
March 22, 2000).

Review and Analyze Successes and Failures of Recent Mars and Deep Space Missions

The Mars and deep space missions, reviewed and analyzed by this team, were implemented over a period of about 6 years
(1994–present). Mars Global Surveyor (MGS) was launched in 1996 and was the first Mars mission to employ some tenets of
Faster, Better, Cheaper (FBC). MGS is an extraordinary success and continues to be a highly productive science mission.

Mars Pathfinder was launched in 1996, landed on Mars on July 4, 1997, and captured the excitement of the public with
lander and rover operations on the Mars surface. It was the first complete Mars FBC mission and was an engineering, science,
and public success.

Deep Space 1 (DS-1) was a successful technology mission launched in 1998. It provided a space demonstration of
numerous new technologies, including ion propulsion and onboard autonomous operations. These technologies are now space
proven and available for future deep space missions.

Mars Climate Orbiter (MCO) was launched in late 1998, followed by Mars Polar Lander (MPL) and Deep Space 2
launched in early 1999. MCO failed to achieve Mars orbit because of a navigation error, resulting in the spacecraft entering the
Mars atmosphere instead of going into the planned orbit. The “Report on the Loss of the Mars Climate Orbiter Mission,” dated
November 11, 1999, and the “MCO Mishap Investigation Board Phase I Report,” dated November 10, 1999, provide details on the
failure cause and corrective action.

The following is a summary of the MCO findings. Spacecraft operating data needed for navigation were provided to the
JPL navigation team by prime contractor Lockheed Martin in English units rather than the specified metric units. This was the
direct cause of the failure. However, it is important to recognize that space missions are a “one strike and you are out” activity.
Thousands of functions can be correctly performed and one mistake can be mission catastrophic. Mistakes are prevented by
oversight, test, and independent analysis, which were deficient for MCO.

Specifically, software testing was inadequate. Equally important, the navigation team was understaffed, did not
understand the spacecraft, and was inadequately trained. Navigation anomalies (caused by the same units error) observed during
cruise from Earth to Mars were not adequately pursued to determine the cause, and the opportunity to do a final trajectory
correction maneuver was not utilized because of inadequate preparation.

MPL and the two Deep Space 2 microprobes were integrated on a common cruise stage for the trip from Earth to Mars.
Separation of the microprobes and the lander was planned to occur about 10 minutes prior to the planned Mars landings. The
design of the lander precluded any communications from the period shortly before separation from the cruise stage until after
Mars landing. The planned communications after landing did not occur, resulting in the determination that the MPL mission
had failed. Extensive reviews, analyses, and tests have been conducted to determine the most probable cause of the MPL failure.
This is documented in the “Report on the Loss of the Mars Polar Lander and Deep Space 2 Missions.” Several possible failure
causes are presented, which include loss of control due to spacecraft dynamic effects or fuel migration, local characteristics of the
landing site beyond the capabilities of the lander, and the parachute covering the lander after touchdown. Extensive tests have
demonstrated that the most probable cause of the failure is that spurious signals were generated when the lander legs were
deployed during descent. The spurious signals gave a false indication that the lander had landed, resulting in a premature
shutdown of the lander engines and the destruction of the lander when it crashed into the Mars surface.

Without any entry, descent, and landing telemetry data, there is no way to know whether the lander reached the terminal
descent propulsion phase. If it did successfully reach this phase, it is almost certain that premature engine shutdown occurred.

It is not uncommon for sensors involved with mechanical operations, such as the lander leg deployment, to produce
spurious signals. For MPL, there was no software requirement to clear spurious signals prior to using the sensor information to
determine that landing had occurred. During the test of the lander system, the sensors were incorrectly wired due to a design
error. As a result, the spurious signals were not identified by the systems test, and the systems test was not repeated with
properly wired touchdown sensors. While the most probable direct cause of the failure is premature engine shutdown, it is
important to note that the underlying cause is inadequate software design and systems test.

Deep Space 2 (DS-2) was a technology mission to demonstrate microprobe technology for future applications in exploring
various solid bodies in our solar system. The DS-2 design provided no data from the time it was integrated on the cruise stage at
the launch site until after the Mars landing; therefore, there is no knowledge of probe health following cruise stage integration.
No communications were received after the expected landings, resulting in the determination that the two DS-2 microprobes
failed. Reviews and analyses of the DS-2 development process have been performed and are documented in the earlier
referenced  “Report on the Loss of the Mars Polar Lander and Deep Space 2 Missions.”

DS-2 had an inadequate test program that deviated significantly from the proven practice of “test-as-you-fly,
fly-as-you-test.” No “most probable cause” has been identified for the DS-2 microprobes; however, it is clear that the
microprobes were not adequately tested and were not ready for launch.

The discussion on the previous pages summarizes the three successful and three unsuccessful Mars and deep space
missions reviewed and analyzed by the MPIAT. The important question is: “What are the lessons learned from these successes
and failures?”

There are common characteristics of the successful missions and of the unsuccessful missions. The following summarizes
the lessons learned from the MPIAT review of these missions.

Experienced project management or mentoring is essential.

Deep space missions are inherently difficult. These difficulties include long-duration operations, precision navigation,
hazardous environments, landing sites with unknown hazards at the scale of the lander, and, in many situations, the first use of
sophisticated hardware and software. Launch schedules typically have little flexibility. As an example, Mars launch opportunities
are approximately 1 month long and are separated by about 26 months.

The management challenges are enormous. MGS and Pathfinder had experienced project managers who contributed
significantly to their successes. DS-1 had a competent, but inexperienced, project manager who was augmented by senior JPL
management. MCO, MPL, and DS-2 had competent, but inexperienced, project managers. The lack of senior management
involvement to compensate for the lack of experience contributed to the MCO, MPL, and DS-2 failures.

The number of JPL projects has increased significantly. There are not enough experienced managers for the large
number of projects for which JPL is currently responsible. This situation requires significant involvement by senior
management to compensate for the lack of experience.

Project manager must be responsible and accountable for all aspects of mission success.

For MGS, Pathfinder, DS-1, and DS-2, the project managers were responsible for all aspects of their projects, from project
formulation through completion of mission operations. The MCO and MPL project manager was responsible for development
only, with a separate organization and project manager responsible for operations after launch. This arrangement contributed to
the MCO failure.

Unique constraints of deep space missions demand adequate margins.

Deep space missions are characterized by a fixed launch date (which fixes the schedule), a given launch vehicle (which
fixes the available weight), competitively selected science payloads (which establish the performance requirements), and for the
missions that were analyzed, fixed cost. When these four constraining parameters are fixed, there are only two remaining
variables—margins and risk. If adequate margins are available, risk can be effectively managed; if not, risk will grow to an
unacceptable level.

MGS and Pathfinder had adequate margins, and the risks were effectively managed, contributing to successful missions.
The technology mission, DS-1, did not have adequate margins; however, relief was provided because this was not a science-driven
planetary mission with a fixed launch opportunity. DS-1 performance requirements were effectively descoped, and the launch
schedule was delayed several months. Without this performance and schedule flexibility, DS-1 would have had excessive risk.

MCO, MPL, and DS-2 did not have adequate margins. MCO and MPL were managed as a single Mars ’98 project. The
project was significantly underfunded from the start for the established performance requirements. By comparison, MGS was a
single orbiter with science instruments and several subsystems developed for an earlier mission (Mars Observer). The
development cost plus the estimated value of the inheritance was approximately $250 million. Pathfinder is the standard for a
Mars FBC mission. Development cost for Pathfinder was about $200 million, including $25 million for the rover. Mars ’98, which
included an orbiter, a lander, and about three times as much science as Pathfinder, cost about $190 million. All costs are
constant-year 1999 dollars to allow for a direct comparison.

Mars ’98 (which included both MCO and MPL) cost approximately the same as Pathfinder. This clearly indicates the
significant lack of sufficient budget for Mars ’98. It was underfunded by at least 30 percent. There were many reasons for the
underfunding, including an aggressive proposal from LMA.

The selection of a launch vehicle with little margin, some growth in the science payload , and the fixed planetary launch
window, alsol contributed to inadequate margins. The result was analysis and testing deficiencies as well as inadequate
preparations for mission operations. These resulted in excessive risk and contributed to the failures. This is illustrated for Mars
’98 in the following figure.

  [FIGURES NOT PROVIDED]

Appropriate application of institutional expertise is critical for mission success.

For more than four decades, significant investments have been made in developing the deep space capabilities at JPL. As a result,
JPL is a center of excellence for deep space exploration. A primary reason for doing deep space missions at JPL is to take
advantage of this unique capability. This expertise was effectively used for MGS, Pathfinder, and (to some degree) DS-1,
resulting in a significant contribution to the success of these missions. Use of the JPL capabilities was significantly curtailed on
Mars ’98 largely because of funding limitations. Consequently, a significant opportunity was missed that may have resulted in
recognition of inadequate margins and excessive risk in the Mars ’98 project. JPL institutional support for DS-2 varied
considerably, but was inadequate for the technical complexity of the microprobes.

National capabilities can also contribute to the success of deep space missions. As an example, the atmospheric entry expertise at
NASA’s Langley Research Center, and Ames Research Center Center, and LMA was the primary source of this capability for
Pathfinder. The air bag technology for Pathfinder came from Sandia National Laboratory. Industry, academia, NASA Centers,
and other Government organizations were also important participants in DS-1.

A thorough test and verification program is essential for mission success.

FBC encourages taking prudent risk in utilizing new technology and pursuing important science objectives and innovation.
However, risk associated with deviating from sound principles should not be allowed. Sound principles include:

Efficient, competent, independent reviews

• Oversight, analysis, and test to “eliminate” a single human mistake from causing mission failure
  
• Clear definition of responsibilities and authority
  
• Prudent use of redundancy
  
• Test-as-you-fly, fly-as-you-test
  
• Risk assessment and management
  

This is not an exhaustive list, but rather important examples.

MGS and Pathfinder rigorously followed sound principles. DS-1 execution was mixed. DS-2 deviated to such a degree that it
leads to the conclusion that the microprobes were not ready to launch. Mars ’98 did the best that could be done with the limited
resources, but deviated significantly in analysis, testing, and the conduct of reviews.

Effective Risk risk Identification identification and Management management is are Critical critical to Assure assure Successful
successful Deep deep Space space Missionsmissions.

Risk is inherent in deep space missions.  . Effective identification and management of risk is are a critical responsibility
responsibilities of project management and often determine whether a mission will be successful. This was clearly a problem in
the implementation of MCO, MPL, and DS-2.

Faster-Better-Cheaper Faster, Better, Cheaper encourages taking prudent risk where justified by the return.  . The MPIAT found
that the lack of an established definition of FBC and policies/procedures to guide implementation resulted in Project project
managers having different interpretations of what is prudent risk..  . Senior management needs to establish that risk associated
with new high-return return technology and innovation is acceptable and as is innovation and risk associated with pursuing
high-value science is acceptable.  . Risk associated with deviating from sound principles is unacceptable.  . Risk must be assessed
and accepted by all accountable parties, including senior management, program management, and project management. All
projects should utilize established risk management tools such as fault tree analysis and failure effects and criticality analysis.  .
NASA Headquarters currently has a Termination Review.  It is probably an appropriate time to change the name and function of
this review to risk assessment.

Institutional management must be accountable for policies and procedures that assure a high level of mission success.

Institutional management must assure project implementation consistent with required policies and procedures.

Senior management is responsible for and accountable to establishto establish standards for the conduct of deep space missions;
to assure ; to assure that these standards are being followed; to assure that ; to assure that adequate resources, including
institutional expertise, are available and used;; and to assure that to assure that projects are being implemented with prudent
risk. In the case of  Mars ’98, this did not happen at NASA Headquarters, JPL, or Lockheed Martin AstronauticsA. A clear
example is the absence of critical entry, descent, and landing descent telemetry on MPL.  . It was also absent for DS-2.

MGS and Pathfinder success can be directly attributed to the experienced project managers and their effective use of expertise
from numerous sources. JPL senior management contributed significantly to the success of DS-1.

Telemetry coverage of critical events is necessary for analysis and ability to incorporate information in follow-on projects.

The lack of communications (telemetry) to provide entry, descent, and landing data for MPL was a major mistake. Absence of
this information prevented an analysis of the performance of MPL and eliminated any ability to reflect knowledge gained from
MPL in future missions. It is a prime example of the Mars Program being treated as a collection of individual projects as opposed
to an integrated program.

The final observation that needs to be made is:

If not ready—do not launch.

Planetary launch opportunities are typically separated by periods of many months or years; Mars launch opportunities are
approximately every 26 months. Not being ready for a scheduled launch opportunity is serious, but not as serious as launching
proceeding without being ready. Senior management needs to to make it unambiguously clear that “if not ready—do not launch.”

Interfaces and Relationships

The MPIAT charter includes an examination of relationships among JPL, Caltech, NASA Headquarters, and Lockheed Martin.
An assessment of the effectiveness of the involvement of scientists was also required. Among the interfaces and relationships
reviewed, two significant areas of concern were identified: “The interface between NASA Headquarters and JPL” and “The
interface between JPL and Lockheed Martin.” 

The interface between NASA Headquarters and JPL was found to be highly ineffective. A simplified example of the
ineffectiveness of this interface is illustrated in the following figure comparing intended versus perceived communications.

[FIGURES NOT PROVIDED]

NASA Headquarters provided objectives, requirements, and constraints for the Mars Program and projects to JPL. They
appropriately considered this a Headquarters responsibility. JPL interpreted these objectives, requirements, and constraints as
launch vehicle, cost, schedule, and performance mandates. As an example, for Mars ’98, the JPL management perception was that
no cost increase was possible. The response from JPL was more one of of an advocacy for the program and presenting a positive
image to the customer (NASA Headquarters) than a rigorous risk assessment with appropriate concerns expressed. What NASA
Headquarters understood was JPL agreement with the objectives, requirements, and constraints. The result was an ineffective
interface that did not resolve issues or manage risk. This directly contributed to inadequate margins for Mars ’98, which in turn
contributed to the MCO and MPL failures. The lessons learned from an analysis of this relationship are:

Frank communication of objectives, requirements, constraints, and risk assessment throughout all phases of the program is critical to successful
program/project implementation.

Senior management must be receptive to communications of problems and risks.

Another aspect of the interface was the absence of a single Mars Program interface at NASA Headquarters responsible for all
requirements, including those from other NASA Headquarters’ NASA organizations. Absence of a single interface resulted in
multiple inputs to the JPL Mars Program that were in some instances conflicting and in general added to the confusion and poor
communications. The lesson learned is:

A dedicated single interface at NASA Headquarters for the Mars Program is essential. This individual should have responsibility for all
requirements (including human exploration) and funds. The position should report to the Associate Administrator for Space Science.

The day-to-day relationship between JPL and LMA was positive during the conduct of the Mars ’98 project. However, the
relationship was ineffective when it came to informing senior management about risk. Lockheed Martin senior management did
not formally identify risk or deviations from acceptable practice. The lesson learned is:

Contractor (Lockheed Martin) responsibilities (Lockheed Martin) must include formal notification to the customer (NASA/JPL) of project risk
and deviations from acceptable practice.

Mars Program Implementation

The final responsibility identified by the charter is a review of the Mars Program to assure that the identified lessons learned are
utilized.

JPL has historically been responsible for individual projects. With NASA delegating program management responsibilities from
NASA Headquarters to the NASA Centers in 1996, JPL was assigned this responsibility for the Mars Surveyor Program.

The MPIAT does not believe that this responsibilitythe Mars Program has been effectively managed, . It has been
resulting in the Mars Program being managed as a collection of individual projects rather than as an integrated framework in
which projects fit to accomplish more than the sum of individual projects.  . Not including entry, descent, and landing telemetry
is a prime example of this deficiency.

As a result of moving to the FBC concept, the number of flight projects at JPL has increased over a three 3-year period from a
historical average of one 1 to 4 to four in a given year to a current level of ten 10 to 15 to fifteen at the same time. This increase is an
enormous success fora result of the FBC approach, which has as its objective smaller spacecraft with more frequent missions.
This increase in the number of projects requires additional capable project managers. There has been a loss of experienced,
successful project managers through retirement. The net effect is to use competent, but inexperienced, managers for the increased
number of projects. An earlier lesson learned is the need for senior management involvement and mentoring to compensate for the
lack of experience.

Currently, all flight projects, the Mars Program, and numerous other instrument and program responsibilities are in one
organization at JPL. This results in an extraordinary workload and span of control for this organization.

The conclusion of the MPIAT is that the current organization at JPL is not appropriate to successfully manage the Mars Program
in combination with other commitments for the reasons discussed above. The following organizational changes would be
responsive to this concern:

Establish an integrated Mars program Program office Office at JPL reporting to the Center Laboratory Director.

Establish a new, independent organization at the Directorate level dedicated to implementing major flight projects.

Summary

Based on more thanupon three months of intensive review by the MPIAT, there are several general observations important to the
future Mars Program:

• Mars Exploration Is an Important National Goal That Should Continue.
  
• Deep Space Exploration Is Inherently Challenging. The Risks Are Manageable and Acceptable.
  
• NASA, JPL, and Industry Have the Required Capabilities to Implement a Successful Mars Exploration Program.
  
• JPL Is a Center of Excellence for Deep Space Exploration with unique Unique capabilitiesCapabilities.
  
• Faster-, Better-, Cheaper, Properly Applied, Is an Effective Concept for Guiding Program Implementation that Should Continue.
  
• Significant Flaws Were Identified in the Formulation and Execution of the Mars Program.
  
• All Identified Flaws Are Correctable in a Timely Manner to Allow a Comprehensive Mars Exploration Program to Successfully Continue.
  

Appendix A

MPIAT Membership

• A. Thomas Young, Chair, Lockheed Martin (Ret.)
  
• Joanne Maguire, TRW
  
• James Arnold, NASA Ames Research Center  
  
• Robert Pattishall, National Reconnaissance Office
  
• Thomas Brackey, Hughes Space and Communications
  
• Larry Laurence Soderblom, U.S. Geological Survey
  
• Michael Carr, U.S. Geological Survey
  
• Peter Staundhammer, TRW
  
• Douglas Dwoyer, NASA Langley Research Center
  
• Kathryn Thornton, University of Virginia
  
• Gen. (Ret.) Ronald Fogleman, U.S. Air Force (Ret.)
  
• Peter Wilhelm, Naval Research Laboratory
  
• Maj. Gen . (Ret.) Ralph Jacobson, U.S. Air Force (Ret.)
  
• Brian Williams, Massachusetts Institute of Technology
  
• Herbert Kottler, MIT, Lincoln Laboratory
  
• Maria Zuber, Massachusetts Institute of Technology
  
• Peter Lyman, Executive Secretary, Jet Propulsion Laboratory (Ret.)
  
• Kurt Lindstrom, NASA Headquarters
  

MPIAT Consultants

• John Casani, Jet Propulsion Laboratory
  
• Peter Norvig, NASA Ames Research Center
  
• Brantley Hanks, NASA Langley Research Center
  
• Robert Sackheim, NASA Marshall Space Flight Center
  
• Bruce Murray, California Institute of Technology
  
• Steven Zornetzer, NASA Ames Research Center