Aerospace Safety Advisory Panel (ASAP) 2001 Annual Report (Excerpt)

Excerpt from “The 2001 ASAP Annual Report” (Acrobat 2MB file)

Pivotal Issues

This section addresses issues that the Aerospace Safety Advisory Panel (ASAP) believes are currently
pivotal to the safety of NASA’s activities. Some of these issues have widespread applicability
and are therefore not amenable to classification by program area in Section III.

Others, even though clearly applicable to a particular program, are of such sufficient import
that the Panel has chosen to highlight them here.

A. Planning Horizon and Budgets

NASA and, in fact, the entire Country are undergoing significant change. The inauguration of
a new administration and the events of September 11 have shifted national priorities. In turn,
NASA’s control of its finances and need for realistic life cycle costing for major programs, such
as the Space Shuttle and International Space Station (ISS), have been emphasized.
The purview of the ASAP is safety. Inadequate budget levels can have a deleterious effect on
safety. Clearly, if an attempt is made to fly a highrisk system such as the Space Shuttle or ISS
with inadequate resources, risk will inevitably increase. Effective risk management for safety
balances capabilities with objectives. If an imbalance exists, either additional resources must
be acquired or objectives must be reduced.

The Panel has focused on the clear dichotomy between future Space Shuttle risk and the
required level of planning and investment to control that risk. The Panel believes that current
plans and budgets are not adequate. Last year’s Annual Report highlighted these issues. It
noted that efforts of NASA and its contractors were being primarily addressed to immediate
safety needs. Little effort was being expended on longterm safety. The Panel recommended
that NASA, the Administration, and Congress use a longer, more realistic planning horizon
when making decisions with respect to the Space Shuttle.

Since last year’s report was prepared, the longterm situation has deteriorated. The aforementioned
budget constraints have forced the Space Shuttle program to adopt an even shorter
planning horizon in order to continue flying safely. As a result, more items that should be
addressed now are being deferred. This adds to the backlog of restorations and improvements
required for continued safe and efficient operations. The Panel has significant concern with
this growing backlog because identified safety improvements are being delayed or eliminated.
NASA needs a safe and reliable humanrated space vehicle to reap the full benefits of the ISS.
The Panel believes that, with adequate planning and investment, the Space Shuttle can continue
to be that vehicle.

It is important to stress that the Panel believes that safety has not yet been compromised. NASA and its contractors maintain excellent safety practices and processes, as well as an
appropriate level of safety consciousness. This has contributed to significant flight achievements.
The defined requirements for operating at an acceptable level of risk are always met.
As the system ages, these requirements can often be achieved only through the innovative
efforts of an experienced workforce. As hardware wears out and veterans retire, this capability
will inevitably be diminished. Unless appropriate steps to reduce future risk and increase reliability
are taken expeditiously, NASA may be forced to choose between two unacceptable
options – operating at increased risk or grounding the fleet until timeconsuming improvements
can be made.

Safety is an intangible whose value is only fully appreciated in its absence. The boundary between safe and unsafe operations can seldom be quantitatively defined. Even the most
wellmeaning managers may not know when they cross it. Developing as much operating
margin as possible can help. But, as equipment and facilities age, and workforce experience
is lost, the likelihood that the boundary will be inadvertently breached increases. The best
way to prevent problems is to maintain and increase margin through proactive and constant
riskreduction efforts. This requires adequate funding.

Finding 1: The current and proposed budgets are not sufficient to improve or even maintain the safety risk level of operating the Space Shuttle and ISS. Needed restorations and improvements
cannot be accomplished under current budgets and spending priorities.

Recommendation 1: Make a comprehensive appraisal of the budget and spending needs for
the Space Shuttle and ISS based on, at a minimum, retaining the current level of safety risk.
This analysis should include a realistic assessment of workforce, flight systems, logistics, and
infrastructure to safely support the Space Shuttle for the full operational life of the ISS.

B. Upgrades

The Space Shuttle is not unique compared to an aging aerospace vehicle that still possesses substantial flight potential and has yet to be superseded by significant new technology. Any replacement for the Space Shuttle will likely take a decade or more to be designed, built, and
certified. Commercial airlines and the military have faced the same situation and have implemented
timely product improvement programs for older aircraft to provide many additional
years of safe, capable, and costeffective service.

The Space Shuttle program is not presently able to follow this proven approach. Responding to budgetary pressures has forced the program to eliminate or defer many already planned and engineered improvements. Some of these would directly reduce flight risk. Others would improve operability or the launch reliability of the system and are therefore related to safety. In addition to the obvious safety concern of loss of vehicle and crew, the Panel views anything
that might ground the Space Shuttle during the life of the ISS as an unacceptable increase in
safety risk due to the potential loss of the ISS and associated risk for people on the ground.

The Panel also believes it is not prudent to delay readytoinstall safety upgrades, thus continuing
to operate at a higher risk level than is necessary. When riskreduction efforts – such
as the advanced health monitoring for the Space Shuttle Main Engines, Phase II of the
Cockpit Avionics Upgrade, orbiter wire redundancy separation, and the orbiter radiator isolation
valve – are deferred, astronauts are exposed to higher levels of flight risk for more years
than necessary. These lost opportunities are not offset by any real life cycle cost savings.

The stock of some existing Space Shuttle components is not sufficient to support the program
until a replacement vehicle becomes available. Some of the upgrades, in addition to improving
safety, solve this shortfall by providing additional assets. If these upgrades are not going
to be implemented, the program must plan now for adequate quantities of long leadtime
components to sustain safe operations.

Finding 2: Some upgrades not only reduce risk but also ensure that NASA’s human space
flight vehicles have sufficient assets for their entire service lives.

Recommendation 2a: Make every attempt to retain upgrades that improve safety and reliability,
and provide sufficient assets to sustain human space flight programs.

Recommendation 2b: If upgrades are deferred or eliminated, analyze logistics needs for the
entire projected life of the Space Shuttle and ISS, and adopt a realistic program for acquiring
and supporting sufficient numbers of suitable components.

C. Infrastructure

The Panel recognizes that safe Space Shuttle operations require a fully capable ground infrastructure,
including facilities, ground support equipment (GSE), training devices, and test
and checkout gear. These assets, like the vehicle itself, are aging. Much maintenance and
improvement of this infrastructure has already been deferred to conserve resources for operations.
As a result, there is a large backlog of restoration and upgrade work. Unfortunately,
rather than improving, the situation becomes worse each year. If restoration continues to be
delayed, it will reach a point at which it may be impossible to recover.

Infrastructure becomes increasingly unreliable as it ages. At best, this will be a costly nuisance
prompting delays and the need for expedited repairs. At worst, safety can be compromised if
systems fail at inopportune times or multiple, simultaneous failures occur.

NASA has initiated an Infrastructure Revitalization Team to plan the activities needed
through 2012, which is a necessary step in addressing infrastructure problems, but there is not
sufficient funding committed to reduce the backlog of work that needs to be done. It may
seem expedient to defer infrastructure maintenance and upgrades as long as the existing assets
can be made to perform. While this approach can accommodate immediate needs, it has two
major shortcomings: it creates a backlog of work that may never get accomplished; and it only
meets shortterm program needs. Neither the program’s full life cycle requirements nor the
needs of successor efforts are met. Infrastructure upgrades can be a valuable legacy to any vehicle
that follows the Space Shuttle.

Finding 3: Much of the Space Shuttle ground infrastructure has deteriorated and will not be
capable of supporting the Space Shuttle for its realistic service life.

Recommendation 3: Revitalize safetycritical infrastructure as expeditiously as possible.
The infrastructure for both the Space Shuttle and the ISS includes unique training and test
facilities. Though critical to the proper preparation of flight crews, some are not heavily
used – which makes their unit cost seem high. As a result, they have become candidates for
outright closure or for “mothballing.”During times of declining budgets, mothballing is tantamount
to closure. The Panel agrees that it is prudent to assess all facilities to determine if
they are adding significantly to the readiness level of the crews or the vehicle. In cases of duplication
or when an objective assessment indicates that a facility is no longer needed, closures
are appropriate. However, if a facility is necessary for crew readiness, it must be retained. Also,
it is essential to consider that unique workforce skills associated with the facilities to be closed
may be permanently lost.

Finding 4: NASA is considering closing or deactivating some training and test facilities in an
effort to economize.

Recommendation 4: Perform a detailed full life cycle safety and needs analysis including consideration
of critical skills retention before making closure decisions.

D. Space Shuttle Privatization

NASA is exploring the concept of privatizing the Space Shuttle by securing a contractor to
accept many of the responsibilities now held by the Government. It is premature to comment
on any specific plans. The Panel, however, is concerned that any plan to transition from the
current operational posture to one of privatization will inherently involve an upheaval with
increased risk in its wake. It must be remembered that the Space Shuttle program is over 20
years old and has already undergone several transitions that were distracting for the workforce.
If a new program were conceived and designed to operate in a privatized environment, there
is every reason to believe it could be successful. The salient issue is whether it is wise and beneficial
to transition the Space Shuttle program to privatization. Currently, there are significant
longterm safety issues that are best addressed by a fully engaged and highly experienced
workforce operating in a familiar environment.

Finally, one of the stated motivations for seeking privatization is the inability of the
Government to retain sufficient qualified staff given downsizing mandates. The Panel believes
it is in the best interest of safety to retain a core of highly qualified technical managers to oversee
complex programs such as the Space Shuttle. As long as NASA is going to be ultimately
accountable for safe operations, either directly or by indemnifying a contractor, it is necessary
to have the ability to make independent technical assessments. This system of checks and balances
between the Government and contractors has worked well. The challenge is to define
the appropriate levels of workforce and task sharing to achieve the desired benefits without
excessive costs.

Finding 5: Space Shuttle privatization can have safety implications as well as affecting costs.

Recommendation 5: Include in all privatization plans an assessment by safety professionals of
the ability of the approach to retain a reasonable level of NASA technical involvement and
independent checks and balances.

E. Workforce and Critical Skills

Workforce concerns continued to be a focus of the Panel during 2001. It is a tribute to the
Government and contractor management that the Panel has seen no safety shortfall attributable
to workforce or labor negotiation issues.
An Independent Assessment of Space Shuttle Ground Operations Processing Capability was
conducted by a team from the Headquarters Office of Safety and Mission Assurance (OSMA) 12 during the spring when there were four orbiters in flow at Kennedy Space Center (KSC). The
Panel reviewed the team report and concurs with OSMA’s conclusion that United Space
Alliance (USA) Ground Operations likely has the capability to safely accomplish a flight rate
of up to seven per year, if staffing remains at present levels and if flights are not unreasonably
clustered.

One of the current workforce challenges is the announced relocation of the sustaining engineering
functions currently maintained by the Boeing Reusable Space Systems (BRSS) operation
in Huntington Beach, California, to the Johnson Space Center (JSC) and KSC areas.
The move has the potential to impact functions that are critical to the safe operation of the
Space Shuttle, particularly since the expected number of experienced people who have actually
agreed to move is well below expectations. The Panel has confirmed that BRSS and NASA
managers are acutely aware of the safety sensitivity of their planned actions. Continuous oversight
and unswerving vigilance by Government and contractor management will be required
to safely accomplish this realignment. The Panel will continue to monitor progress as the
move unfolds.

As the Space Shuttle ages, it will require innovative technical and management actions
(ÒbandaidsÓ) to continue flying safely. Many of the most experienced NASA and contractor
personnel are at or nearing retirement age. The eventual departure of these individuals will
deprive the program of some of the highly skilled and experienced professionals needed to formulate
and execute these Òbandaids.Ó It will therefore become increasingly difficult to know
if adequate safety margins are being maintained.

The Panel believes that three major actions are needed now to compensate for the expected
attrition of significant portions of the Government and contractor talent pool responsible for
safe Space Shuttle operations. First, both NASA and its contractors should begin vigorous hiring
and training programs as soon as possible so new people will be available to work together
with the prospective retirees before they leave. Second, engineering drawings and processing
work paper should be updated by the experienced workforce to assure that drawings and specifications
reflect their latest experience (see Section III, A). Third, the upgrade program discussed
earlier and a meaningful lifeextension effort should be accomplished by the
experienced workforce before they retire. It will be much more effective and efficient to task
these individuals to do this work now than to have a less experienced workforce perform it in
the future. If modeled after successful commercial and military aircraft lifeextension programs,
such efforts will reduce safety risk and simplify the tasks facing future generations of
the Space Shuttle workforce. This approach will reduce the increasing reliance of the Space
Shuttle on workforce experience to maintain safety.

Unfortunately, some recent ideas for achieving lower cost Space Shuttle operations could
result in a reduction of NASA and contractor workforces. For example, significant cost savings
from reducing from six to four flights per year will only come from staff reductions,
which could exacerbate critical skills problems and disrupt the workforce.

Finding 6: The safety of NASA’s human space flight programs will always be dependent on
the availability of a skilled, experienced, and motivated workforce.

Recommendation 6: Accelerate efforts to ensure the availability of critical skills and to utilize
and capture the experience of the current workforce.

F. Mishap Investigation

NASA has an extensive and largely effective approach to mishap investigation. First, the severity
of the event is assessed against predetermined criteria. For example, a Class A mishap is
one involving death or injury or damage equal to or in excess of $1 million. Second, a mishap
investigation process is prescribed as a function of the severity classification of the incident.
The Panel typically examines the processes used in NASA mishap investigations and the
resulting reports. The analysis of several of the mishaps investigated during this year led to
ideas to strengthen the process.

Currently, severity classification is a function of actual losses. For example, an accident resulting
in $1 million in damage would necessitate a detailed investigation even if that dollar loss
were the most severe possible outcome. That is fully appropriate. On the other hand, a mishap
resulting in small economic loss but having potential for significant loss of life or assets would
not necessarily result in an investigation at the highest level. NASA managers do have the prerogative
to elevate an investigation to whatever level they deem appropriate, but this is seldom
done as they are not required to do so.

It would not significantly increase the workload or cost associated with mishap investigation
if all mishaps were prescreened by a panel of independent specialists, including the skills of
accident investigation, human factors, and industrial safety. Under this approach, such a
panel would review each mishap shortly after it occurred. This group would be chartered
only to determine if the preset severity criteria were appropriate for structuring a meaningful
investigation. If not, they would have the power to increase, but not reduce, the severity
class of the event.

Finding 7: Mishaps involving NASA assets are typically classified only by the actual dollar
losses or injury severity caused by the event.

Recommendation 7: Consider implementing a system in which all mishaps, regardless of
actual loss or injury, are assessed by a standing panel of independent accident investigation
specialists. The panel would have the authority to elevate the classification level of any mishap
based on its potential for harm.

A second issue with NASA mishap investigations concerns the membership of the Mishap
Investigation Boards (MIBs). In general, cognizant NASA managers populate an MIB with
technical specialists in the discipline related to the accident. This is fully appropriate to provide
subject matter expertise to the board. Mishap investigation is, however, a discipline of its
own. Many NASA mishaps also involve complex humanmachine systems. It would therefore
appear appropriate to require that all MIBs (or at least those for Class A and B events) include
specific expertise in mishap investigation and human factors. These disciplines are often key
to determining true root causes and deriving useful lessons learned. The participating specialists
need not be expert in the specific technical area, as they will draw that information
from other experts on the board. It is also helpful to have experts (NASA employees or outsiders)
independent of the investigated effort participate in mishap boards because they provide
an important additional perspective.

Finding 8: There is no requirement for MIBs to include individuals specifically trained in accident investigation and human factors.

Recommendation 8: Adopt a requirement for the inclusion of accident investigation and human factors expertise on MIBs.

G. Security

NASA has always been sensitive to the security of its personnel, facilities, and computing systems.
In the aftermath of September 11, NASA, like many other agencies, has expanded security
activities and broadened its efforts to consider nontraditional threats. The Panel has not
yet had the opportunity to examine the security posture of all of NASA’s Centers and facilities.
Several Panel members, however, did assess the security efforts at KSC in preparation for
the STS-108 launch. The Panel’s computer team has also maintained an ongoing look at
NASA’s information technology security.

Based on these preliminary activities, the Panel believes that the ongoing processes used to arrive at security decisions are sound and capable of adapting to changes in the threat environment
and/or available security capabilities. Interactions between NASA and other relevant
Government organizations appear uniformly good and supportive of an integrated
security activity.