Letter From IFPTE To Congress Regarding NASA Handling of NAOMS Data Release

Hon. Bart Gordon, Chair
House Science Committee
2320 Rayburn Building
Washington, DC 20515

Hon. Alan Mollohan, Chair
House CJS Approp. Subcommittee
H-310 The Capitol
Washington, DC 20515

Hon. Bill Nelson, Chair
Senate Space, Aero & Sciences Subcommittee
716 Hart Senate Building
Washington, DC 20510

Hon. Barbara Mikulski, Chair
Senate CJS Approp. Subcommittee
144 Dirksen Senate Building
Washington, DC 20510

Dear Chairpersons Gordon, Mollohan, Nelson, and Mikulski:

On December 31st 2007, NASA Administrator Michael Griffin released redacted NAOMS data to the public. That moment should have been the beginning of a redemptive process in which NASA could move past this embarrassing episode. Alas, NASA Administrator Michael Griffin decided not only to repeat the inaccurate derogatory claims made at the October 31st House Science Committee hearing, but also to add a number of new inaccuracies to the mix (see a summary of ongoing disinformation below). His words, actions, and bellicose public behavior have seriously damaged NASA’s credibility. While this week’s announced departure of NASA’s Associate Administrator for Aeronautics Lisa Porter is a step forward, this alone does not fully resolve the current debacle. Dr. Griffin must take further corrective action.

After the October 31st 2007 House Science Committee hearing, the International Federation of Professional and Technical Engineers (IFPTE) gave the Administrator the benefit of the doubt that the misstatements included his testimony, particularly those disparaging research performed by NASA employees, resulted from a biased and inaccurate briefing from Dr. Porter. At this point, however, it is clear that the Administrator himself has played an active role in spinning a web of falsehoods that have undermined the House Science Committee’s oversight of NASA’s Aeronautics programs. In addition to outright falsehoods, Dr. Griffin and Dr. Peggy Gilligan, the Deputy Associate Administrator for Aviation Safety at the Federal Aviation Administration (FAA), continue to express reckless, subjective “opinions” that they see no value in the NAOMS data or that the NAOMS data show ridiculous deviations from “reliable” FAA data, despite both admitting that they have only anecdotal knowledge of the results. Most outrageously, since October, Dr. Porter has been preventing NASA’s experts from completing a scientific analysis of the data and publishing the findings as every taxpayer would reasonably expect them to do.

The Administrator is entitled to his own opinions, but he is not entitled to his own facts. Policy makers should defer to experienced, apolitical, career technical experts for their facts and these experts must never fear retaliation for any political inconvenience generated from publishing their (unclassified) findings. That technical independence is one of the essential values of the civil service in our democratic society. In simple words, it is not unreasonable for Dr. Griffin to say that he wishes to terminate NAOMS because, with limited resources, he believes that it has a lower priority than other programs. However, it is entirely wrong for him to make false statements about the technical value of NAOMS or about the performance/motivation of the research team simply to shore up a misguided Administration agenda. NASA’s R&D — indeed all of the government’s R&D — must be protected from the vicissitudes of political spin or it will become fatally compromised and make informed policy decisions impossible.

To repair the damage to NASA’s reputation from the Administrator’s mishandling of the NAOMS matter, IFPTE asks that Congress clarify its direction in the FY2008 omnibus report and explicitly require that NASA’s NAOMS researchers be allowed to complete their analysis of the existing survey data and to publish their findings (as opposed to outsourcing this important work to an outside entity to examine redacted data). The value of NAOMS can then be properly evaluated by objective reviewers of NASA’s and/or Congress’ choosing.

The NAOMS debacle is not an isolated incident, but rather part of a pattern of covert decision making by the Bush Administration. Under the current Administration, NASA Aeronautics mission has lost more than half of its annual budget. NASA’s Aeronautics R&D will be subject to an additional $100+ million cut this year, so the public should not be surprised that promising new ideas (not just NAOMS) have been and will be sacrificed to meet this Administration’s R&D shrinkage quota (while Europe is wisely expanding its Aviation R&D funding). The disappearance of key aeronautics capabilities at NASA under Dr. Porter’s tenure was a deliberate policy; the abandonment of NAOMS is merely one symptom of a much more extensive and harmful reprogramming disease.

IFPTE is not alone in its concern about NASA’s mismanagement of its Aeronautics R&D programs; in his Congressional testimony on May 7th 2007, the Inspector General (IG) of the Department of Transportation (DOT) expressed serious warnings about NASA shirking its responsibility to the FAA and the Joint Development Planning Office (JDPO) in their efforts to develop the Next Generation Air Transportation System (NGATS or NextGen):

A key, short-term cost driver for NextGen is the role that NASA will play. Historically, FAA’s R&D efforts have focused on short-term research, with NASA conducting the majority of long-term air traffic management research…

FAA’s Research, Engineering, and Development Advisory Committee (REDAC) has raised concerns about NASA’s efforts to restructure its aeronautics program and its potential impact on NextGen. … FAA would have to assume a larger burden and the associated costs to complete development and bring new systems to fruition. To accommodate changes in NASA investments and to address this gap, the REDAC estimated that approximately $100 million would be needed annually.

If NASA is unable to provide projects at a level that FAA can transition to prototypes, the JPDO and FAA will have to determine how this R&D will be completed, managed, and paid for.

It is clear from the above testimony that NASA’s new Aeronautics program is failing to meet the nation’s civil aviation needs. The FAA is now throwing its funds at this problem, but that inefficient effort will not likely be able to build the requisite R&D capabilities needed to keep NextGen on schedule and budget. In particular, the DOT IG states that the “FAA needs to conduct sufficient human factors research to support anticipated NextGen changes.” This is clearly an activity that NASA researchers should be participating in more extensively than their current limited involvement, but they have been unable to because of Dr. Porter’s policies. Indeed, NAOMS was designed to monitor the trends of safety precursors as new NextGen changes come on-line in order to identify problems before they become accidents. It would now appear that the JPDO and the nation will not have the benefit of that powerful new capability.

The Administrator has repeatedly and correctly pointed out how safe the U.S. airspace currently is. NASA’s track record of significant contributions to aviation safety over many decades has been a major factor in that success. It is the safety of the nation’s future NextGen airspace that is now in jeopardy.

In closing, IFPTE would like to thank you and your Committees for your keen attention to NAOMS in particular and NASA in general. We are grateful that Congress recognizes the importance of NAOMS and has directed NASA to properly assess NAOMS. Dr. Porter’s departure was only a down payment on NASA management’s redemption. Given that the Administrator has managed to offend the public, the press, his employees, and thousands of dedicated pilots who volunteered to participate in NAOMS, additional corrective action is urgently needed to complete the rehabilitation process. To begin with, just as with his embarrassing statements on the value of data on Global Warming, Dr. Griffin should retract his equally embarrassing statements about value of the NAOMS data and, most importantly, stop interfering with the reporting of scientific findings by NASA researchers to the American people. We at IFPTE are comforted that Congress shares our commitment to NASA maintaining a vigorous, independent Aeronautics R&D program.

To set the record straight, IFPTE has included with this letter an itemized correction of the misstatements about NAOMS made by the NASA Administrator. Should you have any questions, I, or IFPTE Legislative Director Matt Biggs can be reached at (301)565-9016. Sincerely,

Gregory J. Junemann,
President

False Statement #1: NAOMS has never been peer-reviewed.

At the December 31st press conference, the Administrator stated that “(t)he actual responses contained in these surveys and the methodology used to acquire them … have not been peer-reviewed to date..” The public record of multiple reviews indicates that this assertion made at the October 31st hearing and repeated at the press conference is objectively false.

As IPFTE pointed out in our November 29th, 2007 letter to you, the NAOMS survey project was subject to multiple field trials, workshops, and several formal independent technical reviews, including one by the National Academy of Sciences (NAS) in 2004 that concluded that the new NAOMS approach should be combined with ongoing NASA Aviation Safety activities. Dr. Griffin’s reference to “actual responses” is strange in that raw data is not something one submits to peer review, rather one submits analyzed findings to peer review to validate the analysis and conclusions. That process was interrupted by NASA management (see point #2).

It is truly ironic that Administrator Griffin would repeat this documentably erroneous claim and then within minutes refer to the 2004 NAS NAOMS review. At the press conference, Dr. Griffin claimed that “when the National Academy of Sciences was asked to review the NAOMS project, they stated out right [sic] that they saw no need for it to continue.” (A reiteration of a nearly identical false claim made during his testimony to your Committee). The 2004 NAS report actually made the opposite recommendation that “NASA should combine the National Aviation Operations Monitoring Service methodology and resources with the Aviation Safety Reporting System program data to identify aviation safety trends.” This quote is clear evidence that the NAS has already peer-reviewed NOAMS and has provided its overall endorsement of the added value of its methods over existing approaches. It is undeniable that the Administrator has now repeatedly distorted the words and intent of the 2004 NAS review in an effort to mislead Congress and the American people.

The Administrator also made the claim both at your hearing and the press conference that “no product of the NAOMS project, including the survey methodology, the survey responses themselves, or any analysis of the responses should be viewed or considered at this stage as having been validated”. If indeed, as Dr. Griffin has declared, no NASA-funded Aerospace R&D project should mature into an ongoing operational activity unless it is fully validated through an external peer-review process that includes peer-reviewed publications, then why have the Exploration System Architecture Study and the resulting Orion/Ares spacecraft designs not been peer reviewed? Where are the peer-reviewed publications of the Orion/Aries design? Why weren’t the National Academy of Engineering and/or premier academic engineering departments recruited to perform an independent external technical review?

What is required of an $11 million goose should also hold for an $11+ billion gander. Clearly, the Administrator’s peer-review argument for denigrating NAOMS is both false and specious.

False Statement #2: NASA never intended to analyze the NAOMS data.

Dr. Griffin stated at the press conference that “NASA never had plans to analyze this raw data” and reiterated that “NASA did not and does not have any plans to … analyze it. … We don’t have any plans to fund these particular researchers to continue on, and we never did.” These statements are contradicted by NASA program documents.

An FY2005 NAOMS level 2 milestone (number 2.1/11) states: “Demonstrate Value of Pilot Survey: Results of surveys of carrier and GA pilots are statistically analyzed to identify operational issues.” This milestone is documented in the Aviation Safety and Security Program Plan, which was approved and signed off on 10/3/03 by the Program Manager, and it clearly plans for and requires “analysis” of the survey data. This milestone was also part of the approved presentation package for the 2004 NAS review, and some preliminary analyses were presented during the Independent Implementation Review in February of 2004. Prior to the FOIA refusal scandal, NASA researchers had already made considerable progress on this enormous task, but Dr. Porter ordered them to stop all analyses, leaving millions of dollars of taxpayer funded research orphaned. The NASA NAOMS research team stands ready to complete this important task, but they are being prevented from doing so by NASA management.

Associated with the false claim that NASA never planned to analyze the data, Dr. Griffin added the false converse claim that NASA “intended the data to be transitioned, again, to the larger safety community. We, in fact, extended the funding. The originally planned funding for this research was to end in 2004. We extended it for two years to 2006 and, in fact, as you can see, have extended the work even into 2007 in order to properly fund transition of the data and review of it by others.” The transition of the NAOMS raw data to others for analysis was never part of NAOMS’ project plan. The only transition associated with the NAOMS funding extension was to transfer NOAMS methodology to the AirLine Pilots Association (ALPA) and did not include any plan to transfer the existing NAOMS raw data to ALPA. Neither ALPA nor any other outside entity was ever slated to receive these data.

False Statement #3: NASA was never asked to do anything other than release the data.

At the press conference, Dr. Griffin stated “(t)hat is all that we have ever, in fact, been asked for was to release the data, and it is all that we have ever promised …” As you know, Congress has asked NASA to do much more than simply dump raw data on the press and, as stated above, NASA promised more than that during the 2004 NAS review.

In the report associated with the omnibus spending amendment of FY2008 passed by both Houses of Congress and signed into law by the President prior to the Press Conference, NASA was directed “to revalidate the NAOMS survey methodology and, if needed, to restart the NAOMS survey data collection activity as well as identify trends that may emerge.” The Congressional mandate to “identify trends that may emerge” requires that NASA perform the exact analysis of the data that NAOMS researchers had initiated. It will be a criminal waste of taxpayer funds and an arrogant abuse of executive power if Dr. Porter’s cease-and-desist order is not overridden. Relying on the NAS or others to perform a limited “assessment” of significantly redacted data is a poor substitute for a systematic analysis of the original unredacted dataset by the NASA aviation safety experts most familiar with the survey and its data.

NASA is now commissioning the NAS to re-review NAOMS methods (a task it already performed in 2004). Such a redundant effort would be a complete waste of the taxpayers’ money as the NAOMS data-collection methodology has been thoroughly validated through multiple field trials, workshops, and external/internal reviews. Instead, the NAS should be asked to peer-review an uncensored technical report by the NASA NAOMS team (provided within 6 months) that includes trend and other statistical analyses of the data, as well as a list of significant findings and conclusions. The value of the NAOMS research (over the past half-decade) for both the aviation community and the public could then be properly evaluated. Congress could then decide whether or not to direct NASA to continue NAOMS as originally planned, modify it, or terminate it as part of the FY2009 Appropriations process.

NAOMS scientists are not asking to make policy decisions, but rather to be allowed to complete their objective technical work to a level that delivers to policy makers the information they need to make sound decisions. This open functioning of an independent technical civil service is essential to democracy and good governance.

False Statement #4: NASA’s standard format for data release is PDF (portable document format)

At the press conference, the Administrator stated that “our standard format for data release is PDF format. … I am sure that you know that the reason why we use PDF format is that the data cannot then be altered by others without our knowledge and still claim that it is NASA data.” NASA however has no such policy. In response to a union query, NASA HQ responded that “First, the data to which Dr. Griffin refers pertains to data/information released by NASA Public Affairs (e.g., press releases), not scientific and technical information. Second, the use of the PDF format is a standard practice (not a policy) to protect the integrity of this data/information when released to the public.”

NASA’s Public Affairs Office has a non-binding “practice” of using PDF for Press Releases. The NAOMS data release was clearly not a “press release” but rather was a release of scientific/technical data required by Congressional mandate and by law (FOIA). The use of PDF was a political decision made at Headquarters overriding the clear consensus of the technical team assembled to perform the de-identification that recommended that the data be released in a format (e.g., Excel spreadsheet) that would allow for uploading into numerical/statistical analysis software packages. The argument that PDF prevents alteration of the file is silly as any digital file can be altered after NASA has released it. The PDF decision was clearly designed to needlessly create obstacles to those interested in analyzing the data and represents the most petty aspect of the Administrator’s malicious compliance with Congressional direction and law. On January 4th 2008, IFPTEs submitted a FOIA request of its own for the original set of Excel files with redacted NAOMS data.

False Statement #5: The NAOMS project made incredible conclusions

At the press conference, Dr. Griffin reiterated that “(f)rom the conclusions that arose out of the survey itself, it was reported, for example, that the survey unearthed approximately four times as many engine failures as the FAA believes that it has cognizance of.” There is, however, simply no report from the NASA NAOMS team with that conclusion. Furthermore, Dr. Griffin reasserted this claim after the Union and others pointed out that his comparisons of FAA and NAOMS data were inappropriate (see detailed discussion about this in IFPTE’s November 29th 2007 letter to Chairman Gordon).

The NASA NOAMS team has not been allowed to issue a formal report (or reports) containing an analysis of the data and presenting their findings and conclusions. Dr. Porter has prevented that from happening. The engine-failure and unruly-passenger “conclusions” that Dr. Griffin cited at the hearing and the press conference were never made; Dr. Griffin has manufactured a spurious argument by distorting preliminary, tentative analyses performed in the context of informal internal discussions. The NASA NAOMS team never endorsed these, or any other conclusions, as they have not completed their analyses. During her October investigation of NAOMS, Dr. Porter was told in clear terms that these “findings”, leaked to the press, did not reflect any official analysis approved by the team. Despite this knowledge, Drs. Porter and Griffin shamefully chose to deliberately misrepresent these leaked findings, to present them as “conclusions”, and to attribute them to the NASA NAOMS researchers in order to discredit the project and validate their political decision to terminate NOAMS and suppress its data. We have seen this suppress-first, slander-second approach before with the Administration’s handling of NASA’s Global Warming research results.