NASA Planetary Senior Review Panel Report
Director, Planetary Science Division
Science Mission Directorate
NASA Headquarters
300 E Street SW
Washington, DC 20546
Dear Dr. Green:
The NASA Planetary Senior Review panel met at the Sheraton Columbia Town Center Hotel in Maryland on 14-15 May 2014 to discuss extended missions for Cassini and Lunar Reconnaissance Orbiter (LRO). A second panel of essentially different personnel (apart from the Chair and one other) met at the Embassy Suites Hotel Baltimore at BWI on 20-22 May 2014 to discuss five Mars missions: Mars Exploration Rover (MER – Opportunity), Mars Odyssey (ODY), Mars Express (MEX), Mars Reconnaissance Orbiter (MRO) and the Mars Science Laboratory (MSL – Curiosity). All extended missions were rated higher than “Good”, some after adjustments to scope, as it was recognized that they continue to add important new data and observations for our understanding of Solar System bodies and processes. The science value (or science per dollar) of the extended missions exceeds the science gain from any planned mission, and all have important strengths. That is, they all represent added value to the Planetary Science Division and the American taxpayer because they are essentially new missions without the development and launch costs. Therefore, this Senior Review examined each proposal with regard to the “new science” that was being proposed. In addition, we also examined previous productivity, the value of long-term observations, as well as how to improve efficiencies and identify the optimal use of limited funding because of the severe budgetary pressures over the years covering the extended missions. The hard task of achieving the best value for the funding available requires iteration with the projects, programmatic considerations, and input from the scientific community. The Senior Review represents the most effective and efficient way of providing scientific community input.
Each spacecraft contained formidable capabilities when they began their prime mission and this Senior Review commends the high quality of scientific work already achieved by all seven missions. The harsh environment of space, however, has produced a degradation of these capabilities as each mission proceeds. This makes enabling new science more difficult, but follow up observations on the discoveries that these missions have already made bring added value to planetary science. Therefore, every effort should be made to preserve at some level the capabilities that produce new and groundbreaking science.
The panel treated each mission in a similar way, voting on the extended mission as proposed for the Guideline budget. If there was discussion about other proposed budgets (Overguide, Optimal and Descope) and the panel felt that these were more appropriate, they were also voted upon. If the panel thought that a different descope plan was appropriate, this was also voted on. Each extended mission proposal was treated on its own merits and there was no comparison between the proposals the Senior Review panel was asked to examine.
Discussions and Review Format:
The evaluations presented by this Senior Review are based upon everyone being able to freely give their candid opinion on each mission. External reviews were included for each mission at the panel meetings if available. During the panel deliberations of each mission all NASA observers were asked to leave the room. This request was met with some surprise but it was the Chair’s opinion that more candid discussion could be achieved with the panel in executive session.
Despite the logistical challenges of two separate panels meeting at two different times, each mission was treated in the same way. Throughout this report, the term “the panel” encompasses both separate panel meetings. Based upon the submitted proposal, the panel furnished each mission team a list of questions to address during the mission presentation. Each mission was given two hours for this presentation. After the presentation, the panel went into executive session and, on an as-needed basis, the mission team was asked to come back and address further questions that had arisen.
In Table 1 the adjectival ratings are presented. Several of the orbiting Mars missions offer benefits to the overall Mars program for instance by providing communication services for surface operation of the Opportunity and Curiosity rovers or by helping to determine landing sites for future surface missions. The value of these services was not considered in our ratings, although such services are mentioned in the individual reviews. The panel concentrated on the new science return value of each extended mission and the achievements so far (as presented in the proposal) as well as the presentation by each mission team. For the Guideline budget, the Cassini mission received the highest overall being rated “Excellent”. MER and MRO both received a rating of Excellent/Very Good. LRO, ODY and Curiosity followed each with a rating of Very Good/Good. MEX received a rating of Good/Fair (Table 1).
Some of the missions proposed to address the potential Descope budget by taking most of the reductions in the science part of the mission. As in 2012, the panel felt that this was shortsighted. It does little good to collect the data if the science community is unable to validate and analyze them. The panel felt that more balanced reductions involving both science and operations could and should be applied in cases where the full funding is not available. In some instances the panel suggested different descopes from what was described in the proposal. This was the case each for LRO, ODY, and MEX (see below) and on this basis the panel re-voted on the basis of these descopes: LRO achieved a rating of Excellent/Very Good, ODY and MEX Very Good, and Curiosity was rated Very Good/Good.
Table 1: Summary of the 2014 Planetary Missions Senior Review Ratings
The panel findings for each mission are briefly summarized below.
Cassini
The Cassini mission received our highest rating. It was the only mission to achieve a consensus rating of “Excellent” for science merit on the basis of the Guideline budget. The Cassini mission has contributed groundbreaking science in the past and has the potential to continue this during the final three-year Northern Summer Mission (NSM). During the NSM, seventeen new science objectives will focus on five areas of investigation: Titan, icy satellites, rings, magnetosphere, and Saturn. The spacecraft will complete equatorial orbits that enable flybys of icy moons and “ring-free” Saturn observations, as well as inclined orbits that will enable ring studies and high-latitude Saturn and Titan mapping. The seasonal processes will form an important part of the NSM mission and will include Saturn, Titan, the icy satellites (including Enceladus), rings, and the magnetosphere. As the mission proceeds, Cassini will observe Saturn’s aurorae and polar storms. The conclusion of the mission will be the F Ring Proximal Orbit (FRPO) phase offering unique measurements of the mass of the B ring, higher harmonics/moments of Saturn’s gravity and magnetic field, as well as Saturn’s upper atmosphere and ionosphere. End of mission will be when the spacecraft impacts Saturn’s atmosphere.
The proposed NSM has a high likelihood of success based on past performance, the fact that the spacecraft is in good condition, and the unique aspect of the new observations. The Cassini team developed and presented an excellent extended mission proposal, although some aspects lacked detail despite having an extra ten pages to describe the NSM science. Also, the way the guideline budget was met meant that opportunities for new investigators would not be prioritized, although operations would be maintained at current levels. It was a unanimous recommendation that the Cassini NSM be endorsed at the Guideline budget level and NOT at the Overguide. Neither the proposal nor the presentation convinced the panel that the Overguide budget was justified.
Lunar Reconnaissance Orbiter (LRO)
The second Extended Science Mission (ESM2) for LRO will characterize the changes in the lunar surface, as well as beneath the surface, and in the exosphere. Five broad areas of investigation will be highlighted: characterization of the seasonal volatile cycle, determine the current flux of small meteorites and their impact dynamics, characterization of the vertical and horizontal structure of the regolith, investigate the Moon’s interaction with the space environment, and probe the interior using observations of the surface.
The reviews of the proposal generated a large number of questions (49) that were sent to the LRO team prior to the panel meeting. This indicated the lack of detail in the body of the 30-page proposal. The LRO presentation from the Project Scientist at the panel meeting did not yield the level of clarity hoped for, although the presentations by the LRO Camera (LROC) and Diviner instrument leads were compelling. This resulted in another list of questions being compiled and the LRO team was asked to come back and address these. Unfortunately, the second presentation did not result in the resolution of the issues the panel identified in the proposal and from the first presentation.
The initial panel vote on the Guideline budget resulted in an adjectival grade of Very Good/Good. The panel felt that some instruments were at the end of their useful science mission and suggested the descoping, in order of preference, Mini-RF, LAMP, and CRaTER. With Mini-RF operating in a receive-only mode and relying on radar transmitted from Arecibo, it would appear that this instrument is at the end of its useful science mission. The low productivity from CRaTER may suggest a similar situation, although this instrument is still functioning nominally. While although CRaTER is considered to be returning useful data, it is not directly lunar science and NASA may wish to pursue cost-sharing with other programs for continued funding (e.g., Heliophysics/Astrophysics/HEO) for operation of this instrument. The panel re-voted the LRO ESM2 based upon the suggested descope, but maintaining the Guideline budget for the mission and resulted in an adjectival grade of Excellent/Very Good (Table 1).
Opportunity (Mars Exploration Rover/MER)
The Mars Exploration Rover (MER) Opportunity continues to make important scientific discoveries on the surface of Mars. In the latest extended mission it proposes to determine the aqueous, environmental, and geologic conditions that constrain past and present habitability, as well as examine present climate variability. This will be achieved by measurements of rocks and soils as well as atmospheric observations as it traverses from Murray Ridge to Cape Tribulation. This extended mission will focus on the orbitally detected phyllosilicate deposits near Endeavour crater, which are considered to represent deposits from the ancient Noachian period. This would represent the first time that such ancient deposits have been analyzed on the Martian surface.
The proposal and presentation demonstrated how the extended mission would continue to reveal details of the ancient aqueous environment of Mars that are not ascertainable from orbit. However, there is no definitive proof that the phyllosilicates near Endeavour crater are Noachian in age. Opportunity is currently in sufficiently good condition to complete the proposed science, although software and communication issues that afflict the rover could impact its ability to collect/transmit the proposed data.
The presentation to the panel addressed all the questions previously sent to the MER team and demonstrated the validity of the proposed extended mission investigations. Having the Project Scientist and the Science Payload PI present during the presentation allowed new questions to be addressed in real time. The adjectival grade for the MER extended mission was Excellent/Very Good for the Guideline budget. No other vote was taken.
Mars Reconnaissance Orbiter (MRO)
This proposal represents the third extended mission (EM3) and identifies 21 investigations grouped in six broad categories: habitability and aqueous environments of early Mars, recent climate variability, contemporary surface changes, atmospheric observations, and campaigns of opportunity (observations of Comet 2013/A1 Siding Spring and its influence on the atmosphere, and synergistic observations with the MAVEN mission). MRO will also fulfill important programmatic objectives such as present and future landing-site characterization and surface operations communication relay responsibilities.
The panel voted only on the Guideline budget as proposed and MRO received an adjectival grade of Excellent/Very Good (Table 1). This was based upon a presentation that addressed all questions sent to the MRO team prior to the panel meeting, the fact that data from MRO are being used in ~120 papers per year (many from non-team members), the description of the science presented to the panel was compelling, and the spacecraft is in relatively good condition. The only potential issue with the spacecraft is the fact that the Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) only has one functioning cryo-cooler, having thereby lost the level of redundancy it once had. If this one fails, CRISM would not be able to detect new phyllosilicate or carbonate exposures. The panel recommends funding at the Guideline budget level and no other vote was taken.
Mars Express (MEX)
US participation in Mars Express is dominated by involvement with the Mars Advanced Radar for Subsurface and Ionospheric Sounding (MARSIS) and the Analyzer of Space Plasmas and Energetic Atoms (ASPERA) neutral and charged particle instrument. The extended mission identifies eleven objectives, broadly grouped into ionospheric/atmospheric topics (ASPERA and MARSIS Active Ionospheric Sounding – AIS) and surface/subsurface topics (MARSIS Subsurface Sounding – SS and High Resolution Stereo Camera – HRSC). Radio science, ASPERA and MARSIS will investigate the dynamic Mars ionosphere and magnetosphere, exploiting several synergies with the upcoming Mars Atmosphere and Volatile EvolutioN (MAVEN) mission. In addition, continuing support is sought for MARSIS SS, largely to map the North polar-layered deposits and for HRSC to continue image production and validation/calibration activities. Nominal funding is also requested for the Infrared Mineralogical Mapping Spectrometer (OMEGA) targeting of potential landing sites.
Based upon the Guideline budget, MEX received an adjectival grade of Good/Fair. This was based on the inadequate justification within the proposal and presentation of:
— How individuals supported in the extended mission would be involved in the analyses that would result in attaining the cited goals of the proposal;
— Why a mission in its eleventh year still requires funding for HRSC image calibration and validation;
— The strategy for prioritization between the MARSIS SS and AIS modes, particularly in light of the collaboration with MAVEN, which made it unclear whether the science goals for these modes could be attained.
A major weakness of this extended mission was the fact that the HRSC image calibration and validation tasks were not justified, and the team could not quantify how much better the camera alignment or geometric information will become with the proposed tasks, nor how much image quality would improve or specifically what new science could now be achieved as a result. In addition, the HRSC data have been used in only four publications in the last two years. Finally, the availability of MEX data in the PDS/PSA for all four instruments (including OMEGA) remains intermittent.
The panel noted a number of issues related to a lack of communication within the MEX team. For example, the US PI of ASPERA being unaware of instrument mode changes made several months ago, a seemingly ad hoc and unequal arrangement for assigning orbits between the subsurface and active ionospheric modes of MARSIS, and unclear or conflicting information about required FTEs for certain portions of the proposed effort. This needs to be urgently addressed.
The recommendations from this Senior Review are that:
— Only the automated aspects of HRSC image processing (along with occasional spot-checks) be supported at a very low FTE level and all other US support for HRSC be terminated;
— Funding of the MARSIS AIS should be at the Overguide level (additional $100k/year for the requested postdoc at Iowa). Without this, detailed coordinated ionospheric observations with MAVEN will be greatly reduced.
The panel re-voted the extended mission proposal on the basis of these recommendations and it received an adjectival grade of Very Good (see Table 1).
Mars Odyssey (ODY)
The sixth extended mission (EM6) of ODY will move the spacecraft into a new orbit so the instruments can view the terminator. This will allow EM6 to achieve four scientific objectives: 1) Provide an understanding of the distribution of water, carbon dioxide and dust in the atmosphere and how they interact with the surface; 2) Generate a global picture of the Martian surface mineralogy, geomorphology, and thermophysical properties; 3) Provide an understanding of the Martian radiation environment and its variability as a function of time; and 4) Serve as an observatory for monitoring cosmic gamma ray bursts.
Neither the proposal nor the presentation made convincing arguments as to how substantial new science that directly relates to Planetary Science Division objectives as outlined in the Decadal Survey would be possible from all instruments during EM6. Specifically, the remaining instruments in the Gamma-Ray Spectrometer (GRS) suite (the Neutron Spectrometer – NS and the High-Energy Neutron Detector – HEND) primarily address Heliophysics and Astrophysics objectives. During the presentation, it also became evident that certain aspects of EM6 would actually be data analysis tasks rather than new observations resulting in new science discoveries.
The panel did note that JMARS is a valuable tool for the Martian planetary community, as well as the general public: it is sufficiently user-friendly that educators at all levels use it around the country. It would be detrimental to science productivity if the community were to lose this resource due to a lack of funding because JMARS supports multiple data sets.
The panel re-voted the ODY EM6 proposal on the basis of descoping the total GRS suite, as it is conducting science observations to address objectives peripheral to planetary science. The adjectival grade improved to Very Good (Table 1), but ODY may be coming to the end of its life as a productive science mission, as highlighted by the declining rate of publications using ODY data over the last three years.
Curiosity (Mars Science Laboratory – MSL)
The proposal and presentation given to the Senior Review panel described the first extended mission (EM1) of Curiosity. During EM1, Curiosity will traverse about 8 km, visit four different units that presumably represent different climatic stages in the history of Mars, and conduct a total of eight detailed analyses (two from each unit) using the instrument suite on the rover. This will be accomplished as it traverses the foothills of Aeolis Mons or “Mount Sharp” within Gale crater. The science objectives of EM1 are to focus on identifying habitable environments capable of preserving organic (carbon-containing) compounds, and characterizing the major environmental transition from freshwater deposits to those produced under more acidic conditions. The panel was presented a plan that was driven by maximizing the traverse distance that would allow sampling of each of the four units traversed.
Although several strengths were noted during the review, the panel felt that the problems are sufficiently severe that they need addressing at the earliest opportunity. The vote on the Guideline budget resulted in an adjectival grade of Very Good/Good (Table 1). Examples of the significant problems the panel found with the Curiosity EM1 proposal are given here:
— The capabilities of Curiosity provide the only current way to make certain measurements on the Martian surface (detection of carbon, in situ age-dating ability, and measurements of ionizing particle flux). However, in the EM1 plan, these are minimized, as only eight (8) samples will be taken in two years (two from each of the four units to be visited). This means that during the prime and EM1 missions a total of 13 analyses will be made by a highly capable rover. The panel viewed this as a poor science return for such a large investment in a flagship mission.
— Despite identification of two EM1 science objectives, the proposal lacked specific scientific questions to be answered, testable hypotheses, and proposed measurements and assessment of uncertainties and limitations.
— Although the Curiosity team stated they would strive to identify sites where analyses will optimize the ability of addressing the science objectives, the roles of ChemCam and Mastcam could play in this are not discussed. Similarly not discussed was the synergistic role such ground-truthing would have in this mission, or for identifying similar deposits in orbital data for different areas on Mars.
— The proposal did not provide a convincing argument for reaching the upper-most sulfate unit during EM1. The panel is deeply concerned that observations in the clays, which may be more relevant to the habitability question, could be cut short because traverse distance will take precedence over scientific analyses.
— Although the proposal claims that the rate at which Curiosity’s power subsystem is degrading will require conservative analysis beyond EM1, it provided no evidence that, should all EM1 objectives be accomplished, additional science could not be planned for EM2.
— It was unclear from both the proposal and presentation that the Prime Mission science goals had been met. In fact, it was unclear what exactly these were. Upon detailed questioning, the team noted that the Level 1 requirements were actually engineering capability requirements with which the mission launched and are not reflective of the state of fulfilling mission success criteria, which were not addressed quantitatively.
After the presentation and subsequent discussion within the panel during executive session, other questions were formulated and then presented to the Curiosity team. Unfortunately the lead Project Scientist was not present in person for the Senior Review presentation and was only available via phone. Additionally, he was not present for the second round of Curiosity questions from the panel. This left the panel with the impression that the team felt they were too big to fail and that simply having someone show up would suffice. The panel strongly urges NASA HQ to get the Curiosity team focused on maximizing high-quality science that justifies the capabilities of and capital investment in Curiosity. Given that Mars 2020 will be built on Curiosity heritage, maximizing the science return from Curiosity would go a long way to a smooth development of the next US Mars rover mission.
As Curiosity is a flagship mission, the panel was surprised by the lack of science in the EM1 proposal (the Overguide budget would support greater roving distance over samples analyzed, with only a promise of a maximum of eight analyses throughout EM1). Therefore, the panel voted on all three budget scenarios. The Overguide budget received an adjectival grade of Good/Fair, and the Descope budget receiving an adjectival grade of Very Good/Good (Table 1).
In summary, the Curiosity EM1 proposal lacked scientific focus and detail. The recommendation of the Senior Review panel is a descope in the traverse distance as EM1 would better serve science by focusing on the Paintbrush, Hematite, and possibly the Clay units and doing a better job of characterizing these, than focusing on the upper layers in EM2. The panel specifically recommends that the Overguide budget should NOT be implemented for EM1.
Summary
Of the seven missions examined under the 2014 Planetary Science Senior Review, four have significant problems: LRO, Mars Express, Mars Odyssey, and Mars Science Laboratory (Curiosity). The panel has suggested a number of measures to mitigate these problems, some of which will require NASA-HQ action in restructuring/refocusing the scientific leadership/emphasis. In other cases, instruments may need to be descoped in order to achieve a lean, effective mission that continues to produce groundbreaking science.
Lessons Learned
The experience of the 2014 Planetary Science Senior Review has demonstrated a number of things that should be improved for the 2016 experience. These are documented below.
— There should always be one Senior Review panel – not two that meet at separate times as there was in 2014. The Senior Review is for the Planetary Science Division, not the Mars Program and then everyone else. Having one panel assures that ALL missions are treated equally and fairly. Obviously, the composition of the panel will be critical to ensure fair treatment. In 2016, early identification of a panel chair, who is involved in the selection of the panel, should smooth this process.
— Given the detail of each proposal and the important implications for planetary science, the Senior Review requires sufficient time to thoroughly review each proposal. Although the Chair believes each panelist did their best to be thorough and diligent in the review, the timeline for preparing questions for the teams was rushed. More time is needed to be able to do due diligence for each proposal and to have every panel member read every proposal. In 2016, at least two weeks should be given for the panel to prepare questions for each mission team.
— Each of the 2014 panel meetings was too short to accomplish the tasks with ALL panel members present. A lot of work had to be conducted after the meeting via email, where panel members are not 100% focused on the issues. In 2016, an entire day free of mission presentations should be added to the end of the Senior review panel meeting, with the panelists being released the morning after, not the evening of the last day.
— The 2014 Senior Review found that not having the lead Project Scientist or the instrument leads present at the face-to-face meeting caused confusion and generated more questions than were actually answered. In 2016, it should be a requirement that the lead Project Scientist/Principal Investigator be present in person for the mission presentations to the Senior Review.
— The NRESS staff did an excellent job of supporting the Senior Review before, during, and after the face-to-face meetings. In 2016, change nothing!
— While it is understood that a number of potential budgets are suggested to the teams as they write their extended mission proposals, more guidance should be given. For example, science always suffers in terms of budgets less than the Overguide/Optimal with very little impact on operations. In the proposals reviewed in this Senior Review, adequate justification for maintaining operations at the expense of science was not given. The panel’s interpretation is that the proposals gamble with science, expecting the Senior Review to support Overguide requests to maximize science. In 2016, the instructions to the mission teams should make it clear that in the proposal a detailed and unambiguous explanation as to why operations should not be reduced in preference to science should be presented and this should be within the proposal page limit.
Respectfully submitted,
Clive R. Neal
PMSR-2014 Chair
cc: William Knopf, PMSR-2014 Panel Members
