NASA Internal Email: NOMAD Outage Caused by IdMAX Processing

Subject: NOMAD Outage Caused by IdMAX Processing
From: “Nomad-Outreach” NOMAD.Outreach@nasa.gov
Date: Wednesday, September 10, 2008
This notice is being sent directly to all NOMAD customers.
NOMAD Outage Caused by IdMAX Processing
Date: Wednesday, September 10, 2008
Time: 5:15 p.m. – 8 p.m. Eastern / 4:15 p.m. – 7 p.m. Central / 2:15 p.m. – 5 p.m. Pacific
Customers Affected: All NOMAD Customers
Systems Affected: Agencywide NOMAD
Details:
The Agency NASA Operational Messaging and Directory Service (NOMAD) experienced a problem as a result of a data update to Active Directory (structure used to construct the address book) caused by IdMAX processing. The @nasa.gov address was removed from the Active Directory for approximately 3,400 NOMAD user accounts.
During this outage, the 3,400 impacted users did not have access to their email client, Outlook Web Access (OWA), Instant Messaging, and their handheld mobile device (BlackBerry, Treo and Windows Mobile Devices).
Messages sent to the impacted users were returned as non-deliverable. NOMAD users who sent mail internally to any of the impacted 3,400 NOMAD users would have received a non-deliverable reply from the System Administrator with the text “Undeliverable” in the Subject line. The original messages were not delivered and must be resent to the recipients.
In an effort to minimize the number of non-deliverable error messages, the processing of all incoming mail was stopped at approximately 7 p.m. Eastern time / 6 p.m. Central time / 4 p.m. Pacific time. This action caused all in-bound mail, outside of NOMAD, to be queued for all NOMAD customers. The process was restarted at 7:45 p.m. Eastern time / 6:45 p.m. Central time / 4:45 p.m. Pacific time. All mail was then released from the queue.
The issue was resolved at approximately 7:15 p.m. Eastern time / 6:15 p.m. Central time / 4:15 p.m. Pacific time when the IdMAX team rolled back the data. All @nasa.gov email addresses were restored for the affected 3,400 user accounts.
A root cause analysis will be performed by the IdMAX team.
Should you have any questions regarding this activity, please contact the ODIN IT Help Desk at https://www.odin.lmit.com/nomad/nomadoutreach.html