161
Chapter 12:
Network Disaster Recovery
Have a Plan for Communications and How You Will Communicate
Following the CTO's announcement, some asked, "What about those who don't
have web access at home?" As IT staff, we asked, "What if the web servers
themselves had all been destroyed?" (In fact, ceiling debris in the room in which
they were housed fell very close to them, but the servers were not damaged
and the service was never down.) Still others asked, "What about those who
missed the message and don't know to check the Web? These questions, as well
as "What to do in the event of ...?" could be addressed with a clear, ever-ready
communications plan. Ironically, such plans had been developed down to the
last detail for other events, but in the case of a real "emergent" event, we as a
department had not identified a plan to follow. A priority for our department now
is to reexamine that situation and develop a plan, using communications plans
developed for the Y2K event and the like as models.
Another point: As previously mentioned, our staff is not responsible for
supporting the EOC on a routine basis. We are more than happy to be directed
to assist in that support, and as evidenced, have done so on a few occasions.
Almost immediately following the earthquake, I received a page indicating that
I was to dispatch technicians to the EOC to support the city officials who report
there during emergencies. While our team was under no agreement with the EOC
to provide support even "on demand," I immediately asked two of my senior
technicians, who had worked at the EOC in the past, to respond. They reported
for duty there and supported the facility until the assigned staff arrived. There
was never a doubt that we would pitch in whenever asked, but I made it a point
to ask our divisional director if developing some clearer expectations, or even an
SLA, between our staff and the EOC would be appropriate, and he agreed. I did
find out that those in the EOC are granted power by legislation to use "all" city
resources in the event of an emergency, but a clear agreement could also permit
me to identify a rotating on-call staff person who could be proactive and call the
EOC in such instances.
I must point out that none of these preparations can substitute for dedicated,
intelligent people. The shining example is one of my technicians who supports
programmers responsible for the city's payroll application. He had the presence of
mind to come early to work the day after the quake, and he somehow persuaded
the construction crew and inspectors to permit him access to the building. He
walked up 13 flights of stairs, picked up a PC and peripherals, carried it back down
the stairs and to another building, and configured it to work on the segment in
the new building. This made it possible for the programmer to run the operations
necessary for the city's payroll run that weekend, and employees received their
checks on time, as expected. You cannot ask for more than that.