A good disaster recovery plan is like an information insurance policy for a small or large business. Also called a business continuity plan or an information availability strategy, a disaster recovery plan is a detailed, step-by-step course of action for getting a business back on its feet -- and quickly -- after a natural or manmade disaster.
When Hurricane Katrina slammed the Gulf Coast of the United States in 2005, it claimed more than 1,800 lives, wreaked $200 billion in damage and wiped out the communications infrastructure of a whole region. It uprooted 1,000 wireless towers and knocked down 11,000 utility poles.
The telecommunications sector tallied $400 to $600 million in damages alone and critical businesses were forced to shut down entirely, including 25 hospitals and 100 TV and radio broadcast stations.
In a disaster of Katrina's magnitude, there's only so much that can be done to salvage a business and keep essential services online. But as you'll see in this article, the right disaster recovery plan with the right contingency plans in place can help keep the core services of a company up and running in even the worst conditions.
For example, the disaster recovery company SunGard was able to keep its Gulf Coast clients in business by relocating many of them to SunGard hotsites, off-site facilities equipped with the computing power and backed-up data to keep systems and services online. Sungard's clients occupied these hotsites for an average of 22 days after the storm. Others relied on mobile hotsites -- 18-wheelers with servers and office equipment inside -- for an average of 18 days.
Besides the obvious threat of natural disasters, there are plenty of reasons why disaster recovery plans have become a requirement for doing business:
- Increased reliance on computer networks, databases and online services means increased vulnerability in the case of a network outage, whatever the cause: employee sabotage, cyber attacks, viruses, sudden loss of Internet service, malfunctioning equipment, etc.
- The SQL Slammer worm of 2003 shut down the ATMs of major banks like Bank of America and Washington Mutual for days and caused the cancellation of several Continental Airlines flights.
- According to the 2006 CSI/FBI Computer Crime and Security Survey, 52 percent of the 616 large corporations surveyed said they'd experienced unauthorized use of computer systems within the past 12 months. The same survey says companies lost $16 million in virus contamination alone.
- Several recent U.S. government regulations including the Health Insurance Portability and Accountability Act (HIPAA), Gramm-Leach Bliley Act, Sarbanes-Oxley Act require that the health and financial industries have detailed contingency plans in place to safeguard confidential client information.
- Customers expect essential online services like banking and e-mail to be accessible 24/7. These companies need to consider a lengthy list of potential disasters, both small and large, that could interrupt service to their clients and take steps to address all of them.
In this article, we'll go through each step of the disaster recovery planning process from the first proposal to regularly scheduled testing of the plan. Keep reading to learn how a company gets started on the road to disaster recovery.
Disaster recovery plans deserve the full attention of executives at the highest levels of an organization. To ensure that upper-level management takes ownership of the plan, experts recommend that all good disaster recovery plans begin with an official proposal. That proposal can be presented to the board of directors to the chief executive officer or to the chief information officer.
The proposal isn't the plan, but rather a "plan for making the plan." If the disaster recovery plan is going to be created in-house, the proposal should recommend which specific employees will manage the project and approximately how many work hours it should take. If the proposal recommends a third-party consultant, it should include a budget based on the consultant's proposed services.
If the company has never drafted a disaster recovery plan before, the board might need to be convinced that it's necessary. The online Disaster Recovery Guide suggests using the following reasons:
- The dependency on computer networks and electronic delivery systems increases the odds of everyday business being disrupted by failure of one of these core systems.
- A formal process to deal with potential accidents, disasters or outages is needed and in some industries, required.
- Everyone wants to lower the potential costs of dealing with a disaster.
- Because of the speed at which technology is changing, there's a greater chance of a "knowledge gap" leading to inadequate information technology security precautions.
- An effective system to backup and recover essential business data is essential in case of network shutdown.
- Most importantly, everyone wants to avoid the potential failure of the business in the face of an unforeseen, catastrophic event.
Once the proposal is approved, the real work begins. Drafting a good disaster recovery plan is a slow, methodical process. It requires that each department be broken down into its smallest units, and that each separate function of each unit be analyzed for its importance to the business.
This is why many businesses choose to hire an outside consultant to create their disaster recovery plan. Third-party disaster recovery experts have the experience and impartiality to conduct employee interviews, design questionnaires and analyze day-to-day practices in order to come up with the most comprehensive recovery plan possible.
Creating a Business Impact Analysis (BIA) is the first step. A BIA questionnaire collects all the information about a single business function so it can be ranked in the order of importance. According to the Texas State Office of Risk Management (SORM), a good BIA should include the following information and more:
- A detailed description of business functions and operations for each department
- Any other functions with a direct or indirect impact on this function
- When the function loss would have the most impact
- The time it would take the business to realize that the function had failed, both operationally and financially
- The replacement equipment needed to recover from this function's loss such as phones, PCs, software and workstations
- If the function can be accomplished by working from home, or if it can be shifted to another part of the business
The disaster recovery planning team uses this information to rank all business functions according to time tiers. For instance, Tier One includes those functions that need to be back online within a few minutes up to 24 hours. Tier Two includes those functions that need to be back online within 24 to 36 hours and so on.
Another piece of the pre-planning research, according to the Disaster Recovery Guide, is assembling important contact information and emergency procedures. You'll need emergency contact information for all employees, all vendors and partners and equipment inventory for all information technology and administrative departments. You'll also need paperwork and procedures for dealing with evacuations, floods, fires, earthquakes and insurance.
Keep reading to find out what kind of information goes into the actual disaster recovery plan.
What to Include in a Disaster Recovery Plan
The safety of personnel and their families should be the first priority of any disaster recovery plan. In a white paper published by SunGard called "Lessons Learned from Katrina", the authors say that successful plans not only account for the transportation and safe lodging of employees, but of their families as well. Once family members are safe, employees have a greater chance of being able to concentrate on helping the company.
Each employee should be trained in his role in the disaster recovery plan. Back-up personnel should be designated in the case that an employee is not available, and obviously the back-ups need to receive the same training. Frequent audits of the plan should be made to ensure that the same people are still employed in the same roles with the same contact information.
Multiple lines of communication are crucial to any good disaster recovery plan. Detailed lists of employee and vendor contact information are a minimum. Some companies sign up with third-party conferencing and emergency communications services to send automated messages in times of crisis. The messages can be sent from any platform (phone, e-mail, SMS) and received on any platform.
An advantage of these conferencing services is that individuals can easily update their contact information and indicate the fastest way to reach them in an emergency. Emergency communications services also have the ability to call everyone on the contact list at the same time, ringing pagers, cell phones and sending e-mails until receipt of the message is confirmed.
A third-party disaster recovery specialist can also provide a company with alternate workspaces in the event that the regular office is unavailable. Not only do these workspaces have all of the necessary equipment to do business (desks, phones, PCs, Internet access), but they also have access to all of the company's data. This is because the disaster recovery company has been regularly backing up and storing the company's data in an off-site facility.
After personnel, data back-up and recovery comes next. Experts recommend that data either be backed up on hard disks and stored in an off-site facility or be uploaded to third-party servers at on off-site facility. Depending on the business, this upload process can happen once a month or in real-time.
As part of the BIA process, certain types of data should be identified as crucial to running the business. The back-up and recovery plan should reflect those priorities. Security measures should also be included in this section of the plan so that all employees are trained in the safeguarding of the company's systems and sensitive data.
Read on to find out about insuring relationships with vendors and some more advantages of third-party disaster recovery services.
Partners and Outside Help
A company is both a vendor and a client to dozens if not hundreds of other companies. What happens in the event of a disaster? What are your vendors' obligations to you? What are your obligations to your clients and partners?
Company attorneys draw up a Service Level Agreement (SLC). According to The Business Continuity Institute, an SLC is a binding agreement between two companies, or between suppliers within the same company, that covers the "nature, quality, availability and scope of the service provider." Within the SLC are special provisions related to emergencies, so that one side can be legally liable if it fails to deliver an essential service, even in time of crisis.
We've already talked a little about third-party disaster management services. We've mentioned that they can assist with the interviewing and research process to make sure the disaster recovery plan is thorough. We've also mentioned that they can provide alternative workspaces in the event of an emergency and can host and store data off-site.
But perhaps the most useful service of a disaster management company is regular testing and auditing of the disaster recovery plan. A disaster recovery plan won't do any good if it's five years old and collecting dust in the CEO's drawer. Frequent testing for multiple contingencies ensures that a disaster recovery plan works. It's equally important to diligently audit and update the contact lists of employees and vendors. A company should maintain detailed records of its hardware, software and network equipment.
Creating and updating a comprehensive disaster recovery plan is not a simple task. It's not cheap either. Surprisingly, a number of companies create incomplete plans or ignore them. According to a survey done by the Association for Financial Professions months after Hurricane Katrina, only 24 percent of respondents said that their companies had recently tested their disaster recovery plans. A full 50 percent admitted they had no intention of testing their plan.
To find out more about disaster recovery plans and how you can safeguard your business against unforeseen events, check out the helpful links on the next page.