ISACA CISM – Domain 04 – Information Security Incident Management Part 10

57. Goals of Recovery Operations Part1

Now, goals of recovery? Well, goals of recovery is to get us back to where we were when the incident occurred. I think that’s the easy statement. Your recovery strategies, though, will depend on the size and complexity of the organization. So it’s one thing to say the goal of recovery is to restore operations, but really when we’re planning it, there may be a lot of steps and things we have to go through depending again on the complexity. Now, it also kind of depends on the severity of the incident as well. But in general, some of the things of course we’re looking at is one of the big goals is the elimination or neutralization of the threat. Now the elimination I think, is a key word that says we’re going to make sure that it is gone.

If it was a virus, as I said, it’s been eliminated, it no longer exists. Neutralization could have been a denial of service and we’ve done something. Maybe talk to the internet service provider, block some traffic that’s being sent at us. Maybe we did some patching or updates of servers that were under attack. We changed rules on firewalls as far as the package they got through. So that doesn’t mean that the attack is still not happening. Remember, on a denial of service, we can’t stop somebody from attacking us, but if we neutralize it, hopefully we’ve made it as good as it not occurring at all, minimizing the likelihood of the threat occurring.

Now I think the first thing, of course, is eliminating a neutralization big part, but we don’t want a repeat occurrence. So we want to find what we can do to minimize the likelihood of it occurring again in a way even further reducing the overall risk. And of course, we want to minimize the effects that the threat does as far as when it happens, um, when it occurs. What it can do for us or to us, I guess is the better way of saying that. So in a way, the actual response should actually help harden the security because number one, we’re going to get rid of the threat, we’re going to reduce the chance of it happening again and minimize the effects that, that it could do if, if that attack or that incident were to reoccur.

58. Goals of Recovery Operations Part2

Now, here are some other goals we might look at as far as recovery options. Some of our solutions might use a temporary fix until a full recovery can actually be accomplished. Now, one of these temporary solutions might be in the use of what we call a recovery site. Now, we have a lot of things we can look at as a recovery site. Obviously, the first one we talk about is a hot site. Now a hot site is a site that is pretty much equipped, maybe even has some staff, but they’re not ready to fully go. We may have to transport backup tapes and logs to the hot site to restore that information, to be able to bring it up and online. Many times those hot sites might not have the same processing capability, communications abilities that we have the main site.

But we use them because again, during a time of a recovery, especially, let’s say if it’s a flood, being able to move to a hot side means hopefully within a few hours at most, we’ll be back up online working, but maybe not at 100% capacity.Now, a lot of these decisions we’ll talk about as far as how long we can stay there, we talked about some things like the MTOs that tell us how long we can stay there before really become unviable. As a business, a warm site is not as expensive as a hot site. By the way, out of these choices so far, the most expensive I’ve talked about, it’s just the one of them is a hot site.

Because we have to have personnel, we have to have equipment, we have to have the majority of types of servers and other hardware and peripherals that we need to be able to function already ready to go. A warm site usually has most of the equipment, it’s certainly wired, ready to go. But we may have to bring other actual material with us, other servers, other hardware to install it, to bring it up online, and certainly restore a lot of the backup information. Generally they’re not already manned with people, so it’s cheaper for us. But it’s downside is it will take longer to restore operations up to a minimum capability.

59. Choosing a Site Selection Part1

A cold site means that you have a building and that’s it really and it’s wired, it’s got power, hopefully appropriate HVAC, but it doesn’t really have any equipment. You have to be able to bring your remaining equipment over, back up equipment or the vendor has to supply things for you. And remember by the way, sometimes you might be moving here because of a natural disaster. So you might not even have equipment that’s south eligible at the original location. So a cold site even harder to get up to speed because you have to hope for vendor replacement parts and hope that you have offsite backups that you can restore. But it is certainly cheaper than the alternative of warm and hot sites.

Now a mobile site, I actually had a little bit of a part in helping work with a mobile site for the local county law enforcement where I live. And here’s what they did is they realized that if they lost their facilities that they wouldn’t be able to respond to 911 calls, that they would not be able to deal with dispatching of emergency medical crews of police departments and the rest of it. And so what they did is they created this huge RV that had in it the ability to plug into any central exchange already with an agreement with the telephone company to be able to receive the 911 calls from phone consoles that they had inside of the van or in the RV. They also had the radio equipment to be able to do transmissions.

They literally had a repeater that they would pull out from a storage area on top of the roof, set it up with a stand and a portable power generator so they could be able to broadcast radio transmissions. And so they were pretty well ready to go. And of course mobile connectivity to be able to try to maintain computer connections to gather records or if they had to, they could informally make phone calls to different agencies to get record information. But it was ready to go in case there was a major problem at their facility. And then they found that during really big events that they could bring the mobile site to a location and be able to provide services on site rather than depleting existing services to handle the emergency within the main site.

So really the idea was that if it was a really huge emergency, other things are still going on. So the main site, rather than taking the burden of that hit, they would just call people in, move the mobile site to that scene and it worked pretty well. You might have something similar that you might do with recovery of your services. Duplicate sites are much more expensive. A duplicate site is pretty much just that it is the clone of your existing site. It may be sitting there in a standby mode and by the way, in a duplicate site you are exchanging information and data so there’s no backups to restore.You’re basically synchronizing the information. You’re just waiting for the first site to fail and then the duplicate site will take over.

We’ll reroute all the traffic to that duplicate site and it will be as though nothing has changed. Now one of the things you have to remember about a duplicate site is that we needed to be in a different location than the actual site. And when I say different location, people are saying oh yeah, it’s not going to be next door. But I mean in an area so far removed that whatever natural event occurred at the main site would not disrupt the duplicate site. That means if you had a power grid failure, you want to be in an area that’s a different power grid. If it’s an issue of earthquakes, you want to be in an area that might not even feel the tremor, tornados and the rest of it. I realize I’ve worked with a lot of companies, as I said, that they had sites that were up in the mid part of the country, in the north, down in the south and on the west coast.

And the way they were set up is that any one disaster of any kind, man made, if it was acts of war, if it was power grids, or if it was natural types of storms, earthquakes, floods, they were separated. And of course the duplicate sites are fully manned. They’re also makes them very expensive. A mirrored site is much like the duplicate site, but it’s working at the same time. In other words, we’re trying to take advantage of the power of both of them, often with something like a load balancer that can distribute the traffic between them so that they can both do the work simultaneously without one just really sitting idle. So those are options you have. In the old days we used to go with reciprocal agreements. A reciprocal agreement was saying maybe like this I’m cellular provider A and I have things that just completely go out.

I’d like to let my customers work with cellular network B and they agree to let us utilize their facilities until we can bring ours back up online so that our customers don’t see the outage. Fire departments had something similar. If a fire department station was sent their crew out on a large fire and they weren’t going to be back for a while, neighboring cities would put fire, manned fire engines in those buildings so that they could take care of any calls that might occur in that same district so that they would have a better response. We call that a reciprocal agreement or they might even to come and help him fight the fire. So we could also if use other vendors like I said, or third parties or maybe we just have something that’s off the shelf that we can use. But these are some options we have for recovery.

60. Choosing a Site Selection Part2

Now the decisions about what type of site you want, the temporary sites really depends on what the needs are of your recovery time because some of the selections as I talked about might take a very long time like a cold site. So we have to look at that business impact analysis and understand first of all, what’s the maximum interruption that’s going to be allowed and then we have to look and see do we have recovery time objectives, recovery point objectives, service delivery objectives, maximum time offline? All of those can help us make the decisions.

Do I need a mirrored site, duplicate site, hot site, warm, cold, mobile? Also you need to consider the distance to the site, you know, a duplicate site, mirrored site that, you know, that’s probably not an issue about distance only because they’re already manned. Some hot sites are already staffed as well. But you know, if I have a warm site, cold site and I chose one that’s 300 miles away, well now I’ve got a whole another problem and that is how to get people there. So sometimes that’s a distance, that’s something you have to think about as well. And of course you have to decide what kind of disruptions are you planning for. Because if it is something of a natural disaster, if it is too close it may be subject to the same disaster faster and then become pretty much useless to you.

61. Implementing the Strategy

Now, depending on the type of response and recovery strategy that the management has decided to use, we need a detailed recovery plan, one that’s developed on that choice. Now, some of the factors that you should look at when you’re developing the plan, of course, are things like preparedness. Are we prepared? How much training have we had? Do people understand their roles and responsibilities? Do we have the resources and facilities in place? We may have to deal with evacuation procedures as well, especially if our main site becomes uninhabitable. We need to know when and how or what criteria we can declare a disaster. Now, again, it’s kind of like the incident response idea is that just because you might be having a problem, it doesn’t mean it’s a full fledged security incident.

But we need to have some criteria by which we can say, all right, this does count as one of those times. I’d hate to have somebody or me to be responsible of saying, oh, I think it’s a disaster when it’s really not. Start mobilizing people to a hot site. A hot site, start transferring data over, calling everybody out of bed. They come up, they start responding, and they’re like, this was not a disaster. What were you doing? But if I have a clear set of criteria where I can say, oh, but I hit this and this point and you said, if all those occur, that’s a disaster, then I feel better about my decision. Listing, of course, the processes and resources that need to be recovered. That’s one of the things about a hot site, is we don’t have all of the equipment, but we have those bits and pieces of technology that help us in the processing or the resources that we need.

And resources, by the way, is more than equipment. It’s also software. If there are certain programs I need running, they need to be there. Whether if it’s a hot site, they should already be installed. If it’s a warm site, I need to be able to know that I have the software I need to install on these systems, same as a cold site. What are the roles and responsibilities that I have? Who’s responsible for evacuations? Who’s responsible for looking at how much damage has been done? Who’s responsible for making the notifications and starting these things up? Who’s responsible at the hot site, warm site, or other alternate site? Of course, we have to be able to contact the responsible parties. There should be a communication path already set up. And we may, because of the distance of those sites, have to also have plans for the logistics, for the staffing and the housing, albeit temporary, as well as transportation, to get people to that location.

Now, one of the companies I talked about in a different domain, it was a company that was the owner of a lot of different restaurant chains. I made mention that they had already planned for a remote site and it was in a different state. And the idea was that if they needed to, because it was not manned or at least staffed, I should say what they did is they had either I don’t remember, it was a charter or private jet that was at the waiting. Should there be something like an approaching hurricane. This company was in Florida and trust me, they get a lot of storms down there, a lot of lightning, a lot of disruption of power down there. So there’s a lot of planning down there about backups and generators. Anyway, the goal was that they would then fly the people to that location.

Now, of course, in today’s technology, we have a lot of warning about hurricanes. So it’s not like they had to drive through the rain and get into the plane and hope it could take off. It’s something they would do early. But then I thought to myself, you know what, I might not volunteer for that plan because I’m not going to maybe leave my wife and six kids in harm’s way. I’m going to be there protecting them. And so they said, no, they covered that too because the family went with them when relocated and of course they had housing and all those logistics already planned out. It was a part of their plan. And so that’s kind of the things that we have to look at as we are creating the plans, we have the strategy and begin the implementation of this, that we have all of these factors already considered.

• Category: Uncategorized