CompTIA CYSA+ CS0-002 – Frameworks, Policies, and Procedures Part 2

5. Audits and Assessments (OBJ 5.3)

Audits and assessments. In this lesson we are going to talk all about audits and assessments and all the different components that are comprised as part of this auditing and assessments. Now, this is going to include things like quality control and quality assurance, verification and validation, assessments and evaluations, audits themselves, and schedule reviews and continual improvement. As we go through this lesson, we’re going to define each of these these terms and talk about how they differ. First, quality control. Now, quality control is the process of determining whether a system is free from defects or deficiencies. For example, if you’re putting together a car, you want to go back and look at that car and make sure it meets the quality control standards.

This is how manufacturing plants work on a daily basis. Now, working with that is something known as quality assurance. Now, quality assurance is the process that analyzes what constitutes quality and how it can be measured and checked. So quality control is the actions that happen at the time of putting something together. So if you’re building a new piece of software, for instance, you might go through a quality control of that software. Quality assurance is the program around it that includes things like quality control to make sure we reach a certain level of quality. Now, when you’re dealing with QC and QA, a lot of times this will take the form of something known as verification and validation, or V and V. This is especially true within the software development world. So when we talk about this, we have to first start with verification.

Verification is a compliance testing process to ensure that the security system meets the requirements of a framework or regulatory environment or that the product or system meets its design goals. Essentially, verification is a form of quality control, but we call it verification inside the digital world. Now, when we start dealing with validation, this is the process of determining whether the security system is fit for purpose. Does it actually meet the set designs we have and has it been installed correctly? That’s really what we talk about with validation. Verification, we might look at a piece of code and say, yes, this is doing what it’s supposed to. Validation is once we install that code, has it been installed properly, has it been configured properly and does it do what it’s meant to do? Is it fit for purpose? Now, fit for purpose is kind of a weird term and it comes from the idle world.

In the idle framework, the term fit for purpose is known as utility. And utility really is just defined as meeting the design needs of a software or a service for a particular use case. So if I have something like a word processor, it needs to be able to do those functions that I want a word processor to do. If it can do those things, it has good utility. It is fit for purpose. I can use it for the thing I want to use it for. Now the next two terms we’re going to talk about are assessment and evaluations. And these also sound very similar, but there is a distinction between them. When we talk about assessment, this is the process of testing the subject against a checklist of requirements in a highly structured way for measurement against an absolute standard. Now, that’s the key here. When we talk about an assessment, it is an absolute standard that we’re going against.

So often you’ll see these used with a checklist approach. Now an evaluation on the other hand, is a less methodical process that’s used for testing, that is aimed at examining the outcomes or providing usefulness of a subject being tested. So when I start talking about evaluation, evaluation is much more likely to use comparative measurements and is more likely to depend on the judgment of the evaluator than on a checklist or a framework. So a lot of times if you’re evaluating something, you’re saying, I think this looks good, when you’re assessing it, you’re going to have it against a checklist. So if you ever bought a used car, one of the things they sell you now with the certified pre owned used cars is we have this 83 point checklist that’s an assessment. They’re going to go down and check all 83 things.

Whereas if you took that same car to your local mechanic, he might do an evaluation for you and just look over the car, test a couple of things and say, yep, looks good to me. I think this is a good car. That’s the difference between an assessment and an evaluation. The evaluation is much more fluid. It is much more of an opinion based thing than a checklist based thing. Now, the next term we want to talk about is audit. And an audit is a much more rigid process than an assessment or an evaluation. In this case, the auditor is going to compare the organization against a predefined baseline to identify any areas that require remediation. Now, audits are generally going to be required in regulated industries. So if you’re dealing with the payment card industry or healthcare data processing, all of these things are highly regulated.

And so you’re going to go through either quarterly or annual audits. Now, when you go through those, they are going to do it not based on an evaluation, not based on their opinion. They may not even do it based on an assessment or a checklist. They’re going to have something much more rigid and they’re going to compare you against a known standard. Everybody needs to meet this standard, whereas an assessment, everyone needs to meet this standard for your organization, it’s more tailored to you. So that’s one of the differences between audits and assessments. Now we start talking about scheduled reviews. These are similar to a lessons learned review, except that it occurs at a regular interview such as quarterly or annually. Now, having a scheduled review is really important.

And when you go through a scheduled review, there’s going to be some things you’re going to be thinking about. You’re going to be thinking about the previous major incidents you’ve had in the last period. You might think about the trend and analysis of threat intelligence that might affect your company based on the cybersecurity posture. You might think about changes in additions to your security controls or your systems that may increase or decrease risk. And you want to think about the progress you made during the last period as you’ve adopted or updated different compliance within your frameworks. All of these are things you have to consider as part of your scheduled reviews to see if you’re on track to make progress towards your goals. Now, speaking of making progress towards your goals, one of the best ways to do this is through continual improvement.

Now, continual improvement is the process of making small incremental gains to products and services by identifying defects and inefficiencies for further refinement. Now, what this means is we are going to do small changes over and over, and all those small changes add up to big changes. Now, this is important because we can’t take on some big project and think we’re going to get it all done in a week. It’s just not going to happen. But if I could take that big project and say I have this goal of being here in a year, I can then break that down into what do I need each week to happen to get to that final outcome? And this is the idea of continual improvement. By making small changes over time, we can make big changes over a longer period of time. Now, there are many different continual improvement methods out there, including the Six Sigma method, the Deming cycle, the Lean method, and even the Seven Step improvement method within Idle for the CYSA exam.

Though you don’t need to know any of these specific methods, just know that it’s important to be continually improving in your organization. All of these methods do follow a similar concept of defining your goals, establishing a baseline through measurement and then analyzing it, putting it into practice with some kind of improvement action, monitoring that action results, and then measuring again. Making more changes and continuing through this incremental loop over and over and over again until you get to the final place you want to be, which ultimately is a great system that works perfectly every single time. Now, that’s the hope and the dream, but a lot of times we never get there. But we do get close. By doing all single improvements over time, they add up to big change.

6. Continuous Monitoring (OBJ 5.3)

Continuous monitoring. Now, in every organization we do some kind of moderate. Unfortunately, not all of us do continuous monitoring. Oftentimes an organization may take weekly measurements or monthly measurements or they’ll rely on assessment once a year. That is not continuous monitoring, that is monitoring over time and it’s not nearly as effective. So instead, we want to focus on doing continuous monitoring. Monitoring. This is the technique of constantly evaluating an environment for changes. That way new risks can be more quickly detected and business operations can be improved upon. By doing continuous monitoring, we are constantly looking at our systems and figuring out exactly what they’re doing and what they’re reporting to us. Now, continuous monitoring is an ongoing effort to obtain information that is vital in managing risk within your organization.

It has a lot of benefits. For instance, by doing continuous monitoring, you’re going to gain more situational awareness. You’ll know, what systems are in use, what systems are having issues, and how you can act on those quicker. Another benefit of continuous monitoring is you have the ability to do routine audits. Because you don’t have to wait for a quarterly or an annual assessment. You can do an audit whenever you need to. You can pull out the logs from that system, you can generate automated logs in that system and pull out that information and do the audit anytime you want. Additionally, you can have real time analysis by doing continuous monitoring. By continually monitoring those systems, you’re going to be able to do that real time analysis and see those alerts when things happen.

Instead of looking back a week, two weeks, a month, or even six months later. You’re getting real time analysis when things happen and that helps minimize your risk posture too. Now, continuous monitoring can help transform your organization from using reactive processes into proactive processes. This way you can get out ahead of these incidents and you can figure out what’s going on much more quickly. Now, one of the things I see a lot of organizations do with continuous monitoring that is a bad thing is they rely way too much on metrics and don’t have the metrics properly defined. When you’re looking at metrics and you start setting up things like a seam, that can have lots of different information being sent into it, but it’s not necessarily continuous monitoring if nobody’s looking at that data. So a lot of organizations will set up a dashboard. If you’re going to do that, you need to make sure that data is being sent in and it’s being analyzed and assessed. This way you can create actionable metrics that represent your risk to your organization.

Because certain metrics might be easy to collect, but they may not actually tell you anything valuable about the security of your network. So you want to make sure whatever you’re using is your metrics. These are things you’ve thought about and they’re actually going to have benefit to your organization. Because if you’re collecting data just to collect data, you’re really not doing continuous monitoring. Now an effective implementation and maintenance of a continuous monitoring capability is complex and time consuming and that’s why a lot of people don’t do it. For instance, if you want to have 24/7 coverage of your system, that means you have to have people who are working twenty four, seven, and that costs money.

And so these are things you have to think about in your organization is do you need continuous monitoring? And if so, how many hours a day are you continuously going to be monitoring? If you’re only going to have people there from nine to five, that’s only 8 hours a day. You’re missing 16 hours a day. So you’re really not doing continuous monitoring in that case. But if your organization determines that’s okay, and you’re willing to accept that risk because of your risk appetite, that is something you can consider. So these are the things you have to weigh and it’s all a risk management decision. Now one of the things that’s come out recently from the US Department of Homeland Security is a program known as CDM. This stands for the continuous diagnostics and mitigation.

Now it is required by the US government that governmental agencies adopt a program of continuous security monitoring. And to do that a lot of them are using CDM. So it’s important for you to understand this. A lot of us who work in cybersecurity do work for the government or for a government contractor. And so you may be using CDM in the real world as well. Now CDM is going to provide US government agencies and departments with capabilities and tools to identify cybersecurity risks on an ongoing basis. They’re then going to prioritize these risks based on the potential impacts and enable cybersecurity personnel to mitigate the most significant problems first. That is the whole concept of CDM. Now if you log in to look at CDM, normally the first thing you’re going to see is a dashboard. This dashboard is going to aggregate data and display it in a way that is useful for people at the agency and federal levels to make sense of.

This gives them that situational awareness we’re talking about. Then inside of that we have different tiers. Underneath that we have asset management. This helps answer the question, what is on our network? We have to know what’s on the network so we can secure it, right? Then inside of that we have who is on the network, which is identity and access management. Then we start thinking about what is happening on the network and that’s network security management. And then finally we start thinking about how is our data protected? And that is done through data protection management. Each of these circles is comprised of different systems and sensors that aggregate gate data together and then roll that up into this overall dashboard so that managers, analysts and executives can look at that information and know the status of their networks.

• Category: Uncategorized