The Complete Course from ExamCollection industry leading experts to help you prepare and provides the full 360 solution for self prep including AWS Certified SysOps Administrator - Associate: AWS Certified SysOps Administrator - Associate (SOA-C02) Certification Video Training Course, Practice Test Questions and Answers, Study Guide & Exam Dumps.

EC2 for SysOps

16. CloudWatch - Unified CloudWatch Agent – Overview

So let's talk about a way for us to collect metrics and logs from within our two easy Java instances. These are the Unified Cloud Watch agents. So this is for virtual servers; it could be for your EC2 instances or your on-premise servers, and you're going to collect additional system-level metrics such as the run processes, use of space, etc., etc. And you can also send the logs to Cloud Watch logs because, by default, if you launch an ETI instance, there will be no files or logs that will be sent from the ETI instance to Cloud Watch logs without using an agent. And the agent could be the CloudWatch unified agent. So, if you wanted to get a memory metric from within your IS two instances, the only way to do so was to use the Cloud Watch unified agent. Then, if you wanted to configure your agent, you could do so by using the SSM Parameter Store and storing the configuration in a central location, or you could specify a configuration file. Alternatively, you have your ECU instance with a Unified Cloud Watch Agent, and you want to send metrics and logs to Cloud Watch. So for this, you would just configure the agent and make sure you have the right permissions. This is also true if you wanted to use a server from within your own corporate data center. So on premises, you would still install Unified Cloud. Which agents? You would specify the Ionians, and then you would be able to push the logs and push the metrics. So they're important because you may be interacting with the SSM parameter store as well as the club wish logs and published metrics services. Then you need to be able to have the correction permissions attached to your Im role on your ETI instance or attached to your access keys that are deployed on your on-premises servers. Finally, any metrics delivered by the UnifiedClub or Agent begin with the prefix CW Agent. So this is going to be in this namespace; you can configure and make changes, but at least this is the default one. So something you need to know that comes up in the exam is that there is a Procstat plugin on the cloud with agents, and that means that with this Procstat plugin you're going to collect metrics and monitor system utilisation of individual processes running on your Linux or Windows servers. For example, you will get some information around how much time a CPU will be used by a process, how much memory a process will be using, or the processes that are running directly on your EC2 instance. So you can select which processes to monitor by PID file. So you can get the PID from the process ID number, or you can get the name that you have for the process or the pattern. If you wanted to filter the processes to monitor again, as well as all the metrics related to the statistics of your processes on your Linux or Windows servers, they would begin with a prostitute prefix, such as proxy that underscore CPU time, proxy that underscore CPU usage, and so on. So again, we need to remember out of this that if you wanted to get some information about the processes running and their associated metrics, the only way to do it would be to use the Unified Cloud Watch agents deployed on your two instances and configured to use the plugin. So that's it for this lecture. I hope you liked it, and I will see you in the next lecture.

17. CloudWatch - Unified CloudWatch Agent Part I

So let's go through the new Car Watch unified agents, and this is a new kind of agent which does metrics and logs at the same time, so it will be a great time to look at this agent in details. So we're going to launch an instance, and I'll use Amazon Linux 2, and I'll select this one; I'll use a T-2 micro, and then for the instance detail I will create a new IAM role, and for that IAM role, I'm first going to create it and say it's for Easy to click on Next permissions, and in there I'm going to search for policy, and it's called Cloud Watch, so let's, so let's, Cloud Watch, so let's I think this one will be great; let's take a closer look at it so it has event logs. Create a log stream and create log groups so it will allow us to log stuff, and it also has the Cloud Watchput metric data so it will allow us to send metrics. Custom metrics into Cloud Watch, but it also has the SSM get parameter, which we'll see in this lecture about how we can fetch Cloud Watchagent configuration from SSM, so this is ideal. We'll use this one; let's click on Nexttag and Next review, and we'll call this one the AWS Cloud Watch role for EasyTo. OK. This is great Create the role and let's verify that everything is correct, so I'll go to this role and the policy is there, so let's attach this role into our instance, and here it is, and for now we will not do anything else; we'll keep this. We won't use tags. Let's just tag let's just name our easy toinstance logging instance because it's going to log somethingand configure security group this is fine. We're doing launch and launch, and I'll use this keypad yes to launch my instance, okay, so our instance is launched, and we're going to install Apache on it to just have a simpleweb server, and we'll stream the log of Apache onto CloudWatch. So back on our instance, let's connect to it using Easy to Instance Connect, and that's going to be simpler, and from this I'm going to install Httpd, so for this So if we go to our instance in here and then to our security group to the inbound rules, we're going to add the HTTP rule just to test whether or not Http was correctly installed, so here we go, this is our rule that has been edited, and back into our logging instance, I'm going to copy the publicDNS, and it says it's not for shoes because we haven't started it, so that makes sense. So we'll do pseudosystemctl httpd starthttpd once more. Start HTTPD, and now this should have started my Apache server, so if I refresh this page, it now says Hello World. Okay. So everything is good. We've installed Httpd, and now we'd like to trim the log from VAR Log, and in there oops and in there we have different logging available to us, but the one log we're interested in is Httpd, and then we have an access log, for example, and we can also get an error log if we wanted to in here. So these are two kinds of logs that we'd like to have in Cloud Watch. Okay, so let's get back into Cloud Watch. And so the first thing we have to donon Cloud Watch actually but the first thing wehave to do is to install the Unified CloudWatch agent to get started with Cloud Watch logs. So there is this new thing, which is that previously there was a Cloud Watch agent for logs specifically, known as the Cloud Watch logs agent, as well as a metric system and a script to send data from your easy to instance, such as RAM and disc info, into Cloud Watch metrics. So now they have created something called the "Unified CloudWatch," which allows you to upload both sendmetrics, customer metrics, and logs into Cloud Watch. And the cool thing with it is that you can store and retrieve its configuration into the SSM parameter store, and that will allow you to have a quick setup for all your instances if you want to have them all configured the same way. OK, so what we'll do is go through the pain of configuring it. If you go through the documentation, you'll find it's relatively complicated to see how to get started, but thankfully you've done this before, and I've just summarised the information on how to do it here. So the first thing we have to do is go to Webgets and download this file. So let's get back into our EC2 instance connection. I'm going to exit the root user and I'm going to issue webgets to download this Amazon Cloud Watch agent rpm file. After that, we need to install the Cloud Watch agent, so we'll do pseudo-RPM u and then examine the downloaded file. So here we go. Now it's doing this, and finally we need to run the wizard, and the wizard will go through the configuration of the Cloud Watch engine for us. So let's launch the wizard and see what is happening. So it's saying which OS you are planning to use for the agents, and we are planning to use Linux. The reason why it's not smart, quote unquote, is because you're using Linux. So obviously you want Linux so that we could configure the Cloud Watch agent for Windows directly from the Linux machine, and that would be fine. So that's why, but here we're doing Linux, so one, and are you trying to fetch the default region for the estimated metadata? Are using EC Two or onpremise host whilesaying okay, we're using EC Two, which useris planning to run the agents as of? So we can use Root, CW Agent, or others? I'll use root. Do you want to turn on the stats demon? So this daemon is used to collect StatsD metrics from your applications, and this is something that could be quite nice because if your application is a StatsD endpoint, then the Cloud Watch Unified agents could send this directly into Cloud Watch. So we'll say yes, although we will not use this and so on. What part do you want to use? 8125 is absolutely perfect. Okay, now what is the collect interval for the stats? The demand will keep it at 10 seconds, and what is the aggregation interval? We'll keep it at 60 seconds. Okay, do you want to monitor metrics from Collectde? So this is another kind of daemon we can collect metrics from, so we'll say yes. And do you want to monitor any host metric? For example, CPU, memory, etc. Yes, we definitely want to have CPU and memory. Do you want to monitor CPU metrics per core? And this is something we'll say yes to. So here, thanks to these Cloud Watch Unified agents, we're able to get CPU metrics not just at the aggregate level, the way we see it in the Cloud Watch console, but per core. So this is great. And do you want to add EC two dimensions, for example, image ID, instance ID, and so on, into the metrics? We'll say absolutely. And then, do you want to collect your metrics at a high resolution, even down to the minute? And this is something we've seen for customer metrics, so we could be every 1 second, every 10 seconds, every 30 seconds, or every 60 seconds. And to make sure that we don't overpay, we'll just keep it at 60 seconds. But these are all the options that we have because we have a custom metric. What default metrics for conflict do you want? one basic standard in advance. And I'll just ask you to refer to the documentation for this. But we'll just keep it with Basic, and this shows us the configuration that we have as follows for now: so we're saying okay, the agent is collecting every 6seconds run as root and here's the dimension that weneed and here are the metrics that we collect andso on and we're going to collect the disc andalso the memory and the memory used person. So we'll get some RAM information into Cloud Watch logs and so on. So are you satisfied with the config? We'll say yes, and now we're done. Now, I said, do you have any existing Cloud Watch logs? This is the old agent because if we had the old agency, we would be able to import a configuration file directly into this unified agent, but because we don't have one, we'll just say no. And do we want to monitor any log files? We'll say yes, and log file files are going to be VAR log httpd access underscore log, and this is going to represent the access log of our Apache server, and the log group name could be access Lugs. Excellent, we'll just keep it simple andthe log stream name will be InstanceID. Perfect. Do you want to specify any other files? Yes, we will absolutely say yes, and this time it will be VAR LIGBHT Httpdairslug. Excellent, and we'll keep everything as default. Do you want any additional files? I'll just say no, and this should make it fully ready. So here we go. So we have the entire configuration here, and as we can see, there is a logged part in this JSON configuration, which is fine; you should collect these files, and the collect list that you need to have is these file paths with this group name and stream, so that everything can be edited later. Right? So please check the above content of the configuration, and the configuration is also located in this file name, so we can go ahead and copy this. Excellent. And you can edit it manually if you need to. And the question is, do you want to store the config in the SSM parameter store? and the answer is yes. What parameter store name do you want to use for your config? If you use the manage policy, you must also use Amazon Cloud Watchprefix. So Amazon Cloud Watch Linux is a great one, and I don't think it's going to work unless we have the right item policy for this. So if you go back to the cloud, agent server policy allows us to do a get parameter as well, but we need to edit this to also do a put. So what I'm going to do is attach the policy, and I'm going to type a Cloud Watch again, but in here we can do a Cloud Watch agent admin policy, and this admin policy does everything as before, but this time it allows us to do a put parameter as well on the SSM parameter store. So let's select this one, click on Attach Policy, and now this is good. So now if we go back to our instance, go ahead and press Enter to keep this choice, and say okay, EWS one is a great place to get started, and the address credentials you should use are the ones directly from the SDK, so we'll keep it as one. Here we go. It says I successfully put config in the primary store. The Amazon Cloud Watch Linux programme exists now. So I'm just going to pause this video right now. and I'll see you in the next video to see what happened.

18. CloudWatch - Unified CloudWatch Agent Part II

So we are back in the second part of the Cloud Watch unified agent configuration. So we just put the parameter into the perimeter store. So let's have a look. We're going to SSM for a look at the perimeter store. So I'm going to open SSM, which is AWS Systems Manager, and we'll see Systems Manager in depth in the future section. So here we go. We have parameters in here, and I'm going to look at this one called Amazon Cloud Watch Linux, and we can look at the value. And this is the massive JSON that was created by the wizard and is inserted into this parameter. And why do we have this? Well, it's pretty obvious because we want other easy-to-boot instances to boot up and directly fetch the value of this configuration and use it for the Cloud Watch agent configuration. So that makes a lot of sense. So how do we use this? Because right now, if you go to Cloud Watch, for example, and refresh, I mean, nothing happened; we don't have any logs, and if you go to metrics, we don't have any metrics. So how do we start things, and how do we start things to point at this new SSM parameter? So for this, it's pretty easy. You have the option of two options, okay? You can either boot your Cloud Watch agents from an SSM configuration parameter or name, or you can directly boot them from a file in the file path. So we have two options: either we use this file right here that was inserted into our operating system. So we're saying, OK, look at the content of this file and use this file to boot up." That would be the second command. Or we'll do like the cool kids do: we'll use this one, which is the one you need to start Oops, you need to start the Cloud Watch Agent by fetching a configuration. It'll be a module for easy two, and the parameter store name will be, which we'll need to insert from the one we had in here. So I'm going to go to Systems Manager, copy this, and then come back here and right-click, but it's not working. So let's do it the old way. We're just going to write it out. So it's called Cloud Watch Linux. So, here we go with Amazon Cloud Watch Linux. And so we are getting it; it says okay, successfully fetched the configuration, and saved it into this temp file. Then it starts to configure and validate the configuration. It says it's as good and the validation phasehas succeeded and the second phase has not succeeded. The reason for this is that we are missing a file called types DB. So what we'll have to do is create this file. So I'm going to do "make directory p," user share c," and "collect d." So this is going to create the right directory for me, and I'm going to use this as pseudo, and then I'm going to do sudo, touch User Share, and collect D, and then finally type DB just to create this file. So now this file has been created. And so now hopefully, we can go ahead and restart the agent. And now, yes, the agent has been working, and everything should be started. So now we should start seeing the cloud watch logs and the cloud watch metrics in Cloud Watch. So let's have a look. So I'm going in Cloud Watch, I'm going to refresh this. And now I see my access log and my error log and the log groups that have been created. So here I'm able to see, yes, the mylog stream, which represents the access log that I have in here from my Apache server. And I could go, obviously, here and look at the error log. Excellent. so this log stream as well is great. And then finally, if I go to Cloud Watchmetrics, I could look at CW agents, which are clutch agents, and get some custom metrics. For example, I can get disc percent use, so I can see how much of the disc is being used by every different type of directory. But I could also go one up and look at this one. And here, this shows me the RAM I use. So here, 12% of my RAM is used. And again, this is a customer metric that has been inserted by my UnifiedCloud Watch agents into Cloud Watch metrics. So, really cool. I know it's a long configuration, I know, but the idea with this new Unified Agent is that the configuration itself is stored in the parameter store and can be changed, obviously, whenever you want. And all of the instances that are using simple user data would simply fetch this configuration directly from SSM and use it to configure the Cloud Unified Agent, which is a little bit easier to do than going ahead and configuring this file to appear on your instance magically every time. So I think this is quite an improvement from Amazon, especially as it provides us a lot of different metrics and a lot of different facilities for luck files and so on. Finally, you may ask me what metrics can be collected by this new Cloud Watch agent. Well, if you scroll down, you can see that there are a lot of different metrics that can be collected by the Cloud Watch agent. could be around the CPU, so CPU time, active guests, and so on. So we get some CPU information, and a lot of those actually give us some disk-free information. So how much disc is being used on my machine? That is interesting. So disc free, disc use as a percentage, IoT time, and so on. And if we scroll down, we get some information about the memory. So this is Ram. So how much is active and how much is available? Available as a percentage off, and so on. And we get some information around the network interfaces as well, so we can get some information about the number of packets sent and received and so on. And if I scroll down, we get some information about the processes: how many are dead, blocked, idle, paging, running, stopped, and so on. So we learn a lot more and finally swap free and used people. So a lot of new metrics can be collected by the Cloud Unified Agents. And for that, it would be super simple. You would need to go into the parameter store and edit the configuration and add on these metrics that are collected by the Cloud Watchagents, and you'll be all done. So that's it for this lecture. We are pretty happy with the state of it now. We have two log groups to work with. So, until the next lecture, we'll see you.

19. EC2 Instance Status Checks

So, let's look at status checks. So, status checks on your EC2 instances are automatic checks done by AWS for you that will identify hardware and software issues with your two instances. So there are two types. The first type is a system status check, which is a monitoring of problems that may occur with a reserve list's systems, such as a software or hardware issue on the actual physical host that your issue instance is placed on. For example, if your host lacks system power, and so on. To get an overview of these issues, a place to look is the Personal Health Dashboard, which will give you information about any scheduled critical maintenance by Italy that will affect your HTTP instance host and therefore require you to take on it.So a way to act on it is to stop and start your instance. So let's have a look. As we can see here, we have a host; this is hidden from us, okay? But this is what happens in the data centre of AWS, and our EC2 instance is launched onto this host one. If AWS experiences a hardware failure, this system status check will change from zero to one. And what we have to do is stop and start the EC 2 instance. So, whenever you stop an easy instance and then restart it, So not a "rebootlook at status checks. So, status checks on your EC2 instances are automatic checks done by AWS for you that will identify hardware and software issues with your two instances. So there are two types. The first type is a system status check, and this will be a monitoring of problems that will happen with the systems of a reserve list, for example, a software or hardware issue on the actual physical host that your issue instance is placed on. Or, for example, if your host has a lack of system power and so on. To get an overview of these issues, a place to look is the Personal Health Dashboard, which will give you information about any scheduled critical maintenance by Italy that will affect your HTTP instance host and therefore require you to take action. So a way to act on it is to stop and start your instance. So let's have a look. As we can see here, we have a host; this is hidden from us, okay? But this is what happens in the data centre of AWS, and our EC2 instance is launched onto this host one. Then say that AWS has a hardware failure, then this system status check will go from zero to one. And what we have to do is stop and start the EC 2 instance. So anytime you stop an easy instance and then you start it again So not a "reboot," but a "stop and start." Then internally, what's going to happen is that your EC2 instance will automatically be migrated to a different host within AWS, which also explains why it will get a new public IP because this will be the public IP of the host you're on. Okay? So this is a different host migration. So it goes from host one to host two. And because we just stopped and started, then we know that it's moved and host two does not have any issues, and therefore we have solved our hardware failure problem. The second type of problem is an instance status check. This is to monitor the software and network configuration of your instance. For example, the network configuration becomes invalid, there is exhausted memory, and this kind of stuff. In which case, to just resolve this issue, you should just reboot the EC2 instance or change the instance configuration. So these are the two types of status checks on your simple example. But there is a way for you to automate the recovery and look at metrics. So how much metrics are systemscheck failed, system status check failed? instance or status check failed if you want to regroup these two issues into one metric. So option one is to use a cloud alarm to recover your instance. So there's an action called recover instance, and when it's being used by a cloud alarm, it will recover the instance by using the same private IP, the same public IP, the same EIP, the same metadata, and the same placement group. You can also send a notification becauseit's a cloud alarm likely to SMS. So your easy-to-instance workload will be monitored by cloud watch metrics. For example, for the status checked failed system metric, the cloud watch alarm will be triggered on top of your cloud watch metric in case it goes to one. And the action of your Cloud Watch alarm will be to recover your EC2 instance by placing it somewhere else. You can also send a notification to Amazon SNS. As I just said, option two, which is a bit less conventional, is to use an auto-scaling group with a min and max of one and a health check for the status check of your EC2 instance. What will happen is that in case there is an issue with your EC2 instance, it will be terminated by your auto scaling group, and therefore, because we have a main maximum and desire to run a new EC2 instance, it will be launched within your same auto scaling group. So in this case, you don't have the same EBS volumes; you don't have the same private IP; you don't have the same elastic IP, but at least your instance is back up and running. And if you automate things well, then you could maybe restore its state. OK, so these are two options. Obviously, option one is going to be much preferred if you have an emphasis on one specific, easy instance. So to save this lecture, I hope you liked it, and I will see you in the next lecture.

20. EC2 Instance Status Checks Hands On

Let's have a play with status checks. So if we go into my first instance and lookat the status check tab, as you can see, wecan see that two status checks are being run. So there's a system status check and an instance status check. OK, and in case you are not happy with this check and you believe that there's an error, you can click on Report, Instance, and Status to help address detected issues. So what we can do, though, is because everything's running, we can still go ahead and create our cloud watch alarm that will have a reboot or recovery action on our instance. So I'll take action, and then I will click on "Create status check alarm." So we create a new alarm, and then we have an alarm notification so that we can send it to the default cloud, which is alarms topic if you want a notification, but you can disable this as well, and an alarm action. So what do we want to do when the alarm is triggered? And we have two options that can be very helpful for us. So recover or reboot. So Recover is going to be very helpful when we want to recover from a physical host issue from AWS, or Reboot when it's a software issue. So I will choose "recover" and then we need to look at the alarm threshold. So we want to have the status check fail, for example, if it's either both or just the instance or system. So based on what you want, for one consecutive period of five minutes, here's the alarm name, and here's a sample of metric data. So we can see we are at zero because the alarm state hasn't been triggered. But if there was an issue with the status check, then this would go to one, and so for one consecutive period of five minutes, this would trigger the alarm. So I will click on Create, and something is wrong because it can only be done on the status check system that failed. So let's go on to status checks failed system. Here we go. And now we have the recovery action. Click on Create, and now this collaboration's alarm has been created. So what I'm going to do is go into my Cloud Watch alarm. So let's click on Cloud Watch alarms here, and yes, I'm going to go directly into Cloud Watch alarms from here, and we can see that we have one alarm right here that has insufficient data. So very soon it's going to be okay. So let me wait until it is okay. And my alarm is now in the "okay" state, so I can click on it. And as you can see, the actual instance metric value is zero, but we need a threshold of zero to 99. So, one for the alarm states. So we can do though is that we cansimulate a failure of this alarm to go intothe alarm states and see what happens. So if I scroll down, as you can see, we have the history of the actions of the alarm. So as you can see, we created it, and then it went from "insufficient data" to "okay," so let's issue an API call to make this alarm go into the alarm states. So I'm going to click on Cloud Trail to open a CLI directly from within the cloud. That's going to be properly configured, and that will save you some time. But you can use the CLI on your own terminal if you have configured it in the BT. Okay, so what I'm going to do is launch a Cloud Watch alarm and set alarm states. I'm going to look at version two. Here we go. So this is how you run the alarms. So we need to give the alarm a name, the state value, and the state reason. and the state value is alarm. So let's go into Power Shell. So first, let's get the alarm name. So the alarm name is right here. So I'm going to copy this. So I will type in this cloud watch, set alarm states, and then the alarm name is the one I just copied right here. And the alarm state will be alarm. Sorry, the state value is alarm, and the state reason So let's just change this. So the state value is alarm, the state reason is, and I'll just say testing, and the action is to press Enter. And this will set my alarm to the alarm setting. So let's refresh this. And as you can see now, the alarm is an alarm. And so if you scroll down and look at the actions, there's going to be a notification, and the alarm went into the alarm state. And so, if you look at the history right here, So here's what I want to show you. Sorry. So the alarm went from "okay" to "alarm." And then two actions happened. There was an SNS message sent to an SNS topic right here. And also, the second action that was executed was a successfully executed action that was easy to recover. So my EC2-two instance right here has been recovered thanks to this action. We don't know how it was recovered. Okay, but as you can see, we havethe alarm status, one and one in alarm. The issue instance was then recovered. So we'll take a bit of time to be recovered entirely. But at least it shows you that when this alarm was being triggered, then the recovery action was being launched. Okay, so that's it for this lecture. You could also launch another, create another alarm as an exercise, and put this one on the instance, and then reboot these two instances as an action, and you can try it out. Set the alarm site as well. But for now, we're good to go. What I'm going to do is just delete this alarm right here. I can close my shell. don't need it anymore for now, and then I'm good to go. That's it for this lecture. I hope you liked it, and I will see you in the next lecture.