Amazon AWS Certified SysOps Administrator Associate – Monitoring, Auditing and Performance Part 3

5. CloudWatch Logs Hands On

Okay, so I am in Cloud Watch logs and we can see all the log groups we have right now. So we can see we have eight of them and they were created by some services. For example, this one was created by Lambda, this one was created by Data Sync, this one was created by Glue, and this one was created by us when it did do an SSM run command. And we wanted the output to be populated in this log group. So if we take a look at this example, for example, we have six log streams and so each of them represents a different instance that we did run a specific run command on.

So this is the same run command ID across the six. Here we have a different instance ID for each of the six. So two and two. And then we have STD out and STD air. So if we look at STD Out, we can look at all the logs that was generated by this command and we can have a look at all the log lines and so on.

So this is quite handy. And the idea is that from within the log, for example, you can look for the keyword Http and it will show you all the log lines that contain the word Http. If you just look for the word installing, for example, it will show you just maybe two or three log lines that contain the word installing. So that’s fairly handy.

And so we have Lug for STD Out, STD Air, so we can see really the idea behind different log streams. Now we can create metrics filters in here, and these metric filters is a way for us to find a filter pattern, for example, installing, okay? And then we need to select, for example, a custom data, for example, this log stream. And then we test the pattern and it’s going to give us three matches out of five in the sample logs.

Now, if you went ahead with entering this filter name, as you can see, I call it Demo Filter and Demo Metric Filter. And this is a new namespace, okay? And here is Demo Metric. So this is demotric filter namespace and this is a demo metric filter. And then the metric value when there is a filter pattern or match that occur. And so you can say one, for example, to add one and to count how many times this installing lines have been found and the default value and a unit if you wanted to, then click on Next Create, and this would give you a new metric.

So if you went into Cloud Watch metrics right here, and we’re going to clear this graph and we’re going to find a new metric. So let’s refresh this page. Maybe this is going to help us. Okay, so if we go to all namespaces, as soon as this metric would be Metric Filter would appear, it would appear right here and we could visualize it. But currently because we don’t send any log output, then we don’t see it.

But the idea is that we could create an alarm on top of this metric filter. So we can click on Create alarm and this would create allow us to create an alarm in case, for example, that metric went over a specific value. And again, this metric is calculated based on a filter from the log streams. We can also create subscription filters. So as you can see here, we can create a filter for different outcomes. So elastic search kinesis data streams, kinesis fire hose or a lambda subscription filter if you want to send data into custom lambda functions.

And we can create up to two subscription filters per log group according to this. Okay, now we can also edit the retention settings so we can see that the logs can never expire all the way up to 120 months. Okay, so ten years. And then we can also export the data into Amazon history. So you can click on Export Data. You can choose a range of data to export and then the stream prefix if you wanted to just get specific log streams and then the s three buckets and the bucket prefix and you’d be good to go. And then finally in here you can create a log group and I’ll call it Demo Log Group.

Okay, you can set up the retention settings, the Kms key if you wanted to encrypt that log group and then click on Create. And so the encryption setting would appear then here if a Kms key ID was specified. Okay? And then finally, CloudWatch Logs Insights, okay, is allowing you to use a nice query language to query some specific log groups. So for example, we can query this one and run the query and then this is not going to give us any data because we’re looking for data from the past hour, but if we look at data from the past 60 days and run this query, maybe we’ll find something. So as you can see, we found twelve records, 18 records from this query.

And so this gives us a nice query language to start gaining some insights on top of our logs. And on top of it you can export the results if you wanted to. And on the right hand side you can see that you can save your queries, okay, so you can query and save them here. Or you can look at some sample queries and view the use cases of losing log insights.

For example, view the latency statistics for a five minute interval on lambda or get the top ten byte transfers by source and destination IP addresses for VC for logs. So it gives you for example, if you click on these, some nice insights to how the query language works for Cloud Watch Logs insights. So specific cloud Watch logs, I hope you liked it and I will see you in the next lecture.

6. CloudWatch Alarms

Now let’s discuss clywatch alarms. So alarms, as we know, they’re used to trigger notifications from any metric. And you can define complex alarms on various options such as sampling or doing, percentage or maximum and so on. Alarm has three states, OK? It means that it’s not triggered. Insufficient data means that there’s not enough data for the alarm to determine a state and alarm, which is that your threshold has been breached and therefore a notification will be sent.

The period is how long you want the alarm to evaluate for on the metric. And so it could be very, very short or very long. And it can apply also to high resolution customer metrics, for example, 10 seconds, 30 seconds, or a multiple of 60 seconds. Now, alarms have three main targets. The first one is actions on easy two instances, such as stopping it, terminating it, rebooting it, or recovering an instance. The second one is to trigger an auto scaling action, for example, a scale out or a scale in.

And the last one is to send a notification to the SMS service, for example. And from the SNS service we can hook it to a lambda function and have the lambda function do pretty much anything we want based on an alarm being breached. So let’s talk about EC two instance recovery. We’ve already seen it, but there is a status check to check the EC two VM and the system status check to check the underlying hardware. And you can define a Cloudwash alarm on both of these checks. Okay, so you will monitor a specific EC two instance and in case the alarm is being breached, then you can start an easy to instance recovery to make sure, for example, that you move your EC two instance from one host to another.

When you do a recovery, you get the same private, public and elastic IP, the same metadata and the same placement group for your instance. And you can also send an alert to your SNS topic to get alerted that the EC two instance was being recovered. Now, the cloud alarm has some good stuff to know. The first of all is that as we’ve seen, we can create an alarm on top of a CloudWatch logs metric filter. So remember, the CloudWatch logs are having a metric filter which is hooked to a CloudWatch alarm.

And then when we receive too many instances of a specific word, for example, the word error, then do an alert and send a message into Amazon SNS. And so if you wanted to test alarm notifications, you can use a CLI call called Set Alarm State. And this is helpful when you want to trigger an alarm even though it didn’t reach a specific threshold because you wanted to see whether or not the alarm being triggered results in the correct action for your infrastructure. So that’s it for alarms. I hope you liked it and I will see you in the next lecture for some practice.

• Category: Uncategorized