Amazon AWS Certified SysOps Administrator Associate – Databases for SysOps Part 5

9. RDS Events and Logs

Okay, so now let’s talk about RDS events and event subscriptions. So RDS will keep a record of events related to multiple events related to the database instances, to your snapshots, to changes made to your parameter groups or your security groups, etc. So what is an event? Well, an event can be, for example, that a database state changed from pending to running. So we’re going to have RDS event subscriptions and to be alert of these events.

And it could be an SNS topic to get notified of some events or all events. And you can specify the event source and the event category if you wanted to have a more granular type of events sent to SNS. But also it’s possible for you to get these events coming out of Event Bridge. So it’s possible for you to set some rules and then the Event bridge will react to your RDS events. So here’s an example, for example, of an event. So it could be a backup event of your DB instance that will be sent to RDS event, which can in turn trigger a notification in SNS, or it can trigger an event in Event Bridge.

The second thing I want to talk to you about is around database log files. So your RDS database instance has some logs, for example, general logs, audit log, error logs, low query logs, these kind of things. So you can send these logs into CloudWatch logs and then you can apply on top of CloudWatch logs a metric filter. For example, you can have a look at the error keyword and in case it happens too many times or too often, then you can set up a CloudWatch alarm on top of it. So this allows you to do some log alerting. And then this CloudWatch alarm can send an alert, for example, to an SNS topic, and then this SNS topic can send a notification to the database admin. So this allows you to do some more alerting and some more eventing on top of your RDS database, based this time not on the database events, but on the logs themselves.

So now let’s go see in the console how we can set things up. So here you have on the left hand side the events tab, and in the events you can see all the events that happened within your account. So I think you get 24 hours of data within the console. And so the idea is that you can see, for example, that I created a database here, then some things were changed, a backup was initiated, then I deleted this database and then I created a new one. And so you can see all the messages right here. So it could be instance related type of events or snapshots related type of events, or it could be security group or a parameter group.

And you can see the message of all these events right here. So here’s a cool because it’s your Lug of all the events that happen within your RDS databases. And you can set up your event subscriptions, you can create one and you’re saying, okay, I’ll do a demo subscription and you could have a new topic to create, or you can set up your own ARN topic to send to your notifications and then the source of these events. It could be your instances and you could say, okay, all instances or some instances only, and all event categories or specific range of categories and you can be alerted based on all these ranges. So it’s quite handy and you can have instances, security group, parameter group snapshot, cluster, cluster snapshots and custom engine version as a source type of your event description. So you can get very creative and the destination of that is going to be an SNS topic. And alternatively you could go into Cloud Watch events or EventBridge and set up your own events there as well.

So if I go into my database I can have a look at the logs and events page and so we can see CloudWatch alarms related to My database, recent events from a database and logs in here we can have a look at the logs available to us. So we have the MySQL error log running log, the MySQL upgrade log and you can view watch or download so we can view this one for example, and we get some information around what happened, okay, and we can refresh this and this will display approximately 1000 lines of this log. And if you wanted to send these logs into Cloud Watch logs for analysis and to create metrics filter, what you could do is that you could modify the database and here you can modify the settings.

Now we have to find the Lug related settings so I will go ahead and scroll down when this is available. So here we go. And under additional configuration right here, I scroll down and I see that there is a logs export option so I can export the audit error general log and slow query logs. And if I enable this then this is going to publish those automatically to Cloud Watch logs. Okay? And then you could set up your own metrics on top of those to do some alerting based on some keywords or whatever you want to find or some analytics. Okay, so that’s it for the RDS events and logging. I hope you liked it and I will see you in the next lecture.

10. RDS & CloudWatch

So let’s look at RDS with Cloud Watch. And so as we know, CloudWatch is going to be obviously deeply integrated with RDS. And so we’re going to get from the hypervisor some basic metrics such as number of database connections, swap usage, read I ops, and write IUPs read latency and write latency, read throughputs, and write throughputs disk cue depth and free storage space. So from these, obviously we could do a lot of troubleshooting.

For example, if the latency is high, something is wrong. If the read I ops are peaking, maybe we’ve reached our eye ups for our EBS volume. If the disk queue depth is too high, that means that a lot of operations are waiting to be executed. We can also look at the CPU, ensure that it’s not too high, all that stuff. Right? So this is provided by CloudWatch basic metrics, and you can enable enhanced monitoring. And we’ll do this right away. And enhanced monitoring is usually metrics that are gathered from an agent that runs on the DB instance.

So it is more special, more specific. And what you get with it is that you get a lot of information around the processes and the threads that use the CPU and you get access to over 50 new CPU memory file system and disk IO metrics. So let’s go have a look at how we can enable enhanced monitoring right now. So let’s go to mydb modify, and in there we’re going to be able to enable enhanced monitoring. Because if you remember when we first created our database, we did not enable it.

So I’ll just scroll down and as you can see all the way at the bottom, we can enable enhanced monitoring. In there, you’re able to basically get more information and you need to define a monitoring role. We’ll just create default and RDS will create a role for us. And then you can say how granular you want your monitoring to be. I’ll leave it at 60 seconds. But as you can see, you can get per second granularity, which is quite awesome. And now you click on continue and apply immediately to apply the changes right away. And here we go. It’s working. So it’s just going to give you a little bit of a weird error message, but you just have to wait until everything is created. And now we’re done.

So if we go to monitoring in there, you’re going to get Cloud Watch so we can look at what Cloud Watch gives you. So it gives you CPU utilization, the number of connections that we have, the storage space, the right I ups, or read I ops, the memory that’s free. But then using basically enhanced monitoring, we’re going to get a lot more. So to view enhanced monitoring, pretty tricky monitoring. And then there’s a drop down. And here you can choose Cloud Watch.

So this will just give you the Cloud Watch information. But also you can choose enhanced monitoring and this will give you all the enhanced monitoring. So we have to wait a little bit until the agent starts up and you can also get the OS process list if you wanted to. So let me just wait a little bit for enhanced monitoring to kick in and I’ll be right back. Okay, so now if I go to enhanced monitoring, I start seeing some data. So as you can see, we have free memory, active memory, CPU, user load, average use file system, we have more graphs as well, available number of tasks running, et cetera. So enhanced monitoring does give me access to more metrics.

And if you go to OS process list, you get some information around which processes using whatever of memory and Ram. So this gives you a lot more information around what is happening on your database thanks to this enhanced monitoring. So that’s it going to show you this. But overall the idea is that the exam will ask you questions around how maybe a metric can impact your database and just use your common sense and just troubleshoot using this, using your brain basically. So that’s it, I will see you in the next.

• Category: Uncategorized