CompTIA Linux+ XK0-005 – Unit 08 – System Maintenance Part 4

30. Topic C: Performance Monitoring

All right, so now we’re going to look at some of these proactive things we can do to make sure our system is running well. Now, I keep emphasizing proactive. I’m big into that. I don’t like waiting till there’s a problem and then trying to figure out what’s happening. I want to do something to look for potential problems to be on top of the curve. It makes you look better if you’re on top of it when somebody calls, not this, oh, really? It’s not running well.

Let me go take a look. Or not being able to come up with an answer. So performance monitoring is a start. And I’m going to emphasize right now that when you do performance monitoring, it should be frequently done, especially during times when it’s running well. So you can create a baseline, meaning that this is what it’s normally supposed to do. So when there is a problem, you can look at the performance and compare them and see what has changed. That could help you indicate to you what the problems are.

31. Monitoring Tools

Now, there are a number of different tools that you can use. Some come with a package that you can install called the Sysat package. Now, if you use the Sys Stat package, you’re going to get a couple of tools like the MP Stat that’s good for monitoring the statistics of multiple processors. The IO Stat, which gives you the I O information for each disk. And then this thing called SAR, which is the system activity monitor. Now, there are some other commands that you’ll already have, like Free. It’s a command that shows you your memory and your swap space.

The top command so that when you start looking at your statistics, you can see the top CPU stats or the top memory or top swap, rather than having it in some alphabetic order by the name of the counter to actually put it in an order that makes sense to you. Up. Time to know how long the system has been up since last reboot VM stats for your virtual machines again or not your virtual machines, your virtual memory swap performance. All of these are tools that allow you to look at and see at that time that you ran it, how things are running.

32. Demo – Installing the sysstat Package

Well, we are going to eventually want to do a little bit of monitoring with some utilities that fall under the package of Systat. Well, that package is not installed, so we’re going to go to the Synaptic package. Package manager. You know, I keep telling you, it’s going to get harder and harder as I go to keep saying that over and over, and I don’t know why. But anyway, I could search through here if I wanted to, trying to find figure out, well, where would that be? Library, what, all that sort of stuff. As we’ve talked about before. Remember, click on Search, put in what you’re looking for. CIS Stat. And there is Sys stat right there. Market for installation. Apply your changes and wait for the progress of that file.

Hopefully that one file won’t take very long for it to install. And then we’ll check to make sure that it actually installed by trying some of the commands that might come with it, like MP stat and iostat. All right, so that’s done. We’re going to close that, close this down and very quickly, we’ll just go ahead and pull out the terminal and let’s do a which MP stat. There’s the location. We can say, what is MP Stat? There it is. The report. Processor related statistics. And that was a program, by the way, that wouldn’t have been there right now without having installed Systat. So it looks like we’re good and ready to go out and start doing some monitoring with some of those new programs by installing that Systat package.

33. mpstat

So Mpstat, as we said, was a tool that could help you with multiple processors to look at the statistics. Now, remember that when we look at a processor that we are concerned with the percentage of use, in other words, how busy is it being used? Remember, we had processes that could be utilizing your time and we often broke them down by system and user, system being the kernel running processes and user being the applications that you’re running. You also want to look to see basically an idea of what’s in line, if there are things waiting to be run, sometimes even called threads. But it’s a way of being able to say, okay, I can see how busy something is watching this information. But remember, it is a snapshot. When you run MP stat, it looks at the processor, gets the information right then, but it could now be different.

So you can also choose a time frequency. The time frequency is how often do you want it to go out and ask the processor how busy is it being? You can set it for every second, every 5 seconds, every 60 seconds, so that it keeps refreshing the screen for you. So you kind of have a good picture of what’s going on. Just because the processor is pegged at 90% is not a bad thing. But if every 5 seconds I’m checking, it’s still at 90% and it’s been that way for a few minutes, now we have a problem. I expect processors to hit 100%, open up a new program, boom, it spikes up, and then you’re at 100%. And then you’re back down again because the program is open and now everything’s running. So it’s one of those things that you do want to let run for a bit so you can get a good idea about how busy things are.

34. iostat

The IO stat again was talking about your drives, your input and output. How busy are they? Unfortunately, as I’ve said before, your hard drive is the only moving part on your machine. That means it’s generally a bottleneck if you have a lot of intense reading and writing applications, like a database application where it’s constantly updating a database file or even log files if you have running with not enough memory now, here what I’m saying. I’ve talked about a hard drive, but now I’m talking about memory.

If you don’t have enough memory to be able to supply all of your applications that are running with what they need, then the Virtual Memory Manager is going to be busy swapping things from hard drive to memory. That’s going to cause a high amount of IO going in and out of your hard drive, which is going to slow the system to because it’s not anywhere near the speed of memory. So that might be an indication that your problem isn’t the hard drive, but even it could be the memory. So looking at the I O stats can help you make some decisions about what’s happening. Is it normal? Is it within acceptable range? If it’s not, what might be the cause?

35. Demo – Monitoring with mpstat and iostat

Well, we installed a package called Sistat so that we can actually have the chance to run a program called Mpstat. Now, Mpstat was designed to kind of start telling me about some of the values of the processors. I know, as I said, that in plural, if I had more than one that are running, in this case all my CPUs, which is just one percentage of user time, system time, the IO waste eight process IRQs. It’s really look at that. Idle time, 94. 91. This processor is doing nothing except for giving you information on my screen. So that was pretty straightforward. But it’s a way of gathering information. And don’t forget, you can collect this stuff through a script schedule. It all sorts of great things. Now, let’s try iostat and with the dash c, so we can view the CPU statistics. And again, it’s showing pretty much the same type of stuff that we saw before with input output stat.

The nice thing with Iosat, though, it has a dash D to look at the device, the hard drives, and how busy they are, or if you want to see both just iostat and get both of the blocks of information, the CPU and the rest. So again, it’s just a nice way for you to monitor how things are performing. If you were to see a very low idle time and a high system time, it might let you know that your CPU was being overloaded. If you had a very high IO weight as well on your CPO, then it’s basically waiting too long for the other streams, or what we call threads of executable code coming into the CPU. And of course, we could also take a look at how busy the drives are as far as how much they’re reading and writing and all of that sort of stuff. So it’s something we do often to just verify or see where some of the bottlenecks might be, or basically to see the performance of our server.

36. System Activity Reporter

The SAR System Activity reporter is another one of those pieces of the Systat package. Again, remember, if you want these tools, you can just download or make sure you install Sys Stat as a package. And the system activity is something that’s going to basically tell you what’s going on. It’s even a job that you can schedule with Cron so that it can gather information at different times of the day. It saves the output of that to a file which will be be found in the VAR log SA by default. And you’ll see the files as each time it runs, saving it as SA One, SA Two, SA three. But the goal is that it can look at how busy what’s happening on this machine at these different times of days to kind of give you what I like to call that baseline, what is the activity, what’s it been doing, what’s going on, and you can see the changes by having a history of these SAR packages.

37. Output of sar

If you were to open up this SAR file, what you would see is information about, like, the CPU. If part of the SAR package is to look at the system CPU utilization, then you can see the percentage by user, the percentage by system, the IO waiting, what we sometimes call the threads that are waiting in line, trying to get in, how long it’s been idle or what percentage it’s been idle. It gives a breakdown. And because you can run it over and over again, right, for a length of time, you’re going to be able to see each SAR as it ran and how many times it grabbed that information. So you get a kind of a brief history.

38. Demo – Monitoring System Activity with sar

Okay, now we’re going to take a look at another one of these system activity reporters called SAR. And what I’m going to do is hit the manual on SAR so you can take a look. It collects reports and saves activity information. I’m going to hit page down. The A would be the all flag, which means it’ll do all types of activities, b for input output, and you can see the rest of this as you go through. There’s a lot of options. Well, one of the things I’m going to do with SAR is come up with the option of saying, I want to view every 2 seconds six rounds of activity on the CPU. And that’s what I’m saying, every 2 seconds and do that six times and report on the activity of the CPU. So as I enter that command, you can see every 2 seconds, here comes how busy, or not so busy is my particular CPU.

I can do the same thing with the dash B, which again, as we saw from the man file, is saying that every 2 seconds. I want to look at the statistics of the hard drive and do that six times. All right. So you can see that one of the nice things about being in a lab environment that you don’t actually have a server that’s running out there on the network on the Enterprise. So it’s pretty quiet. We’re going to use the Cat command to see kind of what the default settings are for the SAR command and then we can find that in the Sys stat file. And there we can pretty much see that the options that we see, the additional options to be passed in there look like the D, no secondary options for SA two. And it’s just kind of letting us know that there’s really not much going on.

So what am I seeing here? I’m seeing that when it’s initialized that I’m going to know where the path is and that it’s going to give me the D option without my putting anything else in. And if there are any additional options passed in there, we don’t see it here, especially on the daily running of the Cron job for Systat. So again, remember, these are remarks that are being passed to you when you read the file. They are REMS, but the actual commands are what we care about the stuff without the pound signs. Okay. And again, these are the default settings for the starting up of Systat, for the scheduling of Systat, if you should schedule this type of a job, and for the daily Systat files so that you can see them broken down and how they’re going to be used. In this case, the activity information by default is enabled to be off.

So there is not an initialization, it’s not going to run daily. And on it goes. I think you kind of get the idea that it’s pretty much disabled by default. Okay, the other thing you can do again is cat the settings etsy for the Cron, the scheduling of Sys Stat. And there we go. And again, as I look at these particular settings and that kind of scrolled right on by very quickly. But let’s come down here to where I just typed in that last command. Let me clear the screen and do that command again because sometimes it gets just a bit confusing. There we go. And again, the defaults of where it’s located, it’s not on by default. And if it was the activity report information, that would run every ten minutes or every ten minutes every day, and the rotation of the files, the stat files. So if you did schedule it, then you’re seeing kind of what the Cron scheduling would be if that’s something that you need to use.

And then finally, if you want to look at the contents of the script which tells about how this thing is running. And boy, I tell you, this is going to just be a little bit messy. So I’m using the less command, so I don’t run about running out of space with moving up and down the pages. And let’s just look at the SAR one. Here the SA one and I’ll hit enter. And here it’s telling you the collect and store binary data in my system activity file, the SA one file. And here you can see kind of the scripting and the programming that would go through and putting that together. So, one more time, I’m going to encourage you that if you are interested in the scripting and you want to know what’s really happening in this if then statement and what the variables are doing and what these other collection objects are really get into studying how scripting works, because you can do a lot of cool stuff.

You could actually do this yourself if you knew the scripting languages to put it together. All right, so that’s anyway, just looking at and viewing some of the information of how SAR would go about collecting system activity, how it would run if you schedule it with Cron, how it runs as a default state and everything else, it is a cool little program to gather information. It allows you to see more than just one snapshot. As you saw. I can have it run repeatedly just on my screen and you can schedule it and just have a good way of gathering data so you can see how you’re performing and if anything, at least figure out how to troubleshoot why something is performing slowly. If that was the example complaint that you’re trying to investigate.

• Category: Uncategorized