Amazon AWS Certified SysOps Administrator Associate – Advanced Storage Section Part 2

5. Storage Gateway for SysOps

So here are some additional bits and pieces of information that could be helpful for the sysaps exam regarding the storage gateway. So the file gateway is what is pizza complaints because it is a Linux file system, that means you get information around the metadata, ownership permissions and timestamp store in the objects metadata in s three. If you wanted to reboot your storage gateway VM for maintenance, you have two ways of doing it.

If it’s a file gateway, you simply restart the storage gateway VM. But if it’s a volume or a tape gateway, you need to stop the storage gateway service from the console or from the VM local console, or using the storage gateway API, then you reboot the storage gateway VM and then you start again the storage gateway service, okay, using the APIs or the console.

So this is something to know then around how to activate your storage gateway. So, there are two ways to get an activation key for your storage gateway. The first way is to get it using the gateway VM CLI. And so when you start it, there is an option to just do zero get activation key and you pass this activation key directly into the storage gateway console. Or the older way of doing it is to make a web request directly into the gateway VM on port 80. And for this, you need to open the port 80. So this is a diagram of how it looks like.

So once the flag gateway has port 80 open and you can access it from the console, then you can activate it. Okay, two ways of doing it, and so activation failures can come up in the exam. So when would an activation failure happen? Well, for example, if you’re using the second method, you need to make sure that the gateway VM has the port 80 opened. And then in case, and it’s a very edge case, but you need to make sure that the gateway VM also has the correct time and it’s synchronizing its time automatically using a network time protocol server. And this is to ensure that the file gateway time is not too far off from the actual NTP server. So let’s talk about the volume gateway cache mode.

So in cache mode, only the most recent data is stored on your volume gateway storage gateway. And so you need to look at your cache efficiency. So the idea is that there is a metric in Cloud Watch called the cache hit percent metric and you want it to be super high. Because if you have a very high metric, for example 80%, that means your cache is efficient and that you’re using it a lot and you get lower latency. If it’s very low, then it could be a problem. And you also need to look at the cash percent used, okay? You don’t want it to be too high, so you want your cache to be filled accordingly. But if it’s too high, then you need to do something about it. So the second example.

So there is a cache volume gateway in your servers. And so when you have a cache hit, then your traffic stays within your corporate data center. But if it’s a Cash miss sorry. Then you need to get the data back from the SD bucket, and that could be a bit longer. So obviously, when the Van gateway is in cache mode, you need to make sure that you are efficient at using the cache. So in this example, in my diagram, the cache hit percent is going to be 30%. Okay? And so an option to increase the cache is going to be the Cash efficiency is to create a larger cache disk.

In this case, you need to clone the volume to a volume of larger sites. So first take the Cash volume and you clone it, and then you select the new disk as the cache volume onto the volume gateway. So it’s a small operation. You need to clone the volume and then take the new volume and assign it as the Cash volume for your gateway. So that’s it. Let’s a lot of details, you know, for this is up to exam, but you have no choice around it. So remember those. And I will see you in the next lecture.

6. [SAA] Amazon FSx – Overview

So let’s get into a newer kind of storage offering from AWS, which is called Amazon FSX. And Amazon FSX is awesome and it comes in two different flavors that you have to know for the exam. And so before introducing you Amazon FSX for Windows File Server, which is the first type of FSX, I want to introduce you to the problem. So remember EFS. EFS for Elite big file system. It’s a shared POSIX file system that means that it’s going to be used only by Linux easyto instances or on premise machines. Therefore you cannot use EFS with your Windows servers. So how do you share storage between your Windows servers? Amazon came up with FSX for Windows. It’s a fully managed Windows file system, sharedrive.

It supports two important things. Remember, it the SMB protocol and Windows NTFS. It supports Active Directory integration because it’s a Microsoft thing. It makes a lot of sense ACLs and user quotas. It’s built on top of SSD, it has a massive scale. It can scale to tens of gigabytes per second, millions of IOPS and hundreds of petabytes of data. So it’s a scalable file system for distributed file system for Windows that is managed by AWS. It can also be accessed from your on premise infrastructure and it can be configured to be multiad and gets high availability. Finally, data is backed up daily to Amazon is free.

So you can always recover your file system directly from SRate. So, Amazon FSX think anytime you have shared storage for your Windows instances. This is Amazon FSX for Windows. And there’s another Amazon FSX, which is totally different. I wish they’d named it differently. But anyway. So Amazon FSX for luster. And Luster is a type of parallel distributed file system for large scale computing. So you need to know what Luster means. Luster is derived from the term Linux and cluster.

So now it makes a lot of sense. So Luster is for Linux instances and because it comes from cluster, it’s meant for large scale computing. And I think as soon as someone teaches you what Luster means, amazon FSX for Luster is super easy to remember. So what do we use Luster for? We use Luster for for machine learning and high performance computing or HPC, and you will find HPC to be mentioned in the exam quite a few times. So anytime you need a file system to perform these highest edge performance computing, then Luster will be a good answer.

So we can also do video processing, financial modeling, electrodes and automation. Anything that requires a high level of distribution for your file system and your computation. So this is just a file system though and it scales up to hundreds of gigabytes per second, millions of IOPS and has sub millisecond latencies. So it is really meant for high performance computing or HPC. And it has a seamless integration with S Three. That means that you can read your S Three buckets as a file system through FSX for Luster and you can write the output of whatever computation you’re doing back to S three again through using FSX for Luster. So FSX for Luster is a way to expose your sree bucket as a file system as well to your Linux instances. It can also be used from on premise servers if you needed to.

So these two offering from Amazon Web Services on FSX for Windows and FSX for Luster, they’re very different, but you need to remember both use cases. So FSX for Windows is going to be around a distributed file system for your Windows instances and FSX for Cluster is going for you for Linux. And it’s going to be to have a cluster, a high performance computing cluster that has a file system that is shared with high IOPS, high throughputs, very low latency and integration with S three as a back end. Okay? For FSX you also need to know the file system deployment options. And there’s two you need to know. There is scratch file system and persistent file system.

So scratch file system is going to be temporary storage and the data will not be replicated. That means that you have a file and you will lose it if the underlying server fails. But thanks to this optimization, we get really high burst. So we get six times the performance of a persistent file system and you get for example, 200 megabits megabytes per second per terabyte of throughput. So it’s actually really, really big. So the use case of a scratch file system is going to do short term processing of data and you want to optimize your cost by not having data being replicated. So that means that you have FSX. Your compute instances are going to connect on AZ One and AZ Two.

And then the FXX for Luster scratch file system only has one copy of your data as is shown on this diagram right here. Just one copy. Okay? Finally, you can also have optional extra buckets underlying for the data repository for a persistent file system, it’s going to be for long term storage. The data is going to be replicated within the same availability zones. So not across AZ but within the same AZ. But that means that if you have a failure of an underlying server, then the files will be replaced transparently within minutes. So the use case for a persistent file system is, as the name indicates, long term processing and storage of sensitive data.

So the idea is exactly the same in terms of the architecture. Remember, FSX only lives for Luster only within one single AZ. And the FSX for Luster file system in persistent mode will have two copies of the data. So you can see there is some replication right now from one data volume to the next data volume. So that’s it. The exam will ask you a scenario question on how to choose between scratch or persistence. So you should know enough. Now, I hope you like this lecture and I will see you in the next lecture.

7. [SAA] Amazon FSx – Hands On

Amazon. FSX is not something we can play with because there is no feature for it. But I still want to show you through the console the different options. We have to play with FSX and see the kind of file systems we can create. So I am in us west two. Maybe I should select a region a bit closer to me. So I’ll select something in Europe. For example, Ireland. And here we go. So I’m going to create a file system and we have two options. We have FSX for Windows File Server and FSX for Luster. So let’s go ahead with FSX for Windows file server. And if I scroll down it tells you exactly what I told you right now. So don’t need to read it. Click on next. And we have to specify our file system. So FSX file system for Windows, do we want a multiaz or a single AZ deployment? This is for high availability, the storage capacity. So we can see here that we have to provision the storage in advance.

So the minimum is 32GB and the maximum is 64 terabytes. So I can say, for example, I want 32 terabytes of storage for my Windows and then the throughput capacity. So do we want to have the recommended throughput at half a gigabytes per second or do we want to specify the throughput and go all the way to two gigs per second or all the way down to eight megabits per second? And obviously the pricing will depend on these things but so as you can see, this FSX file system for Windows is not something that’s elastic, it’s something that we have to provision in events in terms of capacity and in terms of storage for network and security. We have to deploy this within our VPC and attach a security group to control who can access your file system.

And there’s a preferred subnet and a standby subnet. The standby subnet is going to be because we have multiaz and we want to have failover to a standby subnet in case something goes wrong. Then do we want to have Windows authentication? Is it going to be through AWS managed Microsoft Active Directory or a self managed Active Directory where we have to enter all the connection information and we don’t have any of those so we can’t really do anything with it. Do we want encryption of our file system? Do we want the FSX default key or a key that we create ourselves? And then some options for maintenance around backups and so on. And then some tags. If we wanted to tag our file system now this would cost us a lot of money if we tried to create this.

So I’m not going to try to create our FSX file system for Windows, but we saw the options and we understand better what it is. Then if I go back to FSX and create a file system, this time I’m going to choose for Luster. And this is for high performance computing for your Linux machines. So let’s click on Next and see the options. So again, FSX for Luster as the name, the storage capacity now is 1220, 400GB or incremented of 3600gb. So 3. 6. So I can do 707. 2 terabytes. I can do something like this. And then automatically the throughput capacity is going to be computed as the storage capacity in terabytes times 200 terabytes per second. So we get a 1. 4gb/second in that case. So if I keep on increasing the storage so for example, I can do this storage, I won’t go to get more throughput capacity.

And obviously I think we can go really, really high on the capacity. So let’s have a look if we can increase the capacity to something really high such as 108 terabytes yes, we can. And what about even something better? How about petabytes? Yes, we can. So we can really have a really, really big file system for FSX for Luster. And the throughput capacity of that system is 210 gigabyte per second, which is huge. Then the security, same as before, a default VPC, a security group and a subnet.

So this one is not multi AZ and then encryption it is automatically encrypted at rest. So we don’t even have an option to select no. Do we want to integrate with Amazon Sri as a source for our data repository? And do we want to have a specific bucket? And this is awesome because now FSX will integrate with our SF bucket. So I’m just going to say no. And maintenance preferences for the weekly maintenance windows, do we want to have no preference?

Or do we want to say define the maintenance window we need? So that’s it. Again, I cannot create this file system. It will cost me an insane amount of money. But you get the idea of how it works and hopefully that gives you a better idea of what is the difference between FSX for Windows and FSX for Luster. And when you go into the exam, you will have an easy way of selecting the right one. All right, that’s it. I will see you in the next lecture.

8. FSx for SysOps

Okay, so just a few notes on FSX for sysaps. So there is FSX for Windows in single AZ and multiaz. So in single AZ, well, you have a single AZ file system and it will automatically still have data replication, but just from within that AZ. Okay? And you have two generations for that option. You have single AZ one, which is just SSD and single AZ two, which is SSD and HDD into in terms of how the storage is done. The other option, if you want to have multiaz is to use the FMC for Windows multiaz option, in which case where we have two az’s, and it will be synchronous replications between the two file system, between the primary one and the standby.

So this will automatically replicate data across AZ. It’s synchronous, and then in case you have a problem, there will be an automatic failover. So in case one file system fails, failover will happen automatically to the standby replica. So it’s always recommended in FSX for Windows to use multiaz instead of using two single AZ and doing your own replication between them. Okay? Because this option exists in the exam, you should always choose multiaz over single AZ in case you want to have failover. That’s it for this lecture. I hope you liked it and I will see you in the next lecture.

• Category: Uncategorized