SAP-C02 Amazon AWS Certified Solutions Architect Professional – New Domain 5 – Continuous Improvement for Existing Solutions Part 7

45. Overview of Storage Gateway – File Gateway

Hey everyone and welcome back. In today’s video we will be discussing about file gateway. So, file gateway is one of the gateway types of storage gateway. So let me quickly show you this before we proceed with what exactly this is and a nice demo. So I am in my management console and let’s go to services and we’ll go to storage gateway. Now, within year you will see that I already have a file gateway which is created. So this one is for the demo.

However, when you go ahead and create a gateway, there are three basic types of gateway that can be created. First is file gateway, second is volume gateway and third is tape gateway. So the volume gateway is where the cached volumes and the stored volumes reside in. So this is something that we have already discussed in a theoretical perspective. However, with theory, understanding storage gateway is a little bit challenging.

It is necessary to have a practical session where we can understand on what exactly storage gateway does. So in today’s video, we’ll do exactly that. So the AWS storage Gateway file interface, also referred as the file gateway, provides an interface via the Network File system or SMB for synchronization of data from on premise to s three bucket. So let’s understand the storage gateway overview diagram. So on the left hand side you have files, you have volumes, you have tips. On the right hand side you have s three bucket, you have glacier, you have EBS snapshots.

So what organizations might want, let’s say that organization have lot of files. What they want is they want a proper and automatic synchronization to s three bucket. And this hybrid scenario is generally well achieved with the help of storage gateway. So storage gateway is like a medium which sits between both of them in such a way that it can automatically synchronize, let’s say, files with the s three bucket. So since currently we are discussing about file gateway here, we’ll be discussing about files.

So again, storage gateway will remain the same year. Now, in terms of file gateway, it will go ahead and sync whatever files are present to the SRE bucket. And here the interface is NFS or SME. So let’s quickly look into what exactly this is. So, as we were discussing about the file gateway, let’s go back. Now, I already have one file gateway up and running here. Now, this a storage gateway appliance. So this appliance can be an easy to instance. And in fact, if you will see over here, this appliance of storage gateway is running on an EC two instance.

And within the EC two instance I have a file gateway appliance which is currently up and running. All right, so we have the appliance which is up and running over here. Now, this appliance that I have, it is integrated with the s three bucket. So this appliance is connected to the s three bucket. And this appliance also has exposed a NFS endpoint if you go to five shares over here so this is the exposed endpoint so here you see this is the NFS based exposed endpoint all right? Now, what organization have to do is they have to use this and connect to this specific instance. So here we have the EC to instance.

So this EC two instance acts as a storage gateway appliance. What organization have to do is they have to connect to this appliance and they have to store the files in this specific appliance there. And how can they connect? They can connect with the specific command so here you have various commands one command is for Linux, one command is for Windows, one command is for macOS so let me quickly show you for Linux in this demo. So I have an EC two instance over here so this is a sample EC two instance. So if you want to quickly see so this is a sample EC two instance that I have created just for the demo purposes. Now what we’ll do is we’ll copy this specific command and let me paste it over here now, the only thing that we have to change here is the mount point all right?

So basically I have a directory called as my tempo here, let me just put an M. All right, so this is an empty folder or an empty directory over here. So what we are doing here is that with the help of storage gateway, we are creating an interface between the S three and the files which are present within our server. So this interface is being created with that specific command. So let me go ahead and run this. All right? So once I run this and let me type LS my temp. And now you see there are two files which are visible over here. In fact, these two files are basically the files which are within the S three bucket. Let’s find it out. So basically this specific file gateway is associated with our s three bucket called kplabs testing. Now, if I open up kplabs testing over here you will see that there are two files over here and these are the two files which are being visible over here.

So here what is happening is storage gateway is acting as a middleman which basically does entire thing like synchronization, caching and various others and this is exposed via the NFS mount point. So our storage gateway is basically exposed via NFS gateway. So now if you are wondering, let’s go ahead and create one more file. I call it as kplabsdemo TXT. All right, before we do that, let’s go to the my temp directory and let’s run this command. And if you do a LS, you will see that there is a kplab TXT file which is created and once I refresh you will see that file is automatically present within the s three bucket. So I hope at a high level overview. Now you understood what exactly storage gateway is all about.

So storage gateway is more of a hybrid setup where on the left hand side, these are the things which are typically present on premise like in the data center and these are the things which are typically in the AWS side. And even here if you type storage gateway, you see it is like a hybrid storage integration here. So now whatever files application might be putting, let’s say in a folder, instead of that they can now put it in an NFS mount point. So for example, if I do a DF, you will see that this specific one. So if you work with NFS, you will understand this pretty quickly. But at a high level overview, this is like a s three bucket which is accessible in this directory. So whatever you store in this directory will be synchronized with the SV bucket. So let’s say that, let me give one quick example.

Let’s say I create a directory called as my temp two, all right? Now let’s say whatever files application is storing in my temp two. So these files are stored in the EBS volume and if the EBS volume is lost, then your files are lost. So this is one example. Now, in case of the on premise setup, each server has their own hard disk drives and if the hard disk drive fails, then whatever data, if you have not really done a replication and all, then your data will be lost. So instead of doing all of those complex replication and other things, what you can do is you can simply mount your storage gateway appliance to a directory called as my tent and tell your application to store all of your files here.

So not only your files will be automatically backed up, it will also be within s three. So the durability will always be there. So that’s the high level overview about the file gateway. Now, in a similar approach, in a similar approach of file gateway where we were working with files, you also have volumes and you also have tapes. So the approach remains similar. So this storage gateway acts as a middleman between the left hand aspects and the right hand aspects. So with this, I hope you understood the overview about what file gateway is all about and also have a better understanding on what storage gateway by itself is all about.

46. Implementing File Gateway

Hey everyone and welcome back. Now in the earlier video we had a great overview about file gateway. So what we will be doing in today’s video is we’ll be implementing it and we’ll look into what are the steps which are involved. So currently I am in my storage gateway service. So I’ll click on get started. Now whenever I click on Get Started again, there will be three type of gateways which will be listed here. We’ll start with file gateway. So I’ll select file gateway. Let’s click on next. And here you have to select a host platform. Now, depending upon your environment, this host platform might differ. Now, good thing here is that we can make use of the storage gateway appliance in EC Two. So it becomes much more simpler for us to perform the practical.

So I’ll select the EC two over here. So this is an appliance. So this is like a middleman that we are launching here. So I’ll click on EC two here and I’ll click on Launch Instance. Now, do remember that storage gateway appliance does require good amount of CPU and Ram. So for our demo purposes I’ll be using a t two large instance. Let’s click on next. We’ll leave this to be default one. Now here under the root volume. So you have an 80GB of root volume over here. So this is required for the appliances application to be running. So this is something that you cannot really modify. So now let’s add one more volume because what will be happening is companies or organization would be storing their files within the storage gateways appliance. So we have to have a good amount of storage there. So for our testing purposes I’ll be creating a tenGB of storage. Let’s click on next. Let’s click on Configure security group. And for the time being just for testing purposes.

So since I don’t want to add some more complexity here, I’ll just select all traffic on all right, let’s go ahead and launch the instance. All right, so the EC Two instance is getting launched. I’ll name this as File Gateway appliance so it becomes much more easier for us to understand. So, coming back to the diagram, what we have done is this storage gateway. So we have created this EC Two instance. This EC Two instance is launched from a specific AMI which has the appropriate software. All right, so that software does that bridging. So whenever you upload the data here, it goesychronize it to the Sree and it also exposes various things. So let’s quickly wait for a few moments here till the EC Two instance is up and running. All right, so it has been few minutes. So now let’s go back to the storage gateway here and click on Next. Now you have to put the IP address of the VM.

So here if you see there is an IP address which is associated with this EC Two instance. So I’ll put the IP address here. All right. And let’s click on Connect to Gateway. Great. And if the connection is working as expected, you will see the next screen of Activate Gateway where you can select the time zones. So let me select my time zone here and you can give it a name. I call it as Demo file Gateway. And I’ll click on activate gateway. So now in the next screen what it does is it basically goes ahead and scans for the disks which are present within that easy to instance. Now, if you remember we had added one more additional disk where the files would be stored. Great. So after it has scanned it, you will be able to see the additional disk which is of tenGB over here. Now, do remember that this might take a little amount of time. It might take four to five minutes for you to be able to see the disk.

Now, once you see it, make sure that it is allocated to cache. Now you can go ahead and continue. Great. So this is the gateway which is created. Now, the next step that you need to do is you have to create a file share. So let’s go ahead and create a file share over here. So now file share you have to put the bucket name of S three. So let me put it as. KP labs. Hyphen testing. Now, in my case I have an S three bucket which I want to link. I’ll say it as kplabs hype and testing. Now I associate it with my gateway which just got created and Access Object using. So again, there are two ways. One is NFS and one is SMB. And in fact, if you look into the PowerPoint slide, we have already discussed that the interface is via NFS or SMB. So I’ll select NFS. Let’s click on next. You can leave these things as default and let’s click on Create File Share. All right, so now the file share the status is now creating. So if you go below, depending upon the operating system over here.

So we have already discussed that these files now various applications which might be running on Mac, which might be running on Windows, it might be running on Linux, that application might want to store it. So depending upon the operating system where your application is running, you can go ahead and use one of these. So if I’ll just quickly do a refresh, you see the status is now available. So now what you have to do is you just have to copy this up and run it in a corresponding operating system. Now, do remember here that in here, let’s say that currently if I can just quickly show you this File Gateway appliance here, this has a specific IP address. So whatever IP address which is being used, you have to make sure that your destination is able to connect there. Now, in my case, what I have done is I have created a sample EC two instance within AWS itself.

So let’s connect to this EC two instance here. Now both the EC two instance are within the same VPC and I connect to that EC two instance via SSH. All right? Now the next thing that you have to do is you have to create a mount point. Let’s say I’ll call it as file gateway. All right, so this is basically a directory. Currently there is nothing which is present within that directory. It’s an empty directory. So let’s copy this command up and let’s change the mount point over here. I’ll call it as file gateway. All right. And now when you go to file gateway, you will see that whatever contents which were present within the s three bucket are now being accessible from the file gateway. So if the application goes ahead and creates certain files over here, so let’s call it as demo dot TXT over here. Now this file will be synchronized back to the s three bucket via the storage gateway.

So now within the s three bucket, do remember that it is not a synchronous job. It takes a little amount of time for it to be synchronized. It might take around two to three minutes, or if it is a larger file, it might take more time. So currently if I do a refresh, you see that file has not been synchronized yet. So let’s quickly wait for a minute here. So it has been a minute and now you see the file is synchronized to the s three bucket. So I hope at a high level overview you understood the process on how you can go ahead and create a file gateway and what exactly file gateway and storage gateway services are all about. So that’s about it. I hope this video has been informative for you and I look forward to seeing you in the next video.

• Category: Uncategorized