Amazon AWS Certified SysOps Administrator Associate – S3 Storage and Data Management – For SysOps Part 2

4. [SAA/DVA] S3 Access Logs

Okay, so now let’s talk about Amazon s three access logs. So, say that for audit purposes, you want to log all the access into your S three buckets. So that means that any request that is done to Amazon is free from any accounts authorized or denied. You want it to be logged into another s three bucket. So you can analyze it later. So you can analyze it, for example, using Data Analysis Tool or something we’ll see in this section called Amazon An Athena. So here is the idea with the diagram.

We make request into a bucket, and that bucket has been enabled for logging into another bucket, a logging bucket. And so all the requests, once we’ve enabled the SDX logs, will log all the request into the logging buckets. Very, very easy, very simple. And the log format is defined here. So if you’re interested about how you can read this log, just click on this link. Okay, now, something you need to know about these logging buckets, that is pretty natural, but you need to know about it once.

So do never, never, ever set your logging bucket to be the bucket you are monitoring. Otherwise, if you set the logging buckets and the monitoring bucket to be exactly the same, then it will create a logging loop and your bucket will grow in size exponentially. So it’s very simple to represent it. So say we have a bucket, it happens to be our application bucket, and also the bucket that is going to receive all the logs. And so therefore, whenever a user puts an object, the bucket is going to log inside of itself, create a logging loop, and they will create a new object that will be logged and a new object that will be logged. And so it will create an infinite logging loop. And so that’s why your bucket will grow in size exponentially. So my advice to you is do not try this at home. This will end up in a huge AWS bill for doing this little mistake. So always separate your application bucket and your logging buckets. Now, let’s go into hands on to see how this works.

5. [SAA/DVA] S3 Access Logs – Hands On

So let’s demonstrate. S three access logging. So I’ll call it Demo s Three Access Logs stefan 2020 and then I will leave all these settings on and click on Create Buckets. Okay, so this is creating my buckets and this bucket is going to be used for access logging from my other buckets. So let’s take my demo stiff on on S Three Bucket 2020. And I’m going to turn on server access logging. So for doing so, I’m going to go into Properties and I will scroll down and I will find the Server Access Logging in here. Okay, next I will click on Edit and then enable server access logging. Next I need to specify a target bucket so I can just browse Amazon S Three and look at this bucket that just created.

Choose the path and we can do, for example, logs slash. If you wanted to have all the server access logs to go under the S Three Lugs folder, it’s up to you. It’s optional, but add a trading slash at the end, save the changes and we’re good to go. So now it is enabled. And so the idea is that if I go and for example, list my versions, if I go and take this coffee JPEG file and open it and stuff like this, this is going to generate some traffic onto my bucket. Okay? And this is going to be logged onto my other buckets called the Demo S Three Access Logs. Define. Now this takes, this can take an hour or 2 hours to appear.

So I’m going to wait a little bit for it to be written. But one question you may have is how does this bucket by turning on the Server Access Logging, how is this bucket getting the right to write to my logging buckets? And so it says it here by enabling server access logging, the S Three console will automatically update your bucket access control list, or ACL, to include access to the S Three log delivery group. So let’s check this out. Let’s go to permissions of my demo s three access logs bucket. And under permissions, if I scroll down and go to Access Control List ACL, yes indeed, the S Three Log Delivery Group has the right to write my objects onto my S Three buckets. So this is something that has been added automatically by Amazon S Three when I did enable server access logging.

So just an instagram t, but it’s always good to see the full security picture when I do something. Okay, so now the only thing I have to do is wait. So I’ll pause the video and hopefully within an hour or two, I should be able to see some objects being populated in here. So I will see you very soon. Okay, so I’m in my access log bucket. I’ve waited an hour, so hopefully if I refresh, yes, I start seeing S Three Logs folder that has been created. So perfect and within that folder, it contains a bunch of access logs of what has been done on my S three buckets. So I can take any of this file.

I can take this one, for example, and I can download it. It’s a text file. So I’m going to open it with my text editor to see what’s inside. And so I just opened this file, and this is a text file. This is one line. So this contains one bit of information in this specific file. And this tells me about the request ID the bucket. It was made on the time and date of this bucket request. The IP is coming from the fact that it was a get. So it was a get and the result was 200. So it was a successful get on a bucket at the very top. So it was probably like a request down at the top of the bucket.

So these type of access logs can be analyzed at scale using something like Athena that we’ll see in this course. And so on our own, they’re not very helpful. But if there are any problems, if there’s any authorization issues or attacks or whatever athena and analyzing these files and getting down to the bottom of it will allow you to get more insights into what is happening. So that’s it for access slugs. I hope you liked it, and I will see you in the next lecture.

6. [SAA/DVA] S3 Replication (Cross Region and Same Region)

Okay, now let’s talk about Amazon s three replication that is CRR and SRR for crossregion replication and same region replication. At its core of the idea, we have an S three bucket, for example, in one region and we want to replicate it asynchronously into another region, into another bucket. So to do so, we first must enable versioning in the source and destination buckets and then we can set up two different things cross region replication if the two buckets are in different regions or same region replication SRR if the two buckets are in the same regions, note that the buckets can be in different accounts.

So it is very possible for you to save a copy of your data into another account using Sree replication. The copying happens asynchronously, but it’s very quick. And for the copying to happen, you need to create an im role. We’ll see this in the hands on and that im role will have the permissions from the first Esther bucket to copy to the second estra buckets. So the use cases for cross region replication is for compliance or lower latency access of your data into other regions or to do cross accounts replication. And SRR is the same region replication.

The use cases could be log aggregation. So you have different logging buckets and you want to centralize them into one bucket or live replication, for example, between a production and your test accounts. And so, here is the fine print about S three replication. After you activate s three replication, only the new objects are replicated. So it’s not retroactive, it will not copy your existing states of your SV bucket.

And for delete operations, there is an optional setting to tell you whether or not you want to replicate your delete markers from the source to the target. Or if you specify the deletion of a specific version ID, then it is not going to be replicated to avoid malicious deletes. And finally, there is no chaining of replication. That means that if bucket one has replication into bucket two, which has replication into bucket three, then any objects written in bucket one will be in bucket two, but will not be replicated to bucket three. So you cannot change your replication. So that’s a fine print for entry replication. Now, let’s go in the handson to see how that works.

• Category: Uncategorized