Google Associate Cloud Engineer – Choosing Database in Google Cloud Platform Part 1

1. Step 01 – Getting Started with Databases

Welcome back. Let’s get started with this section on databases. Before we understand what are the different database services available in this specific cloud and how do you choose them? We need to understand a few fundamentals about databases. Let’s get started with those fundamentals. In this specific step, databases provide organized and persistent storage for your data. To choose between different database types, you need to understand a few important concepts availability, durability, RTO, RPO, consistency transactions, ETCA. And that’s what we would be understanding through a simple journey. Let’s get started with a simple journey to understand all this in this specific section.

2. Step 02 – Understanding Database Fundamentals – Snapshot, Standby etc

Let’s get started with understanding some of the important concepts related to database. Imagine a database deployed in a data center in London. So I have a data center in London where I have a database and I have an application which is connecting to the database. Let’s consider a few challenges. Challenge one your database would go down if the data center crashes or the server storage fails. So challenge when is that? Your database will not be available for the application if the data center is not available. Challenge two is if the database crashes, you will lose data. In addition to the fact that database is not available, you might permanently lose data as well. So what we do, we start taking snapshots. Let’s automate taking copy of a database or a snapshot every hour to another data center in London.

So in the corporate center data too, we take a copy of the database. We take a database snapshot. Let’s consider the challenges again. Challenge one your database will go down if the data center crashes. It remains. We have not really tried to fix it at all. Challenge two is partially solved over here. Because we have snapshots, we will not lose the complete data if the database crashes. You can set up the database from the latest snapshot. But depending upon when the failure occurs, you can lose up to an hour of data. Why? Because we are taking a snapshot every hour. So let’s say I took a snapshot at 04:00. Let’s say this entire data center crashed at 450. What would happen is we lose all the data or all the updates that were made during those 15 minutes.

There is a new challenge as well because we are taking snapshots. When we are taking the snapshots, the database will be slow. Let’s see how to improve the situation further. Let’s add in transaction logs. Let’s add transaction logs to the database and create a process to copy it over to the second data center. Let’s look at these challenges again. Challenge one your database will go down if the data center crashes. That remains. Challenge two is solved. You will lose data if the database crashes. How is it solved? You can set up the database from the latest snapshot. And because you have transaction logs, once you get the database set up from the snapshot, you can apply the transaction logs. Challenge three database will be slow when you take snapshots. It remains.

Now let’s try and improve the situation further. What we do is we create a standby. Let’s add a Standby database in the second data center with replication. So we have two data centers, two databases. One is the Master database, the other one is the Standby database. And there is synchronous replication. As soon as you make a change in this database, the change is immediately updated into the standby database as well. The other thing which we did is we are taking the backup or the snapshot from the Standby database. So the application connects to the Master database and we are taking snapshots from the Standby database. Let’s look at the challenges again. Challenge one your database will go down if the data center crashes. That’s solved because you have a Standby database available.

If this goes down, you can connect your application to the Standby database. You can switch to the Standby database. Challenge two is solved as well. You will lose data if the database crashes. No, we don’t, because we have a Standby available. Challenge three database will be slow when we take snapshots. That also is solved because we are taking snapshots from the standby. The actual database is not impacted at all. Applications connecting to Master will get good performance always. The idea behind designing this journey was to give you a highlevel overview of the typical things that are involved when we are talking about a database. Now that we have a good perspective into databases and some of the important challenges, let’s see a little bit more about them in the next step.

3. Step 03 – Understanding Database Fundamentals – Availability and Durability

Come back. In this step, let’s talk about the availability and the durability of a database. What is availability? Will I be able to access my data now and whenever I need it? The percentage of the time and application provides the operations which are expected out of it. What is durability? Will my data be available after ten or 100 or 1000 years? That’s what durability is all about. Let’s look at a few examples. Samples of measuring availability and durability. Four nines 99. 9 911 nines 99. 9 nines. Typically, as far as availability is concerned, four nines availability is considered to be really, really good. However, when it comes to durability, it’s eleven nines, which is considered very good. Why is there such a difference? Let’s look at that right now. Let’s look at the different measures of availability.

99. 95% of availability means you can only have 22 minutes of downtime in a month. 99. 99% of availability means you can only have four and a half minutes of downtime in a month for the entire application. Typically, all online apps aim for this. Remember that this includes the time to make your releases as well. If you are releasing a new version of the application, if you are doing five releases in the month in total, you should not have more than four and a half minutes of downtime if you want to achieve 99. 99% availability. Now, if you extend it further, and if you talk about 99. 3. 9 availability or five nines availability, that would mean 26 seconds of downtime in a month. This is really, really tough to achieve. Now that we understood availability, let’s now switch our attention to durability.

What does the durability of eleven nine s mean? If you store 1 million files for 10 million years, you are expecting to lose one file. That’s what the durability of eleven nine s provides you with. Think. Why should your durability be high? Why should we always go for high durability when it comes to databases? The reason is because we hate losing data. It’s okay if the application is down for a little bit of time. However, if we lose data, the data is lost forever. Once we lose data, it is gone. Imagine losing a financial transaction. Imagine losing a stock trade which was performed. We don’t want to lose data at all. In this quick step, we got a high level understanding of availability and durability of the databases. In the next step, let’s look at how you can actually increase the availability and durability of your databases.

4. Step 03a – Understanding Database Fundamentals – Increasing Availability and Dur

Welcome back. In this quick step, let’s talk about increasing availability and durability of databases. Earlier, we looked at a simple use case where we tried to increase the availability and the durability of the database. Let’s try and summarize that in this specific step. So how do you increase availability? Having multiple standbys available so we can create a standby, and if this database goes down, we can serve the users from the standby database so you can create multiple such standbys. This is also called distributing the database. So you distribute the database across multiple zones, if you can, and multiple regions if you can.

By distributing the database across multiple zones and multiple regions, you would get higher availability. And what do you do to increase durability? The way you can increase durability is having multiple copies of the data available. So you can have standby snapshots, transaction logs, replicas available in multiple zones and in multiple regions. However, the important thing to remember is that replicating data does not come free. It has its own challenges. As soon as we start having multiple copies of data, how do you ensure that all of it is consistent? We’ll talk about these challenges a little later. In this and the last few steps, we looked at availability and durability of databases and how you can increase them. I’ll see you in the next step.

5. Step 04 – Understanding Database Fundamentals – RTO and RPO

Come back in this step. Let’s look at some terminology related to databases RTO and RPO. Imagine a financial transaction being lost. Imagine a stock exchange going down for an hour. Typically businesses are fine with some downtime, but they hate losing data. Availability and durability are technical measures. How do we measure how quickly we can recover from a failure? That’s where RPO and RTO recovery point objective and recovery time objective come into picture. RPO is maximum acceptable period of data loss how much time period can you lose data up to? RTO is recovery time objective it is maximum acceptable downtime.

How much time can you recover within achieving the minimum possible RPO and RPO is always expensive and that’s the reason why we would need to trade off based on the criticality of the data. Let’s look at an example. You are running an application in a VM instance storing its data on persistent data storage. You are taking snapshots every 48 hours. If the VM instance crashes, you can manually bring it back up in 45 minutes from the snapshot. What is your RTO and RPO? I would recommend you to actually look at the previous slide again and try and find out what is the RTO and RPO. In this specific scenario, the RTO is 45 minutes because if you are down, you can come back up from the snapshot in about 45 minutes. However, your RPO is 48 hours.

If I’ve taken a snapshot NAV and let’s say the VM instance crashed after 47 hours 59 minutes, all the data for that time period is lost. So the RPO is around 48 hours. Let’s look at the different trade offs when it comes to RTO and RPO. Let’s look at a few examples. So if I want very small data loss and very small downtime, RPU of 1 minute and RT of five minutes, how can I achieve that? We can achieve that by having hot standby or basically other instances of databases which are able to take load immediately. You need to be able to automatically synchronize data, you need to have a standby ready to pick up load and you need to also have automatic failover from master to standby. The other option of how you can achieve this is also by distributing the data and using multiple masters.

The second scenario is very small data loss. So RPO of 1 minute, but I can tolerate some downtime RTO of 15 minutes. In those kind of situations you go for a warm standby, you would automatically synchronize data and you’ll have a standby with minimum infrastructure and when a failure happens, you scale it up. Data is critical so RP of 1 minute, but I can tolerate downtime of a few hours. You can create regular data, snapshots and transaction logs. And when a failure happens, you can create the database from snapshots and transaction logs. Because your RPO is a few hours, you can easily bring the database up within that time period. However, because you have snapshots and transaction logs, you will not lose a lot of data. So you can achieve an RPO of 1 minute.

The other example is data can be lost without a problem. For example, cached data in those kind of situations. What you can do is ignore the old server and failover to a completely new server. You don’t really need to worry about data so you can create a completely new server. In this step we’ll look at RTO and RPO. Availability and durability are technical measures and RTO and RPO are the measures which are typically used by business recovery time objective is how quickly you can recover from a failure and as we saw in this step, R twoo and RPO is all about trade offs. There is no single solution based on the RTO and RPO you would want to achieve. You need to design your solutions I’m sure you’re having a wonderful time and I’ll see you in the next step.

6. Step 05 – Understanding Database Fundamentals – Read Replicas

Welcome back. Last few steps we looked at availability, durability, RTO and RPO for databases. Let’s make this a little bit more complex and let’s consider a new scenario. You want to add reporting analytics applications on top of your database. New reporting analytics applications are being launched using the same database. However, these applications will only read data. Within a few days, you see that the database performance is impacted. The database is not. The database is not as fast as it was earlier. How can we fix the problem? You can vertically scale the database. That’s basically increase the amount of CPU and memory. The other option is to actually create a database cluster. You can distribute the database typically if you are using relational databases. Database clusters are really expensive to set up.

The other option is also to create a read replica and you can run your readonly applications against read replicas. So we can add a read replica. You can connect your reporting and analytics applications to read replicas. This would reduce the load on the master databases in case of failure of the main database. You can upgrade the reads replica to a master database. This is supported by some of the databases. If you want higher availability and durability, you can also create read replicas in multiple regions. This would ensure that the main database does not have a performance impact because of snapshots in the step. We talked about read replicas. We talked about the scenarios where read replicas will be useful. I’m sure you’re having an interesting time and I’ll see you the next step.

7. Step 06 – Understanding Database Fundamentals – Data Consistency

Come back. Earlier we talked about the fact that when we have multiple copies of the data, we will have problems. And one of the problems is consistency. How do you ensure that data in multiple database instances, that is, standbys and replicas is updated simultaneously? Let’s look at some of the different levels of consistency. Strong consistency is synchronous replication to all replicas. The moment a change is performed to a database, if it has five replicas, the change is synchronously updated to all of the five replicas. What is the problem with strong consistency? It is that the transactions will slow down. You can only commit a transaction when the change is updated to all the databases. And if you have a lot of standbys or replicas, then this might take a long time.

Eventual consistency uses asynchronous replication. There will be a little lag before the change is available in all the replicas. So a change is updated in here. Slowly it will be distributed to all the other replicas. This is what is called eventual consistency. In the intermediate period, different replicas might return different values. Typically, you would use eventual consistency when scalability is more important than data integrity. Good examples are social media posts, Facebook status messages, or Twitter tweets or LinkedIn posts. Even if there is a little delay in replicating the data across all the replicas, it’s fine. Nobody is really worried if the replication is delayed by a few seconds. However, let’s consider a banking transaction.

If you are transferring a million dollars, you don’t want to depend on eventual consistency. In those kind of situations, you would want strong consistency. The next option is read after write consistency. When you’re using read after write consistency, inserts are immediately available. If you are inserting data, it is immediately available in all the replicas. However, updates would be using eventual consistency. If I’m making a change in data, then it will be slowly replicated to all the replicas. So different databases offer you different levels of consistency. And based on the needs of your application, you would need to choose the data database with the right consistency. In this step, we talked about consistency. I’ll see you in the next step.

• Category: Uncategorized