1. QLDB overview
In this section let’s talk about QLDB. QLDB stands for quantum ledger database. It’s a fully managed serverless ledger database. And what this means is QLDB provides us with a builtin immutable journal to record all the change history of our data. Now, a database is in QLDB is called as a ledger and this is transparent and cryptographically verifiable ledger. It tracks each application data change and maintains a complete and verifiable history of changes to your data over time. And QLDB does provide a strong acid compliance. And when we work with QLDB, we use a query language called Particle. Particle is a sequel like Open Standard that we use to query the QLDB data. And the data in QLDB is stored in Amazon Ion format which is a superset of JSON.
It provides the same functionality like JSON and then some more. It’s a self describing hierarchical data serialization format and it offers interchangeable, binary and text representations. So the Ion specification adds additional data types, type, annotations and comments to the standard JSON format or the standard JSON specification. And it does support nested JSON elements. And finally, the typical use cases of Qltb are system of record applications like banking transactions, HR services records, insurance claim histories, vehicle ownership records and so on. So that’s about the overview of Qltb. Let’s continue to the next lecture. Well, we’ll look at the QLDB architecture.
2. QLDB architecture
The QLDB is a serverless service, so you don’t have to provision any servers, and it can scale automatically to support the needs of your application. Now, it’s intended to support high performance OLTP workloads, and the architecture of QLDB looks something like this. You have a ledger that contains a journal and a set of tables that store the current state as well as the historical state of your data. So the database in QLDB is called as a ledger, which is a combination of a journal and a set of tables. A journal is a data store that is append only and immutable. It does not allow any updates, overwrites or deletes, and it stores a sequenced cryptographically verifiable entry of each change to your table data. These changes are changed together as blocks. But remember that this is not a blockchain implementation.
QLDB provides a centralized storage architecture and not a distributed one like we have with Blockchain. So blockchain is used with decentralized use cases. And even if you delete data from your table or from your ledger, you can still access its change history from this immutable journal. And then we have tables. Now, these tables are collection of documents and their revisions. So the tables store the current as well as the historical states of your data in an indexed fashion. And these can include document deletion records as well. All these documents are in the Amazon Ion format, which we already know is a superset of JSON. Now, let’s quickly compare the relational databases with the ledger database. So in relational databases we have a concept of database. In ledger database, we call it as ledger.
Tables and indexes are same in both relational as well as ledger databases. Table rows in relational databases correspond to the Ion documents in a ledger database, columns in relational database tables correspond to document attributes in your ion documents within your ledger database. For relational databases we use SQL query language and for ledger databases we use particle. Now, audit logs in relational databases can be compared with the journal in the ledger database, but journal does provide us additional functionality and features than the typical audit logs in relational databases because journal provides us with verifiable and immutable record of change history of your data. All right, so that’s about the comparison of relational and leisure databases. Let’s continue to the next lecture.
3. QLDB views
Now, let’s look at the Q LDB views. So Q LDB offers three kinds of views of your data user view, committed view, and history view. So what exactly is the User view? User View is the default view that’s available with Q L DB. It shows you the current state of your data or the latest version of your data. So, for example, select Star from Vehicle Registration is going to give you the current state of the data about vehicle registration, for example, and this is the default view. Then we have a Committed View, which is essentially same as the user view. But in addition, it also contains some system generated metadata of your data records.So if you want to query the Committed View, you should prefix the table name with underscore QL underscore Committed underscore.
So as you can see in this example, if you run a query, select star from underscore QL underscore Committed underscore Vehicle Registration, then you are going to receive the latest version of your data along with system generated metadata. And finally, you have a History View, which gives you all the data about your tables. So all the historical document revisions can be viewed by using the History view. And to access the History view, you use the history function as shown in this example here. So all the change history history along with the metadata will be returned to you when you make a History View request. All right, so that’s about the QLDB views. Let’s continue to the next lecture.
4. Working with QLDB
So how do you use QLDB? So you first create a ledger and define your tables. And we already know that QLDB supports asset semantics or asset properties. And once you have your ledger and tables ready, you can use the Particle Query Language to write to your QLDB database and to query the data as well. Your application can use the Particle Query Language to write to QLDB and read from QLDB. And Particle, as I mentioned earlier, is a sequel like Open Standard Query Language. And it provides a SQL compatible access to relational, semi structured and nested data. And it kind of extends SQL to support Ion documents. And in addition to QLDB, Particle is also used with redshift with S three select as well as with Glacier Select.
So that’s something just good to know. All right, now let’s look at the Amazon Ion format. The Amazon Ion format, or the Ion, is a super set of JSON. You can see an example here. It’s a self describing hierarchical data serialization format. Or in simple words, it is a nested JSON format. All right? And as mentioned earlier, it offers interchangeable, binary and text representations. It adds on to the JSON specifications, so it adds additional data types, type, annotations and comments to the standard JSON specification. And because it’s built on JSON, it provides for a flexible data model. And as you can see in the example here, you can use different types of comments in your Ion documents. All right, so that’s about the Ion format. Let’s continue to the next lecture.
5. Data verification in QLDB
Now, let’s talk about data verification capabilities in Qltb. Now, we already know that the QLDB journal provides us with immutable and verifiable history of all the data changes to our data. So it maintains an Immutable and Verifiable transaction log. So how do you verify this? So, for this purpose, QLDB uses what’s called as a digest. So, QLDB uses a digest for data verification, and the digest is nothing but a cryptographic representation of the current state of your data. So it’s a cryptographic representation of your journal or a unique signature of your data’s entire change history as of a given point in time. So a digest looks something like this, and it’s generated using the Shar 256 hash function with a Merkel tree based model.
And you can use it to verify the integrity of your data by calculating the digest yourself and then comparing it with the digest provided by QLDB. So, this screenshot shows the calculated digest as well as the digest provided by QLDB. And if these two digests match, then you are sure that your data is safe and has not been tampered with. And you can do this verification using the AWS console as well as by using the QLDB API. And if there are any errors or if you use improper verification requests, it is going to result in illegal argument exception. All right, now let’s go into a demo and create our first QLDB ledger.
6. Creating a QLDB ledger – Hands on
In this demo. Let’s create a Qltb ledger to see it in action. And we are going to populate this journal with the sample data provided by AWS. Open the Qltb console and from the left menu, click on Getting Started. And from here under Create your first ledger, click this Create Ledger button button. And what this is going to do is it’s going to create a ledger with name vehicle registration. Okay, so we are creating a ledger that contains vehicle data, the history of vehicle ownerships. Simply click the Create Ledger button and this is going to create a ledger for us. Now it’s going to take a few minutes. I’m going to pause the video here and come back once this ledger is ready.
All right, so the ledger is active. Now let’s go back to the Getting Started page. And from here now, select the ledger that we just created and click on the option Load Sample Data. This is going to load the data. So that’s done. And now we can start querying this data. From the third section, open the query editor, and from here we can choose the ledger that we want to query. So this is the only ledger we have. So here are the ledger details. We have four tables person, vehicle Registration, driver License and vehicles. If you click on this, it’s going to auto populate a select star query there. So you can simply run this query and it will show up the result of the query.
All right, similarly, you can do the same with the second table and so on. So you can query your data right from within the Qltb dashboard. So let’s copy this Vin number here. So this is the vehicle identification number here. So let’s copy that and go to the Vehicles table and we can say select star from vehicle as V where Vin equal to the Vin that we copy. Okay, and let’s run this. And we can see that this vehicle is a sedan 2011 Audi and it’s silver in color. So this provides us with the vehicle details of the queried vehicle. Now let’s run a few more queries. Okay, to make our job easier, what I’m going to do is go to the home page of the Qltb console and from here open this link Getting Started with QLDB.
And from here, click on the third step that is query the tables. And here we see a couple of queries. So let’s quickly run these queries. Okay, so what this query is doing is it’s running a join on the vehicle table and the vehicle registration table. And it’s returning us the vehicle registration information like the vehicle identification, license plate number, state, city owners and so on. So we can see that there is a primary owner here and there are no secondary owners. So this vehicle is a firsthand vehicle, right? Now let’s look at the second one. And this query is running a joint on the person and the driver’s license table and returning as the data of different drivers that were added to the ledger.
Right, now let’s go ahead and modify this data so we can review the revision history. All right, so let’s go back here and click on this fourth step and it’s going to give us some sample queries that we can use to modify this information. So let’s copy this one and run it. So what we’re doing here is we are capturing the ID of the person named Role. Let’s copy this ID so we can register this person as an owner of a vehicle. Okay, make a note of this ID and then let’s go back here and copy the next query, paste it in, and we’re going to change this person ID here to the appropriate ID that we copied and run this query.So what this is going to do is it’s going to update the primary owner of this vehicle.
All right, now let’s go back here and copy this query and run it to find out the owners of the particular vehicle. And now we can see that this particular person, which is the ID that we copied, this belongs to Roll, and this person has been set as the primary owner of this particular vehicle with the vehicle identification number that is showing up here. All right, let’s continue. Now what we are going to do is now we’re going to transfer the ownership of this vehicle to Brent. So let’s find the ID of Brent, copy that ID and make a note of it. And then we are going to run this update command and we are going to set the owner of this vehicle to this new person. Okay, so this is the ID of Brand.
So this is the person that’s going to take over this vehicle and run this query. And now the query is successful. And now let’s run this query to check whether the update has been successful or not. And now we can see that the primary owner has been changed to Brent. Now, this is the ID that corresponds to Brent. Now we are going to add Alexis as a secondary owner of the car. Let’s find her person ID, make a note of it, and now we’ll run this insert statement to add Alexis as the secondary owner. So change the person ID to the ID of Alexis and run the query. And the query is successful. Let’s run this select query to verify the same. And we can see that secondary owner has the person ID set to the ID corresponding to Alexis. All right, so this is how we modify documents.
And now we are going to view the revision history of our changes. Let’s look at the data of this particular vehicle from the vehicle Registration table. So first we have to find the ID of the vehicle registration documents. We run this query and this is the ID so make a note of it. And then we can use this ID in this select query to look up the historical records of this particular vehicle. So replace this with the ID of the document and we can remove this. And here we have to provide the name of the table, which is vehicle Registration. All right. And run the query so we get all these records here. Let’s go back here and copy this query. And this is going to give us only the specific data instead of the entire set of data. So earlier we got the entire data, so we are only interested in Vin, CT and the ownership data here.
So this query is giving us exactly that. Okay, so we got to change this ID with our document ID. We do that. And now we got a concise set of results here. And you can see that here is the original document. Then this is the first owner. And then we transferred the vehicle to the second owner. And then we added a secondary owner here. Okay. All the history of the particular vehicle, it’s showing up here. Let’s go back in here. And now let’s copy over this statement. And this query is going to give us the metadata of all different revisions. Okay, so we have the ID of the document and its versions, the time when the transfers happened and the transaction ID. So these are the details of the changes to the vehicle registration document. All right, let’s continue to the next lecture.
7. Data verification in QLDB – Hands on
The document. Verification is an important aspect of QLDB as it ensures the integrity of your data. So let’s find out how to verify a document. So first we have to request a digest, and then we can verify the document revision. So requesting the digest is fairly easy. Go to the ledgers page and select your ledger and click on this button that says Get Digest, and it’s going to show us this digest here. Let’s save that. And if you open that file, it’s a simple JSON file, or rather an Ion file that contains different information like the digest, the digest, IP address, and other information. Okay, so you can use this digest to verify the integrity of your data. Let’s keep this digest here.
And now let’s see how to verify the revisions to your data. Okay, so let’s go back here and pick up this query and go to the Query editor and run it. Now, this is giving us an ID and a block address. Make a note of these two pieces of information. And once you have noted this data, click on the verification option here. And here we have to provide the block address and the Document ID that we just copied. Okay, so paste in the block address that we copied as well as the Document ID. Make sure that you don’t enter any codes here. So add the Document ID without any codes. And now here we have to choose the digest that we downloaded. So let’s choose a digest.
Okay, so we have the digest and the digest tip address here. And now we can verify. Click on the Verify button to verify these two things. And here we can see that the document has been successfully verified, and it shows you the calculated digest as well as the actual digest. And these two are exactly same. So you can just verify that. So both of the values are exactly same, right. So this is how you would verify any revision using the Qltb console. So that’s about this demo. Before we close, let’s delete our ledger to clean it up. All right, that’s about it. Let’s continue to the next lecture.
8. QLDB backup and restore (an alternative)
Now let’s talk about backup and restore in QLDB. So QLDB does not support backup and restore feature yet, so there is no backup and restore and no BITR with QLDB. But what you can do is you can export your QLDB journal to S three, so you can create an export job and export your QLDB journal data to S Three. And you typically do this for analytics, auditing, data retention, verification, or for exporting your QLDB data to other systems. So to export your QLDB journal data to S three, you simply create an export job from within the QLDB console. And remember that there is a limit of two concurrent journal export jobs at a time. So you can only create two concurrent journal export jobs that can run simultaneously. All right, so that’s about the backup and restore alternative in QLDB. Let’s continue to the next lecture.
9. QLDB streams
Now, QLDB also offers us a feature called as QLDB streams. And QLDB streams create a continuous flow of data from your ledger’s journal to a Kinesis data stream. So you can compare this with the database activity stream that you have in Amazon Aurora database. All right, so the concept is similar. QLDB DB also pushes your journals data into kinesis data stream. And QLDB streams provide us with an at least once delivery guarantee. But there are no ordering guarantees. The revisions can be produced in the kinesis data stream out of order. All right, so that’s about the QLDB streams. Let’s continue to the next next lecture.
10. QLDB high availability, durability and an alternative to CRR
Now, let’s talk about the high availability and durability features in QLDB. So the QLDB journal is replicated across multiple az’s within the region. So this provides for the High Availability and with multiple copies for AZ. So this implies a strong durability. All right. And a right operation is acknowledged only after being written to a durable storage in multiple AZ. But remember that QLDB does not yet support CRR or cross region replication. But we do have an alternative. We already know that QLDB journal can be exported to an S three bucket. So what you can do is you can export your QLDB journal data to an S three bucket in the same region and then configure that S three bucket for CRR. So you simply use S three’s replication capabilities or CRR capabilities to replicate your QLDB data to another region. So that’s about the QLDB. High availability, durability and CRR. Let’s continue to the next lecture.
11. QLDB security
Now, let’s talk about QLDB security. So you use Im for authentication and authorization of different QLDB resources. And QLDB supports encryption at rest and InTransit. And it’s important to remember that QLDB only uses Amazon owned keys to encrypt your QLDB data. So it uses the the Amazon owned Kms keys, and it does not support CMKs, so it does not support customer master keys. Now, let’s look at the networking security. We already know that QLDB is a serverless offering. So you can use Interface VPC endpoint to allow VPC resources to connect to QLDB privately.
The Interface VPC endpoints are powered by AWS private link service. And Private Link provides for private and secure connectivity between VPCs AWS services and on premise applications. And private link essentially eliminates the need to use Internet gateways nat devices, VPN connections or the AWS direct connect connections. All right, so that’s about the networking security in QLDB. Let’s continue to the next lecture.
12. QLDB monitoring
Now, let’s talk about QLDB monitoring. And no points for guessing monitoring features are provided by Cloud Watch. So you can use alarms, logs, events, and so on. And here are some of the common metrics that you can use with QLDB like journal storage, index storage, read iOS, write iOS command latency, and so on. You don’t have to remember the names of these metrics, just us know that Cloud Watch is integrated with QLDB for monitoring support. And in addition to this, QLDB also provides log files thatprovide you with additional information. And as usual, API calls and user activities can be logged with Cloud Trail. All right, so that’s about the monitoring aspects of QLDB. Let’s continue to the next lecture.
13. QLDB pricing
Now, let’s talk about QLDB pricing. Now, we already know that QLDB is a serverless service, so you pay only for what you use. So you pay for storage per GB per month, and this includes Journal storage as well as the Index storage. And you also pay for iOS per million requests. So you pay for read iOS as well as write iOS. And just like any other service, you pay the standard data transfer fees. So that’s about Qltb. And that brings us to the end of this session as well, and I hope you do find this information useful. So let’s continue to the next section.
