04 Storage
Overview of cloud storage services.
Types of Storage
There are 4 types of storage in the cloud:
- BLOB (Binary Large Object)/Object Storage
- File Storage
- Block Storage
- Database (not covered in this lecture)
Products
| Provider | Block Storage | File Storage | Object Storage |
|---|---|---|---|
| AWS | Elastic Block Storage (EBS) | Elastic File System (EFS) | Simple Storage Service (S3) |
| Azure | Azure Managed Disks | Azure Files | Azure Blob Storage |
| GCP | Google Persistent Disk | Google Filestore | Google Cloud Storage |
Consistency Models
Consistency models define the guarantees that a storage system makes about the visibility of changes to data. There are 2 main consistency models:
- Strong Consistency
- Eventual Consistency
Strong Consistency
Strong consistency guarantees that any read operation on a data store will return the most recent write operation. This means that all replicas of the data are updated before the write operation is considered complete. Strong consistency is the most strict consistency model, and is the most difficult to implement. It is also the most expensive in terms of performance.
Eventual Consistency
Eventual consistency guarantees that if no new updates are made to a given data item, eventually all accesses to that item will return the same value. This means that the system will eventually reach a consistent state, but it may take some time. Eventual consistency is the most common consistency model in distributed systems, as it is easier to implement and has better performance.
Object Storage
Object storage is a data storage architecture that manages data as objects. As opposed to file systems which manage data as a file hierarchy, and block storage which manages data as blocks within sectors and tracks. Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier.
- All object have a REST-accessible globally unique URL.
- Objects have a key, version ID, and metadata.
- Objects are stored in containers (or buckets, naming varies by provider).
- Object storage is highly durable and scalable (e.g. AWS S3 has 11 9’s of durability).
- Object storage is usually highly available.
- Supported by many design patterns and architectures (e.g. microservices, serverless, etc.).
Different providers let the user select different storage tiers, which have different costs and availability guarantees. AWS S3 has the following storage classes:
- S3 Standard
- S3 Intelligent-Tiering (automatically moves objects between access tiers)
- S3 Standard-IA
- S3 One Zone-IA
- S3 Glacier
- S3 Glacier Instant Retrieval
- S3 Glacier Flexible Retrieval
- S3 Glacier Deep Archive
- S3 Outposts
File Storage
File storage is a type of data storage that is used to store and manage files. File storage systems are typically used in network-attached storage (NAS) environments where files are stored in a hierarchical file system. File shares can be created and accessed by multiple clients over a network (network file systems).
Block Storage
Block storage is a type of data storage typically used in storage area network (SAN) environments where data is stored in volumes, also referred to as blocks. In cloud environments, block storage is mostly used for IaaS (Infrastructure as a Service) deployments. Block storage services are attached as disks to virtual machines and provide a persistent storage solution.
IOPS
Usage of block storage is usually measured in IOPS (Input/Output Operations Per Second). Higher IOPS means better performance, but also higher costs. Providers offer different types of block storage with different IOPS and performance characteristics for various use cases.
Snapshots
Block storage services usually offer the ability to take snapshots of the disk. Snapshots are point-in-time copies of the disk that can be used to restore the disk to a previous state. If the disk was active during the snapshot, the snapshot will be crash-consistent, meaning that it may not be in a consistent state. For consistent snapshots, the disk should be quiesced before taking the snapshot.