Skip to content

Chapter 6 of 8

Databases, Messaging & Analytics

Managed relational databases on AWS start with Amazon Relational Database Service (RDS), supporting MySQL, PostgreSQL, MariaDB, Oracle, SQL Server, and Amazon Aurora (which is MySQL or PostgreSQL compatible and up to five times faster). RDS handles provisioning, patching, backups, and replication so applications focus on data rather than operations. Multi-AZ deployments add a synchronous standby replica in a different Availability Zone for automatic failover (DNS flips to the standby in sixty to one hundred twenty seconds) and high availability, but the standby is not readable, and the goal is resilience rather than read scaling. Read scaling comes from Read Replicas, which use asynchronous replication, can live in different regions, and are explicitly readable. Automated backups are enabled by default with one- to thirty-five-day retention and point-in-time recovery roughly every five minutes; manual snapshots persist indefinitely until deleted, can be shared across accounts, and are copied across regions. Both kinds of backup are stored in S3.

Amazon DynamoDB delivers single-digit-millisecond performance at any scale as a fully managed NoSQL key-value and document database. Tables require a partition key plus an optional sort key, run serverless without infrastructure to manage, and offer both on-demand and provisioned capacity modes (the latter specifies Read and Write Capacity Units with optional Auto Scaling), and you can switch between modes once every twenty-four hours. Global Secondary Indexes use a different partition key and sort key, can be created or modified at any time, and have their own throughput; Local Secondary Indexes share the base table's partition key with a different sort key and must be created at table-creation time, sharing throughput with the base table. Global Tables replicate DynamoDB across regions for active-active workloads.

For messaging and decoupling, Amazon Simple Queue Service (SQS) is a fully managed message queue with Standard queues offering nearly unlimited throughput with at-least-once delivery and best-effort ordering, and FIFO queues providing exactly-once processing with strict ordering and up to three thousand messages per second when batching. Messages persist up to fourteen days (default four). The Visibility Timeout (default thirty seconds, maximum twelve hours) hides a message from other consumers while it is being processed; set it longer than the worst-case processing time to avoid duplicate processing. Long Polling enables ReceiveMessage to wait up to twenty seconds for messages and reduces empty responses and API cost. DelaySeconds hides a message for up to fifteen minutes, perfect for deferred tasks or short retries. Dead Letter Queues receive messages that fail processing after a configurable maxReceiveCount, isolating poison messages so the main queue keeps flowing. Amazon Simple Notification Service (SNS) is the pub/sub counterpart: publishers send to topics, subscribers receive through email, SMS, HTTP, SQS queues, or Lambda functions, and message filtering routes only relevant events. Combining SNS to SQS delivers fan-out with persistence and per-consumer scaling. Amazon EventBridge is a serverless event bus with built-in sources for AWS services, SaaS apps (Datadog, Zendesk, and others), and custom apps, routing based on content filtering rules to Lambda, SQS, SNS, Step Functions, Kinesis, and more, and replacing CloudWatch Events with additional features such as the schema registry.

Real-time analytics flow through Amazon Kinesis Data Streams, which delivers real-time, durable, ordered records to shards processed by Lambda or KCL applications with sub-second latency, with replay up to three hundred sixty-five days through extended retention; Kinesis Data Firehose loads streaming data into S3, Redshift, OpenSearch, or custom HTTP endpoints with near-real-time delivery (lowest around sixty seconds) and optional Lambda transformations, but lacks replay, and Managed Service for Apache Flink (formerly Kinesis Data Analytics) lets you author SQL or Flink code to transform streams end to end. AWS Glue is the serverless data integration service, built around the Hive-compatible Glue Data Catalog, Crawlers that auto-detect schemas, Glue Jobs that run Apache Spark in Python or Scala, and a visual Glue Studio. Amazon Athena queries data in S3 using standard SQL with pricing of five dollars per terabyte scanned, dramatically reduced by partitioning and columnar formats such as Parquet and ORC. Amazon Redshift is the managed cloud data warehouse using columnar storage and massively parallel processing; Redshift Serverless bills by RPU-second, and Spectrum extends queries directly to S3 without loading. Amazon EMR runs Apache Spark, Hadoop, Hive, Presto, Flink, and HBase on EC2 or EKS (with an EMR Serverless option), handling provisioning and scaling for large ETL, ML, and log analytics workloads. AWS Lake Formation centralizes data lake permissions (table- and column-level) atop S3 and the Glue Catalog, while Amazon QuickSight provides serverless business intelligence with ML Insights and pay-per-session pricing. Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) covers full-text search and log analytics with multi-AZ domains, fine-grained access control, and integrations into Kinesis Firehose and CloudWatch Logs. AWS Step Functions coordinates distributed components through Amazon States Language JSON state machines in two flavors: Standard for long-running workflows up to one year with exactly-once execution, and Express for short-lived workloads up to five minutes with at-least-once semantics and high throughput.

All chapters
  1. 1Cloud Computing & AWS Foundations
  2. 2Identity, Security & Governance
  3. 3Compute Services
  4. 4Storage Services
  5. 5Networking & Content Delivery
  6. 6Databases, Messaging & Analytics
  7. 7Application Services, Monitoring & DevOps
  8. 8AI/ML, IoT & Edge Computing

Drill it

Reading is not remembering. These come from the Cloud Computing AWS deck:

Q

What is cloud computing?

Cloud computing is the on-demand delivery of IT resources over the internet with pay-as-you-go pricing. Instead of owning physical data centers, you rent comput...

Q

What are the three main cloud service models?

IaaS (Infrastructure as a Service) — You manage OS, apps, and data; provider manages hardware (e.g., EC2)PaaS (Platform as a Service) — Provider manages OS and...

Q

What is the AWS Shared Responsibility Model?

AWS and the customer share security duties:AWS is responsible for security of the cloud — hardware, networking, facilities, and managed services.The customer is...

Q

What are AWS Regions and Availability Zones?

An AWS Region is a geographic area (e.g., us-east-1) containing multiple isolated data centers. An Availability Zone (AZ) is one or more discrete data centers w...