AWS Infrastructure - Conceptual Framework (How it all Works)

 Let me share what we learned and deployed so far in my Cloud Journey.

AWS Global Infrastructure

AWS Global Infrastructure is a global network of infrastructure to provide highly available, scalable, and reliable services to its users. This infrastructure is segmented into geographical regions, availability zones, and edge locations.

Regions

Regions serve specific geographical areas which are independent of each other so that a problem in one region does not affect another. Regions are where AWS clusters its data centers.

Availability Zones

Within each AWS Region, there are multiple isolated locations known as Availability Zones (AZs). Each AZ has its own data center with redundant power, networking, and connectivity. Each AZ are at least 60 miles apart from each other, providing geographical dispersion for better fault tolerance and stability.

Edge Locations

Edge locations are sites located around the world used to cache content for faster delivery and lower latency.

Local Zones

Local Zones bring a subset of AWS services closer to the users and are designed for latency-sensitive applications. This is really useful for areas that are far from primary AWS regions, local zones bring AWS services (compute, storage, database, and networking services) closer to the users.

Public vs Private Subnets

Subnets are subdivisions you create in your VPC in order to separate and organize your resources based on the security and access needs.

The distinction between public and private subnets primarily depends on their accessibility from the internet and their role in your network architecture.

Public Subnets

In public subnets, your resources are able to send outbound connections to the internet through the Internet Gateway (IGW). Resources in a public subnet are also able to receive inbound connections from the internet. Front end servers, public APIs, Load Balancers, are all examples of what might be placed in a public subnet.

An IGW is a VPC component that provides a route for communication between the resources in the subnet and the internet. The resource in the public subnet would have a public ip address assigned to it, and the subnet’s route table would have a route to the IGW.

Private Subnets

Resources in private subnets should not be directly accessible from the internet. The resources here can send outbound connections to the internet using a NAT Gateway (placed in a public subnet), or a NAT instance. However the resources cannot be directly accessed from the internet. back-end servers, databases, or application servers are examples of what might be placed in a private subnet.

Amazon VPC

A VPC is a logically isolated private network for you and your application in AWS. You have complete control over the resources in your VPC. A VPC’s range is defined by something called a CIDR block. CIDR blocks refer to the IP address range assigned to your VPCs. Any resources within your VPC must have an IP address within this range.

Included in a VPC are:

• Subnets: Discussed previously, subnets are dividing your VPC into smaller sections.
• Route Tables: Route tables guide packets within your VPC, they determine the direction of traffic flow.
• Internet Gateway (IGW): A connection point between your VPC and the internet.
• NAT Gateway: they allow instances in a private subnet to send outbound internet traffic, but prevent inbound traffic from the internet.
• Security Groups: Security Groups protect your resources by controlling inbound and outbound traffic. They are stateful which means there doesn’t need to be an explicit rule allowing return traffic, it is automatically allowed.
• Network Access Control Lists (NACL): NACLs regulate traffic coming in and out of your subnets. With NACLs there does need to be an explicit rule to allow return traffic.

AWS IAM

Identity Access Management (IAM) is a service that allows you to securely control access to AWS resources under specific conditions. You can create and manage AWS users and groups, and user permissions to deny or allow them access to AWS resources.

An IAM user is an identity created in AWS that represents a person or service that interacts with AWS. With an IAM user you can have a human use a username and password to be able to access AWS resources. Service users are backend or application services that need to perform actions towards AWS programmatically.

Here is a few things to keep in mind when dealing with IAM users:

• Principle of Least Privilege: with IAM users we should only grant the permissions that they need in order to perform their job function. This reduces the risk of any accidental or malicious breach.
• IAM Groups: We also want users in IAM Groups and manage permissions at the group level wherever possible.
• Rotating Credentials: We want to rotate and reissue credentials to minimize the risk of all credentials being exploited.

IAM Groups

An IAM Group is a collection of IAM users and it allows managing permissions for multiple users collectively. IAM Groups makes it easier to manage permissions for multiple users who need similar access to AWS resources.

IAM Policies

An IAM Policy defines the permissions that we apply to identities and to resources. IAM Policies control what actions are allowed or denied and on which resources and with what conditions.

There are three types of IAM Policies to be aware of

1. Managed Policies: These are pre-built policies managed by AWS that you can attach to multiple users, groups or roles within your AWS account.
2. Inline Policies: These are the policies you create and manage and are embedded directly into a single user, group, or role.
3. Customer Managed Policies: These are stand-alone policies in our account that we create, offering more flexibility than managed policies.

IAM Roles

An IAM Role is an AWS identity with permission policies that are used to determine what the identity can or cannot do in AWS. Unlike users, roles do not have long-term credentials such as a password or access key. Instead, when you assume a role, it provides you with short-term security credentials for your role session. Roles are primarily used to delegate permissions to an AWS service or another user.

Designing A VPC

It will be easier to understand what constitutes a VPC by seeing a diagram of how one might be designed.

Let me explain the above diagram in detail:

• Subnet Design: Here we have 2 public subnets and 4 private subnets dispersed across 2 availability zones. There are two NAT gateways in this image to ensure high availability. To have high availability for our application and databases, I also included 4 private subnets between both AZs.
• Internet Connectivity: There is an Internet Gateway included in this diagram to ensure internet connectivity to the public subnets, and a NAT Gateway in the public subnets for the private subnets.
• Route Tables: To ensure proper routing for Internet access and internal communication, I have included route tables for the public and private subnets, which includes the database subnets.
• Security: For security, we have the route tables that manage the flow of traffic. But I also included NACLs so that only certain traffic is allowed into the subnets.

Creating A VPC (From The Console)

Now let’s actually create a VPC through the AWS console. In the image below I start by naming my VPC “project-vpc” with a CIDR block of 10.0.0.0/16. Where the 16 represents the bits in the subnet mask, since there are 8 bits in each part separated by a dot. This means there are 32 bits available in an IPv4 address. The first 16 bits will be for the Network ID and are the same for every computer on a specific network segment. The rest will be for the Host ID, the Host ID will be unique to every individual computer (with a few IP Addresses reserved by AWS).

To ensure that this is an IPv4 CIDR block, I make sure to also select “No IPv6 CIDR block”.

And just like in my diagram I choose:

• 2 Availability Zones
• 2 Public Subnets
• 4 Private Subnets
• The only difference is that I don’t include a NAT Gateway in my VPC

The rest of the selections can be left as -

Adding EC2s to my VPC (Bastion Host)

Now I can launch EC2s into my newly created VPC and its subnets. First I want to create a bastion host, which is an EC2 instance that will be located in a public subnet to be able to access another instance that is located in a private subnet. Since the instances in private subnets cannot be accessed from the internet, this is a way to connect to it.

I start by naming my new instance “bastion” and choosing the default Amazon Linux type.

Next, I choose the t2.micro and add a previously generated key pair.

Next, I choose my newly created VPC (project-vpc) and the first public subnet.

I create then a new security group that allows ssh traffic on port 22

Now I copy the public IP address of my newly created bastion host instance:

Next, I'd log onto my instance through my terminal using ssh.

Next I want to launch an instance in one of the private subnets, in this case I call it “EC2AppB”

I again choose my created VPC “project-vpc”, but this time I choose one of the private subnets.

Since I used the same key pair with all of my instances I want to securely copy (using scp) my key pair into my bastion host instance under the “ec2-user”. This way I can use this key pair file to also connect to my instance in the private subnet. I then ssh into my private instance.

And there you have it, I have created an instance in a private subnet and accessed it using a bastion host in a public subnet.

EC2

EC2 Storage

Amazon EBS is like an external harddrive that you can connect to your computer, it provides persistent storage, which means that if you turn off your computer, the data remains intact.

Instance Store is like the temporary storage on your computer. This is similar to the RAM or cache and is physically attached to the computer (EC2 instance). When you stop your EC2 instance however, this data is lost. Instance store is ideal for compute intensive workloads as it has the great I/O performance.

EBS Volume Types:

• General Purpose SSD (gp2): these volumes provide a balance of price and performance, suitable for a wide range of workloads.
• Provisioned IOPS SSD (io1): These are designed for I/O intensive workloads that require high workloads and consistent low latency
• Throughput Optimized HDD (st1): these are ideal for frequently accessed throughput intensive workloads.
• Cold HDD (sc1): suitable for less frequently accessed workloads that require a large amount of low cost storage.

Elastic Load Balancing

An Elastic Load Balancer is a service designed to automatically distribute incoming application traffic across multiple targets. These are the types of Elastic Load Balancers:

• Application Load Balancer: ALB works with HTTP/HTTPs traffic, offering advanced request routing, targeting the delivery of modern application architecture.
• Network Load Balancer: Best for TCP traffic where there are extreme performance requirements. NLB is able to handle millions of requests per second, while maintaining ultra low latencies.
• Classic Load Balancer: Best for applications built within the EC2 classic network. It works at both the connection and request level.

Auto Scaling

Auto Scaling is a service provided by AWS that automatically adjusts the number of compute instances in your deployment, based on real time data. This feature is designed to dynamically scale your application, by increasing/decreasing the amount of instances automatically depending on conditions such as traffic and utilization thresholds.

Components of Auto Scaling:

• Auto Scaling Groups (ASG): Collection of EC2 instances that share similar characteristics and are treated as a logical grouping for scaling and management. When we setup an ASG, we define the minimum and maximum number of instances in the group.
• Launch Configurations and Templates: A launch configuration is a template that an ASG uses to launch new EC2 instances. It includes configuration such as:
• Instance Type
• AMI ID
• Key Pair
• Security Groups
• Associated Block Storage

Scaling Policies: Define the scaling options that the ASG should take in response to specific conditions or alarms.

• Target Tracking Scaling: adjusts the number of instances automatically to maintain a target value for a specific metric
• Step Scaling: increases/decreases the number of instances based on a set of scaling adjustments, depending on the size of the alarm breach
• Scheduled Scaling: automatically scales the number of instances based on scheduled time points

Amazon S3

Amazon S3 is a cloud based object storage service and it allows you to store and retrieve any amount of data from anywhere on the web. S3 has really high durability and availability.

In S3 you store data in containers called buckets. These buckets have globally unique bucket names, which means that it has to be different from every other S3 bucket name.

Objects are the fundamental entities stored in S3. Objects contain the data and metadata of the file. Objects can range in size from 0 to 5 terabytes. Each object is identified with a unique key (or a name).

The key is the full path to the object, including the object name and path.

S3 has a number of storage classes optimized for various use cases and access patterns.

• Standard: Frequently Accessed & requires high durability and availability
• Intelligent Tiering: For unknown or changing access patterns. It automatically moves the data to the most cost effective access tier.
• Infrequent Access: Less frequently accessed data that requires rapid access when needed.
• One Zone-IA: Data is stored in a Single Availability Zone & Costs 20% less than standard IA.
• Glacier and Glacier Deep Archive: Lowest-cost storage for Data Archiving & Long-term backup

There are some other features to know about S3:

• Bucket Policies: JSON-based Access Control Policies that you can attach to your S3 Bucket that allow you to grant or deny permissions to your S3 Resources at the bucket level.
• With Bucket Policies you can deny access to specific AWS accounts, IAM users, or IP ranges.
• You can grant read, write, or delete permissions to objects within the bucket
• You can restrict access to specific HTTP referrers or require certain conditions to be met
• Versioning: Feature of S3 that allows you to keep multiple versions of an object within the same bucket. When versioning is enabled on a bucket, S3 automatically assigns a unique version ID to each object added to a bucket.
• Versioning allows for protection against accidental deletion/overwrites
• Versioning also gives the ability to rollback to a previous state.
• Lifecycle Policies: Allows you to automate the management of your S3 buckets based on predefined rules.
• With lifecycle policies you can transition objects to different storage classes based on age.
• You can expire objects after a specific period.
• You can delete previous versions of objects.

Amazon Relational Database Service (RDS)

Amazon RDS is a managed database service that uses SQL (Structured Query Language). Amazon RDS provides us with an easy to use platform for setting up, operating, and scaling a relational database in the cloud. With RDS the provisioning, configuration, patching, and the backups are completely automated by AWS.

Amazon RDS allows for the following SQL databases:

• Postgres
• MySQL
• MariaDB
• Oracle
• Microsoft SQL Server
• IBM DB2
• Aurora

Under the hood, RDS is built on top of an EC2 instance. RDS also has various instance types for memory, performance, and I/O.

Here are some of the benefits of RDS:

• Multi-AZ deployment for disaster recovery where data from the main database gets synchronously replicated to a standby database in another Availability Zone.
• Automatic Failover. If you have Multi-AZ enabled and there is an infrastructure failure, RDS will start an automatic failover to the standby instance to minimize downtime and data loss.
• Read Replicas. They make it easy to elastically scale out beyond the capacity constraints of a single DB instance for read-heavy database workloads. The writing operations would be done by the master database, while the reading will be done by the read replicas. In the case of read replicas, data is asynchronously replicated from the master.
• Automatic Backups. With RDS we can schedule a backup automatically once a day, at a time that we specify.

Creating an RDS Database

Let’s quickly see how to create an RDS database from the AWS console.

First we have to go to the RDS page in the AWS Console.

After clicking on the “Databases” section, we are brought to a page that lists out the databases created on your account. I haven’t created any RDS databases yet so there is nothing in this list. We can create a new database by clicking on “Create database”

Now we’re brought to the page where we can select between various options for how we want our database created. Since we want to be able to customize the options ourselves we will choose “Standard create”. I also am choosing to create a “MySQL” database.

Next, we have a few templates to choose from, but since I’m not going to use this database in any serious manner, I’ll choose “Free tier”

Next we have the option to create a password to be able to login to the database. We can also choose to use AWS Secrets Manager which can manage the password for us. For now I’ll keep it simple and create a password.

Everything else is left as it is and I click “Create database”.

We can now see the newly created database in the list.

Well this has been a deep dive into the fundamentals of AWS. There is still lots to learn so I will keep updating with my progress.

Also, stay tuned for another article where I will go over how to set up these same services using Infrastructure as Code with CloudFormation.

Thanks, and until next time.

-x-