Amazon AWS

About the trainer

• S.Suresh, Founder & CEO of CloudSiksha
• 30 yrs of experience in IT industry
• Worked in major IT companies like IBM, Wipro and Oracle
• Range of assignments including Developer, Architect and Manager
• AWS Solution Architect Professional Certified
• Cloud, Storage and Linux the focus areas
3

Disclaimer
• All brand names, trademarks, logos are the property of the respective
owners
• All the images, unless credited, have been taken from the Amazon AWS
website
 • We will be working on the labs using the free tier
provided by Amazon. While we will try to ensure
that none of the labs cross the free tier threshold,
Important it is possible for various reasons that this
Disclaimer threshold could be breached and the user could be
charged. We will not be responsible for any
additional charge that may incur in your account
for any reason.
 • PuTTy for login into EC2 instances
Software • RDP for Windows instances
required for • PuTTyGen for converting .pem to .ppk files

this course • Any FTP client like FileZilla or WinSCP for

transferring files
 • Day 1
• Introduction to Cloud
• AWS Compute
• Storage
• Day 2
• Netwworking
The Program • Databases
• Day 3
• HA
• Monitoring
• User Management
• Security
Introduction to Cloud
8

What is Cloud?
• Many definitions exist
• How would you differentiate Cloud from a Large Data Center?
• What is the difference between a Private Cloud and Data Center?
• What are the similarities between Cloud and Data Centers?
9

• Here are five cloud characteristics as defined by

NIST
Cloud • On-Demand Self Service
• Broad Network Access
Characteristics • Resource Pooling
• Rapid Elasticity
• Measured Service
10

Cloud is more than Virtualization

Image Courtesy: Microsoft / HP

11

• Before we define what is a Cloud let us see the

different types of Cloud
• Divided based on Deployment
• Public
• Private
Cloud Types • Hybrid
• Divided based on Service Models
• IaaS
• PaaS
• SaaS
12

Cloud Deployment Models

Image: centerforcloud.nl
13

Cloud Service Models

Image: rividium.com
14

Cloud Infrastructure

Image Courtesy: MSDN

15

To sum it up
Amazon AWS
Amazon Web Services
• Started in 2004
• SQS Service Offered
• Storage Service in 2006
• Large suite of services
• Keeps increasing at a rapid pace
• Free tier available for one year
• Restricted services
• The largest Public Cloud Service provider
18

Cloud
Market Share
 AWS Global
Infrastructure
AWS – Regions and Availability zones

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-
availability-zones.html
Elastic Compute Cloud
(EC2)
Amazon EC2
• Elastic Compute Cloud
• Generally referred to as EC2
• Compute available on the Cloud
• Started from the AWS Console
• Easy to start and shutdown
AMI
• Amazon Machine Image
• A template which contains the operating system and other applications
• Use existing free or paid AMIs
• Create your own AMI
• Search in Amazon Marketplace
• A single AMI can be used to launch multiple instances
AMI
EC2 Instances
• AWS provides different types of EC2 instances for different workloads
• Compute intensive workloads
• Memory intensive workloads
• GPU workloads
• Storage Access based workloads
• General Purpose
http://aws.amazon.com/ec2/instance-types/ gives all the details
 • These are like firewall to the instance
• You can dictate as to which traffic is allowed and
which traffic is denied

Security • By default all traffic is denied

• Need to explicitly allow traffic
Groups • Traffic can be allowed
• For everyone
• For a particular IP
• For a particular set of IPs
 • The IP Address provided when instance starts is
not a permanent one
IP Address • Elastic IPs are permanent IPs
• One per instance given free of cost
• Additional Elastic IPs cost money
Exercise 1
• Start an EC2 Instance and Login into the instance
• Windows or Linux
• Use this data
• AMI : Windows R12 Base or Amazon Linux
• Instance type: t2.micro
• Security group
• Allow SSH if Linux
• Allow RDP if Windows
• Create new key pair
• Storage size default
Exercise 2
• Start an EC2 Instance but in a different Availability Zone
• Windows or Linux
• Use this data
• AMI : Same as in Exercise 1
• Instance type: t2.micro
• Security group
• Same as in Exercise 1
• Same as in Exercise 1
• Storage size default
EC2 Purchasing Option
• You can purchase any type of instance under the following purchasing
options:
• On-Demand Instance
• Reserved Instance
• Scheduled Reserved Instances
• Spot Instance
• Dedicated Instances
• Dedicated Hosts
http://aws.amazon.com/ec2/pricing/ provides the pricing details
Reserved Instances
Significant Standard and
Provide capacity
discounts over On Convertible
reservation
Demand Instances Reserved Instances

Standard
•I yr or 3 yr term. Upto 75%
Launch instance discount
Upto 75% discount
whenever you need •Cannot change instance family.
Can change instance size

Convertible
•3 yr term. Upto 45% discount
Instances are guaranteed •Can change instance family
to be launched
Scheduled Reserved Instances
Reservations that With a specified start time and duration
recur on a daily,
weekly, or monthly
For a one-year term
basis
Suitable for workloads Run at specified times
that don’t run Like end of week processing
continuously

One year
commitment

Priced 5 to 10% lower

than equivalent On
Demand Instances
 How
Scheduled
Reserved
Instances
Work
 • Spot instances allows users to bid for unused
capacity
• Can significantly reduce cost of
operation
• Useful for applications which can be interrupted
Spot Instances or which have checkpoints
• They differ from On Demand instances
• Will not start immediately
• Hourly price will vary
• Can be terminated if current price
crosses bid price
Spot Instances
Dedicated Instances

Dedicated instances
are instances that run Dedicated instances Pricing has two
on hardware dedicated are physically isolated components
to a single customer

From your other instances

Instance usage hourly
They are run in a VPC which are not dedicated
charges
instances

Dedicated per region fee.

Single fee irrespective of
From instances of other how many dedicated
accounts instances are running
•Current fee is $2 per hour per
region (as on Oct 8th 2016)
 Dedicated Hosts

Image: aws.amazon.com
Dedicated
Hosts
Dedicated Hosts
Allows user to control
mapping of EC2 instances Bring Your Own License Compliance and
to the underlying physical (BYOL) Regulatory requirements
hardware

Applications that
User does not get Insight available into need isolation for
control of the number of sockets compliance purposes
hypervisor though and cores can be placed on
dedicated hosts

Hence not a bare Existing server based

metal offering licenses can be used
Dedicated Host Usage and Pricing

Dedicated instances
Auto placement of Affinity can be
Allocate Dedicated Host Billing is done per must be within a VPC.
instances to Dedicated established between
from the console Dedicated Host Autoscaling is not
Host if needed host and an instance
supported

Multiple instances can

Else you can manual
be placed on same Once established, Does not depend on
specify the host on
Dedicated Host but all instance will always number of instances
which instance must
instances must be of start on that host running on it
run
same size
EBS Volume Types
• Magnetic
• General Purpose SSD
• Provisioned IOPS SSD
Exercise 4
• Detach the volume and attach to another instance
• Can you do it?
• If yes, what is the condition?
• If no, why not?
Creating EBS Volumes

EBS Volumes can be • Create a new volume

created in two ways • Create from a Snapshot

Max EBS Volume size is

16TB
• Minimum size is 1GB

Multiple volumes can be • One volume mounted to only one

attached to one instance instance
Exercise 3
• Create an EBS volume and attach it to running instance
• Volume Size: 1GB
• Volume Type: SSD
• No Encryption
• No SnapshotID
• Login into system and check if new disk is available
• Format it
• Attach it as D:
• Write a file into this drive
• Questions?
• Which Availability Zone will you create your volume?
 Instance
Store backed
AMI
EBS Backed
Instance
Snapshot
• Snapshots are Point in time copies
• Create Snapshots of EBS Volumes
• Create EBS Volume from Snapshots
• Volume can be created in a different zone
• Snapshots can be copied across regions
• Create AMI from Snapshots
Exercise 5
• Create a snapshot of the D: drive
• Use the following parameters
• Name: Any name you want
• Disk Type: Same as the original disk
• Disk Size: Same as original disk
• Question: What happens if you give a disk size less than the original?
• Create volume for snapshot
• Attach new volume to a running instance
• Question: Can you create a volume in a different availability zone?
 Creating AMI
• AWS allows us to create our own AMI
• AMI can be EBS Backed or Instance Store
Backed
• AMIs can be Private or Public
• AMIs can be created from EBS Volumes or
from Snapshots
Creating your
own AMI :
EBS Backed
 Amazon Relational
Database Service (RDS)
 • Ready to use production ready Database available
• Maintained fully by Amazon
• Easy to scale resources for the database
Amazon RDS • Support for multiple popular databases
• Oracle, MySQL, Microsoft SQL Server,
PostgresSQL, Amazon Aurora, MariaDB
High Availability of Database
• Automated backups
• Snapshot based backup
• Multi Availability Zone (Multi AZ) deployment
• Automatic failover
 RDS • Provisioned IOPS
Performance • Read Replicas
High Availability &
Scalability
High Availability
• Availability of critical resources at all times
• Database is one of the most critical resource
• Database must be available at all times
• High Availability is different from DR
Scalability
• Associated more with design
• Is the infrastructure and your program designed to scale?
• Scale Out and Scale Up models
Scale Up Vs Scale Out
• Increasing resources vertically is called Scale Up
• Ex: CPU Speed, Memory size etc
• Increasing resources horizontally is called Scale Out
• Ex: Deploying more instances
Database High Availability
• High Availability for Database provided by Multi AZ deployment option
for RDS
• Primary in one zone, secondary in another
• Automatic failover
Multi AZ Deployment
• Uses different engines for replication
• Physical replication in case of MySQL, Oracle and PostgreSQL
• Synchronous logical replication for SQL Server
• Failover due to many conditions
• Failover happens typically in one or two minutes time
• User can also initiate failover
• In case of patch upgrade etc
• Using the reboot
• Cannot use secondary to serve traffic
• Can be used for snapshots and backups
Read Replicas
• Scale Out solution for read heavy workloads
• Native replication engine used to create read replicas
• Asynchronous replication
• Only MySQL and PostgreSQL supported as of now
• Read replicas can be used to serve traffic
• They are not HiAV solution
• They are scalability solution
• Enable automatic backup before creating read replicas
Creating and Connecting to Read Replicas
• A snapshot of the database is taken and this is replicated
• Happens from Primary in case of No Multi AZ deployment
• Happens from Secondary in case of Multi AZ deployment
• Brief suspension of activity when snapshot is taken
• A new DNS end point is given to the read replica
• User can connect to this end point
• Read replicas can be promoted to primary if needed
• Replication will stop
Amazon Storage
Services
Amazon S3
• Amazon Simple Storage Service (S3)
• The very first offering of Amazon Cloud
• Internet Scale Storage
• Object Based Storage
• Access via Console and Programmatically via API
• Pay as much as you store
Storage
• Block Storage
• File Storage
• Object Storage
Objects
• You can think of objects as files except that they don’t have any hierarchy
• Files in a filesystem have an hierarchy
(/root/usr/cloudsiksha/trainingmaterial.ppt)
• Objects on the other hand exist in a flat name space
• Metadata about the object stored in the object
• Object ID used to access the Object
Accessing Objects
• Objects accessed by REST API
• GET / PUT / DELETE
• No in-place editing possible
• Cannot be directly attached to a system
Object, Block & File Differences
Object
No hierarchy
Cannot be mounted
Accessed via REST API
Metadata stored within object
Immutable
High level of scaling possible
Block
Hierarchy
Mounted as Block Device
FC or iSCSI
Metadata stored in a
designated place in filesystem
In place editing possible
Difficult to scale beyond a
certain limit
File
Hierarchy
Mounted as Filesystem
NFS/CIFS
Metadata stored in a
designated place in filesystem
In place editing possible
Difficult to scale beyond a
certain limit
Amazon Storage Classes
S3 Standard Storage Class
• Low latency and high throughput performance
• Designed for durability of 99.999999999% of objects
• Designed for 99.99% availability over a given year
S3 : Standard Storage Infrequent Access Class
• Same low latency and high throughput performance of Standard
• Designed for durability of 99.999999999% of objects
• Designed for 99.9% availability over a given year
S3 : One Zone Infrequent Access Class
• Replication in only one AZ
• Designed for durability of 99.999999999% of objects
• Designed for 99.5% availability over a given year
• Lower cost the Infrequent Access
S3: Intelligent Tiering
• Not exactly a tier of its own
• Objects stored in more high available tier initially
• Access to objects is constantly monitored
• If no access for certain number of days, objects moved to infrequent
access tier
• This lowers cost
• If objects are accessed, object moved back to higher performance tier
Reduced Redundancy Storage Class
• For storing non-critical data
• Redundancy is not as high as standard S3 storage
• 99.99% durability
• 99.99% availability
• Lower cost
• No longer recommended
Amazon Glacier
• Designed for durability of 99.999999999% of objects
• Supports SSL encryption of data in transit and at rest
• Vault Lock feature enforces compliance via a lockable policy
• Extremely low cost design is ideal for long-term archive
• Lifecycle management for automatic migration of objects
Amazon Glacier – Retrieval Times
• Standard
• 3 to 5hrs
• Bulk
• 5 to 12hrs
• Expedited
• 1 to 5mins
Glacier Deep Archive
• Lowest cost storage
• For very long duration storage (7 yrs plus)
• 11 9s durability
• Retrieval time: 12+ hrs
S3 Terms
• Bucket
• All objects are stored in buckets
• Object
• Basic entity which is stored
• Key
• Object Identifier
Amazon S3
Storage
Service
S3 Features
• Detecting Data Corruption
• Versioning
• Lifecycle rules
• Object Expiration
• Event Notifications
• Static Website Hosting
• Secure
S3 Limits
• 100 buckets per account
• No limit to number of objects in a bucket
• 0 byte to 5TB size objects
• Maximum object size is 5TB
• Maximum 5GB in one PUT operation
• Multipart upload option available
• No limit to number of objects that can be stored per account
Amazon Glacier
• Storage for Data Archiving
• Extremely Low cost
• More time required to retrieve objects
Amazon
Glacier
Usage
 • Transfer large amount of data
Amazon •
•
Transfer directly from Storage Device
Using Amazon Internal Link
Snowball • By Passes Internet
Data Management
• Storing in S3 and Glacier
• Automated backups for RDS and DynamoDB
• Snapshots for recovery
Amazon CloudWatch
Amazon CloudWatch
• Cloud Monitoring Service
• Monitor EC2 Instances
• Monitor other AWS resources
Amazon Cloud Watch
• Logs
• Custom Metrics
• Alarms
• Graphs
Pricing
• Basic Monitoring
• Free of Cost
• 5 minute interval
• Detailed Monitoring
• Charged
• I minute interval
Monitoring EC2
• System Status Check
• Status of the server on which the instance is hosted
• Instance Status Check
• Status of the EC2 Instance (VM)
• Basic Monitoring
• 7 preselected metrics at 5 mins interval
• Free of cost
• Detailed monitoring
• Same 7 metrics at 1 min interval
EC2 Metrics
• The metrics monitored by CloudWatch for EC2 instance includes
• CPU Utilization
• Disk Operations
• Network In/Out
• Status check
• Tip: Memory (RAM) is not part of the standard metric for EC2
Custom Metrics
• Custom metrics can be defined and can be monitored
• Memory Usage is not a standard metric
• We can define this as custom metric and monitor it
• Refer this Amazon AWS link to see how to use custom metric scripts
• http://docs.aws.amazon.com/AmazonCloudWatch/latest/Developer
Guide/mon-scripts-perl.html
CloudWatch Monitoring Logs
• Amazon CloudWatch Logs to monitor, store, and access your log files
from Amazon Elastic Compute Cloud (Amazon EC2) instances
• Monitor Logs in Real time
• Archive logs
• Amazon AWS Document on using CloudWatch Logs
• https://docs.aws.amazon.com/AmazonCloudWatch/latest/Developer
Guide/QuickStartEC2Instance.html
Identity and Access
Management
 • Identity and Access Management
• Users
• Policies
IAM • Groups
• Roles
• Single billing statement
 • Credentials
• MFA
• X509
IAM Security • Access Keys
• Password
• Federated Users
Virtual Private Cloud
(VPC)
Amazon VPC
• Virtual Private Cloud
• Your own Cloud within AWS Infrastructure
• Every instance launched in a VPC
• A default VPC is setup for every account
• Define you address range
• Define who is visible and who isn’t
• Free of cost
• If you do not use Hardware VPN
Types of VPC
• VPC with public subnet only
• VPC with public and private subnet
• VPC with public and private subnet and hardware connected VPN
• VPC with private subnet only and hardware VPN access
VPC – How
does it work?
Internet Gateway and Routing
• Internet Gateway for public access
• Routing tables
• Elastic IPs to connect to Internet Gateway
• NAT for internet access for private subnets
NAT Instance
• Instances in Private subnet sometimes need Internet access
• NAT instance can be launched to allow for such access
• NAT – Network Address Translation
• Private instance can access internet but they cannot be accessed
• All traffic passed through NAT instance
VPC Security
VPC Peering and Network Interfaces
• VPC Peering for connection between VPCs
• Enables two VPCs to talk to each other
• Peering only within same region
• Multiple Network Interface Support
VPC Limitations
• Five Amazon VPCs per AWS account per Region
• Two hundred subnets per Amazon VPC
• Five Amazon VPC Elastic IP addresses per AWS account per Region
• One Internet Gateway per VPC
• Five Virtual Private Gateways per AWS account per Region
• Fifty Customer Gateways per AWS account per Region
• Ten IPsec VPN Connections per Virtual Private Gateway
Advance Topics
Elastic Load Balancer
Elastic Load • Classic Load Balancers
Balancer • Application Load Balancers
Elastic Load
Balancer
(ELB)
ELB Benefits
• Load distribution across multiple instances
• ELB Works across zones
• Continuous Health Monitoring
• Can associate a domain name with the Load Balancer
ELB : How does it work?
• Two components
• Load Balancers
• Controllers
• Create a Load Balancer
• A DNS Name is provided for the Load Balancer
• Client will connect on this DNS Name
• IP address of Load Balancer will be returned to the client by the DNS
service
• Accepts incoming traffic and routes request to EC2 instances
 Session • Duration based session stickiness
Stickiness • Application controlled session stickiness
Auto Scaling
 How Auto
Scaling works
Thank You
suresh@cloudsiksha.com