Amazon Web Services

VIBIN ANDREWS
 Introduction

Vibin Andrews
Director
Vismi Technologies Pvt. Ltd.
 Introduction

 Participants Introduction
 Cloud Computing Experience
 Expectations from the training
 Overview

 AWS Fundamentals
 S3
 EBS
 EC2
 IAM
 RDS
 AutoScaling
 ELB
 Route53
 CloudFront
 CloudWatch
 CloudFormation
 Module:1

 Fundamentals of Cloud Computing

 Advantages of Cloud Computing
 Fundamentals of Amazon Web Services
 Regions

 Availability Zones

 Edge Locations

 AWS Services
 Cloud Computing

On-demand delivery of IT resources

and applications via the Internet with
pay-as-you-go pricing
 Cloud Computing

Modes of Cloud Computing

 IaaS – Infrastructure as a Service
 SaaS – Software as a Service
 PaaS – Platform as a Service

Cloud Computing Deployment Modes

 Public Cloud
 Private Cloud
 Hybrid Cloud
 Advantages of Cloud Computing

 Reduce Capital Expenditure

 Low investment on Data Centers, Servers, Real estate etc.

 Benefits from economies of scale

 Scale up or down quickly as per usage
 Increase speed of deployment
 Stop spending money on running and maintaining
Data Centers
 Easy global deployment
AWS Platform
AWS History
AWS vs. On-premises Infrastructure
Flexible Capacity
On Premises vs. AWS
AWS Platform
 AWS Global Infrastructure

 12 Regions
 Multiple
Availability
Zones per Region
 54 Edge
Locations
 AWS Regions

• AWS Region is a collection of computing resources at a specific geographic

location
• Each Region is completely Independent
• Each Availability Zone (AZ) is isolated
• AZ's in a Region are interconnected through Low Latency Links
AWS Regions
AWS Availability Zones
AWS Edge Locations
 AWS Management Console Demo

 http://aws.amazon.com
AWS Platform: Networking
AWS Platform: Compute
AWS Platform: Storage
AWS Platform: Database
AWS Platform: Analytics
AWS: Application Services
AWS: Management Services
AWS Products and Services
 Knowledge Check

 Where can you locate AWS products and services?

 If you have more than one server, which feature

will help achieve high availability?

 How would you control protocol level access to

your EC2 instances?
Module 2: Storage
Amazon S3
Amazon S3: Use Cases
 Amazon S3

 Buckets
 Objects
 Keys
 Regions
 Amazon S3 Data Consistency
Model
 Amazon S3

 Amazon S3 stores data as objects in Buckets

 An object is comprised of a file and optionally any
metadata that describes the file
 You can have up to 100 buckets in each account
 You can control access to the buckets and its objects
 Amazon S3: Buckets

 Container for objects stored in Amazon S3

 Every object is contained in a bucket
 If the object named photos/puppy.jpg is stored in
the johnsmith bucket, then it is addressable using
the URL
http://johnsmith.s3.amazonaws.com/photos/pu
ppy.jpg
 Organizes the Amazon S3 namespace at the
highest level
 Bucket names must be DNS compliant in all regions.
 Identifies the account responsible for storage and
data transfer charges
 Plays a role in access control
 Amazon S3: Buckets

 Bucket names must be at least 3 and no more than

63 characters long.
 Bucket names must be a series of one or more labels.
Adjacent labels are separated by a single period (.).
Bucket names can contain lowercase letters,
numbers, and hyphens. Each label must start and
end with a lowercase letter or a number.
 Bucket names must not be formatted as an IP
address (e.g., 192.168.5.4).
 Amazon S3: Versioning

 Helps you keep multiple variants of an

object in the same bucket
 Versioning-enabled buckets enable you to
recover objects from accidental deletion
or overwrite
 If you delete an object, instead of removing it
permanently, Amazon S3 inserts a delete marker,
which becomes the current object version
 If you overwrite an object, it results in a new
object version in the bucket. You can always
restore the previous version
 Amazon S3: Objects

 Objects are the fundamental entities

stored in Amazon S3
 Objects consist of object data and
metadata
 The data portion is opaque to
Amazon S3
 The metadata is a set of name-value
pairs that describe the object
 Eg. date last modified, and standard HTTP
metadata, such as Content-Type
 Amazon S3: Keys

 Key is the unique identifier for an

object within a bucket
 Every object in a bucket has exactly
one key
 For e.g. in the URL
http://doc.s3.amazonaws.com/2006-03-01/AmazonS3.wsdl

 "doc" is the name of the bucket and

"2006-03-01/AmazonS3.wsdl" is
the key
 Amazon S3 Object Lifecycle

 Lifecycle management defines how Amazon S3

manages objects during their lifetime. Some objects
you store in an Amazon S3 bucket might have a well
defined lifecycle:
 Log files
 Archive documents
 Digital media archives
 Financial and healthcare records
 Long term database backups
 Data that must be retained for regulatory compliance
 Amazon S3: Regions

 You can choose the geographical

region where Amazon S3 will store the
buckets you create
 You might choose a region to optimize
latency, minimize costs, or address
regulatory requirements
 There are 11 Regions spread across
different geographical regions all over
the world
Amazon S3: Data Consistency Models

 Amazon S3 achieves high availability

by replicating data across multiple
servers within Amazon's data centers
 If a PUT request is successful, your
data is safely stored
 However, information about the
changes must replicate across
Amazon S3
 Amazon S3: Data Consistency Models

 A process writes a new object to Amazon S3 and immediately

attempts to read it. Until the change is fully propagated, Amazon S3
might report "key does not exist."
 A process writes a new object to Amazon S3 and immediately lists
keys within its bucket. Until the change is fully propagated, the
object might not appear in the list.
 A process replaces an existing object and immediately attempts to
read it. Until the change is fully propagated, Amazon S3 might
return the prior data.
 A process deletes an existing object and immediately attempts to
read it. Until the deletion is fully propagated, Amazon S3 might
return the deleted data.
 A process deletes an existing object and immediately lists keys
within its bucket. Until the deletion is fully propagated, Amazon S3
might list the deleted object.
 S3 Storage Classes

 Amazon S3 Standard
 Minimum Storage Duration: Nil

 Retrieval Fee: Nil

 Amazon S3 Standard – Infrequent Access

 Minimum Storage Duration: 30 days

 Retrieval Fee: Per GB Retrieved

 Glacier
 Minimum Storage Duration: 90 days

 Retrieval Fee: Per GB Retrieved

Amazon S3: Reduced Redundancy Storage

 RRS enables customers to reduce

their costs by storing non-critical,
reproducible data at lower levels of
redundancy than Amazon S3
standard storage
 RRS provides 99.99% durability of
objects over a given year
 AWS charges less for using RRS
than for standard Amazon S3
storage
 Amazon S3: Bucket Policies

 Bucket policies provide centralized, access control to

buckets and objects
 The policies are expressed in AWS access policy
language
 For example, an account could create a policy that
gives a user write access:
 To a particular S3 bucket
 From an account's corporate network
 During business hours
 From an account's custom application (as identified by a user
agent string)
 Amazon S3: Bucket Policies

 You can use the AWS Policy

Generator to create a bucket policy
for your Amazon S3 bucket
 You can then use the generated
document to set your bucket policy
by using the Amazon S3 console
 Amazon S3: Access Control Lists

 ACL enable you to manage access to buckets and

objects
 You can use ACLs to grant basic read/write
permissions to other AWS accounts
 There are several ways you can add grants to your
resource ACL
 Use AWS Management Console or API
 S3: ACL vs. Bucket Policies

 Unlike access control lists which can grant

permissions only on individual objects, bucket
policies can either add or deny permissions across all
(or a subset) of objects within a bucket
 With one request an account can set the permissions
of any number of objects in a bucket
 Only the bucket owner is allowed to associate a
policy with a bucket
 Amazon S3: Multipart Upload

 The Multipart upload API enables you to upload

large objects in parts
 These object parts can be uploaded independently, in
any order, and in parallel
 You can use a Multipart Upload for objects from 5
MB to 5 TB in size
 Advantages of Multipart Upload

 Improved throughput — You can upload parts in

parallel to improve throughput
 Quick recovery from any network issues —
Smaller part size minimizes the impact of restarting a
failed upload due to a network error.
 Pause and resume object uploads — You can
upload object parts over time. Once you initiate a
multipart upload there is no expiry; you must explicitly
complete or abort the multipart upload.
 Begin an upload before you know the final object
size — You can upload an object as you are creating it.
 Amazon S3: Pre-signed URL

 A pre-signed URL gives you access to the object

identified in the URL
 The creator of the pre-signed URL should have
permissions to access that object
 If you receive a pre-signed URL to upload an object,
you can upload the object only if the creator of the
pre-signed URL has the necessary permissions to
upload that object
 You can generate a pre-signed URL
programmatically using the AWS SDK for Java or the
AWS SDK for .NET
 S3: Pre-signed URL Use Cases

 All objects and buckets by default are private

 The pre-signed URLs are useful if you want your
user/customer to be able upload a specific object to
your bucket, but you don't require them to have AWS
security credentials or permissions
 When you create a pre-signed URL, you must
provide your security credentials, specify a bucket
name an object key, an HTTP method (PUT of
uploading objects), and an expiration date and time
 The pre-signed URLs are valid only for the specified
duration
 Protecting Data in Amazon S3

 Objects are redundantly stored on multiple devices

across multiple facilities in an Amazon S3 region
 Amazon S3 also regularly verifies the integrity of
data stored using checksums
 If Amazon S3 detects data corruption, it is repaired
using redundant data
 Amazon S3 calculates checksums on all network
traffic to detect corruption of data packets when
storing or retrieving data
 Amazon S3 further protects your data using
versioning
 S3: Protecting Data Using Encryption

 Use Server-Side Encryption – You

request Amazon S3 to encrypt your object
before saving it on disks in its data centers
and decrypt it when you download the
objects.
 Use Client-Side Encryption – You can
encrypt data client-side and upload the
encrypted data to Amazon S3. In this case,
you manage the encryption process, the
encryption keys, and related tools
 Amazon S3: Pricing

 Amazon S3 has three pricing components:

 Storage (per GB per month)

 Data transfer in or out (per GB per month)

 Requests (per n thousand requests per month)

 For new customers, AWS provides a free usage tier

which includes up to 5 GB of Amazon S3 storage
 For more details:
http://aws.amazon.com/s3/pricing/
 Amazon S3 Facts

 Able to store unlimited number of objects in a Bucket

 Objects up to 5TB; no bucket size limit
 Designed for 99.99999999% durability and 99.99%
availability of objects over a given year
 http/s endpoint to store and retrieve any amount of data,
at any time, from anywhere on the web
 Highly scalable, reliable, fast and inexpensive
 Optional server-side encryption using AWS or customer-
managed client side encryption
 Access logs for auditing
 Provides standards based REST and SOAP interfaces
 Amazon S3: Use Cases

 Backup and Archiving

 Content Storage & Distribution
 Static Website Hosting
 Big Data Analytics
 Cloud-native Application Data
 Disaster Recovery
 S3 Technology

 Technology developed by Amazon

 Dynamo – Object Storage Architecture
 Highly Available Key-Value Storage System
 http://www.allthingsdistributed.com/2007/10/ama
zons_dynamo.html
 S3 Lab

https://qwiklabs.com/
 Amazon Elastic Block Store

 EBS provides Block Level Storage for use with EC2

 Highly Available and Redundant
 EBS volume data is replicated across multiple
servers in an Availability Zone
 EBS Volume attached to an EC2 instance persist
independently from the life of the instance
 EBS is recommended when data changes frequently
and requires long-term persistence
 Amazon EBS

 EBS provides raw, unformatted, block-level storage

 Well suited for file systems, database etc.
 You can launch EBS volumes as encrypted volumes
 When you encrypt a volume and attach to an EC2
instance, data stored at rest on the volume, disk I/O, and
snapshots created from the volume are all encrypted
 Multiple EBS volumes can be attached to the same
instance
 If you attach multiple volumes to an instance, you can stripe data
across the volumes for increased I/O and throughput performance
 Amazon EBS

 You can create Amazon EBS volumes from 1 GB to 1 TB

in size
 One EBS volume can be attached to only one instance at
a time
 EBS volumes available as: General Purpose (SSD),
Provisioned IOPS (SSD), and Magnetic
 General purpose SSD volumes supports up to 3
IOPS/GiB, with the ability to burst to 3,000 IOPS
 Provisioned IOPS (SSD) volumes - You can provision a
specific level of I/O performance, up to 4000 IOPS per
volume
 Amazon EBS

 Amazon EBS volumes behave like raw, unformatted

block devices
 You can create point-in-time snapshots of Amazon
EBS volumes
 These snapshots are stored in S3
 Snapshots protect data for long-term durability
 You can use snapshots to create new EBS volumes
 Snapshots can be copied across AWS regions
 Amazon EBS

 Amazon EBS volumes are created in a specific

Availability Zone
 It can be attached to any instances in that same
Availability Zone
 To make a volume available outside of the
Availability Zone, you can create a snapshot and
restore that snapshot to a new volume anywhere in
that region
 Snapshots can be copied to other regions, which can
be used to create EBS volumes there
 EBS Snapshots

 Snapshots are incremental backups

 only the blocks on the device that have changed after your
most recent snapshot are saved
 Active snapshots contain all of the information needed to
restore your data to a new EBS Volume
 Snapshots taken from encrypted volumes are
automatically encrypted
 Volumes created from encrypted snapshots are also
automatically encrypted
 Amazon EBS Facts

 Three volume types: General Purpose (SSD), Provisioned

IOPS (SSD), and Magnetic
 Can use to create RAID configuration for a server
 Off-instance block storage that persists independently
 Volumes behave like unformatted block devices for Linux
or Windows instances
 Designed for 99.999% availability
 Seamless AES-256 encryption of EBS data volumes and
snapshots
 Encryption of data as it moves between EC2 instances
and EBS data volumes
 Amazon EBS use cases

 OS – use for boot/root volume, secondary volumes

 Databases – scales with your performance needs
 Enterprise applications – Provides reliable block
storage to run mission-critical applications
 Business continuity – Minimize data losses and
recovery time by regularly backing up using EBS
Snapshots
 Amazon S3 and Amazon EBS

Amazon EBS Amazon S3

Paradigm File system Object store
Performance Very fast Fast
Redundancy Across multiple servers in Across multiple facilities
an Availability Zone and on multiple devices
within each facility
Security EBS Encryption – Data Server Side Encryption
volumes and snapshots
Access from the internet No Yes
Typical use case It’s a disk drive Write once, read many
 Amazon Glacier

 Amazon Glacier is a long term data archival solution

from AWS
 Storage service optimized for infrequently used data
 Secure, durable, and extremely low-cost storage
service
 You can store data for $0.01 per gigabyte per month
 Amazon Glacier is optimized for infrequently
accessed data
 Data retrieval time 3~5 hours
 Glacier: Benefits

 Low cost – starting at $0.01 per gigabyte per month

 Secure – supports data transfer over SSL and
automatically encrypts data at rest
 Durable - highly durable storage infrastructure
designed for online backup and archival
 S3 Lifecycle policies can be used to move
infrequently accessed objects from Amazon S3 to
Amazon Glacier
 Glacier: Use cases

 Media Asset Archiving

 Healthcare Information Archiving
 Enterprise Information Archiving
 Scientific Data Storage
 Digital Preservation
 Magnetic Tape Replacement
 AWS Storage Gateway

 Connecting an on-premise software appliance with

cloud-based storage
 Allows you to securely store data in the AWS cloud
 It provides low-latency performance by maintaining
frequently accessed data on-premises
 Securely stores all your data in Amazon S3 or
Amazon Glacier
 AWS Storage Gateway Configurations

 Gateway Cached Volumes

 Gateway Stored Volumes
 Gateway Virtual-Tape Library
 Gateway Cached Volumes

 You can store your primary data in Amazon S3, and

retain your frequently accessed data locally
 Benefits:
 cost savings on primary storage
 minimize the need to scale your storage on-premises
 retain low-latency access to your frequently accessed
 Gateway-Stored Volumes

 Configure your on-premises data gateway to store

your primary data locally
 Asynchronously back up point-in-time snapshots of
this data to Amazon S3
 It can be used as off-site backup
 Recover the data locally or from Amazon EC2
 Gateway-Virtual Tape Library

 Limitless collection of virtual tapes

 Each virtual tape can be stored in a Virtual Tape
Library backed by Amazon S3 or a Virtual Tape Shelf
backed by Amazon Glacier
 It provides industry standard iSCSI interface which
provides your backup application with on-line access
to the virtual tapes
 Immediate or frequent access to data contained on a
virtual tape is not required, you can use your backup
application to move it from its Virtual Tape Library
to your Virtual Tape Shelf – Reduce costs
 AWS Storage Gateway Use Cases

 Backup
 Disaster Recovery and Resilience
 Corporate File Sharing
 Data Mirroring to Cloud based Compute Resources
 Virtual-Tape Library:
 Magnetic Tape Replacement for Archiving and Long-Term
Backup
 AWS Import Export

 AWS Import/Export accelerates moving large

amounts of data into and out of the AWS cloud using
portable storage devices for transport
 Faster than Internet transfer and more cost effective
than upgrading your connectivity
 AWS Import/Export supports data transfer into and
out of Amazon S3 buckets
 Use Cases for AWS Import Export

 Data Cloud Migration

 Content Distribution
 Direct Data Interchange
 Offsite Backup
 Disaster Recovery
 Knowledge Check

 What AWS Service would help support your web

application to offload serving static assets and store
user uploaded images and video off-instance?

 Important application and operations data (logs, user

content etc.) needs to be durably backed up and
archived. What AWS service can help?

 What AWS service will help with your relational

database on EC2 with log and data files that consume
40GB and grow at a rate of 5%/month?