Copyright 2010, Blue Gecko, Inc.

Send In the Clouds

High Performance Oracle 11g
in the Amazon Cloud
Jeremiah Wilton

Copyright 2010, Blue Gecko, Inc.

Jeremiah Wilton jwilton@bluegecko.net

17 years working with Oracle
Amazon.coms first DBA ~1997
Technical Lead, Blue Gecko, Inc.
! Remote administration
! Oracle, E-Business Suite, MySQL, SQL Server
Univ. of Washington Oracle instructor
President, Western Washington Oracle Users Group
Co-author: Expert Oracle Practices (Oak Table)
Frequent speaker at Oracle conferences
Current focus:
! Oracle performance on the Cloud (Amazon EC2)
! Time-based tuning for SQL Server
Copyright 2010, Blue Gecko, Inc.

Agenda
Cloud Computing Primer
Amazon Web Services overview
Deploying Oracle on Amazon Elastic Compute
Cloud with attention to performance
Copyright 2010, Blue Gecko, Inc.

What the hell is cloud computing?...I

dont know what we would do
differently in the light of cloud
computingother than change our
ads.

Larry Ellison

CEO, Oracle Corp.
Copyright 2010, Blue Gecko, Inc.

Cloud is now an annoying buzzword

Lacks specific meaning
Has many meanings
Appropriated for all kinds of products
Embraced by Marketing / PR
Conversational stand-in for specific technology
Copyright 2010, Blue Gecko, Inc.

My definition
Cloud Computing: A set of methods for
delivering over networks managed computing
resources that can be incrementally provisioned
based on demand, remotely managed via a
uniform web API, and billed based on resource
usage to avoid capital costs.
Copyright 2010, Blue Gecko, Inc.

Cloud Benefits
Little or no up-front (capital) costs
Pay as you go: Pay for as few or as many resources as
you use
Deploy new environments in minutes
Better underlying physical infrastructure than you
could (potentially) afford/build yourself
! Geographic redundancy
! Replication
! Uptime SLAs
! Secure professional data centers
Copyright 2010, Blue Gecko, Inc.

Physical infrastructure is cumbersome to scale

Time
D
e
m
a
n
d
Capacity
Demand
Demand exceeds
capacity
Infrastructure
investment
Copyright 2010, Blue Gecko, Inc.

Cloud infrastructure scales incrementally

Copyright 2010, Blue Gecko, Inc.

Cloud Computing categories

SaaS: Software as a service
PaaS: Platform as a service
IaaS: Infrastructure as a service
(BLOOHP): TBD
Copyright 2010, Blue Gecko, Inc.

Software as a Service
Web application supporting multiple private
enterprises or individuals
Replaces locally-managed enterprise software
Usually costs less
Google Apps: Email, calendar, docs
Salesforce CRM
Copyright 2010, Blue Gecko, Inc.

Platform as a Service
Development and run-time environment supporting
multiple private applications
Applications are developed by the customer on the
PaaS platform
Many have database services and caching
Force.com (proprietary development environment)
Google Apps Engine (open dev environment)
Copyright 2010, Blue Gecko, Inc.

Infrastructure as a Service
Virtual hosts, storage and other infrastructure
supporting multiple private environments
Bring/write/manage your own software
Run anything you want (within the ToS)
Scale/pay as you go
Amazon Web Services (AWS) is the global leader and
stealth behemoth
! Many iPhone, Facebook app backends run on AWS
! Many websites and digital media backed by AWS
! You can run a variety of Oracle software
Copyright 2010, Blue Gecko, Inc.

BLOOHP: Buy Lots of Our Hardware, Please

Buy your own cloud!
Lots of hardware
Lots of employees
Lots of datacenter space and resources
Antithesis of the low-cost, incremental benefits
of cloud computing
Sun Microsystems cloud strategy
Copyright 2010, Blue Gecko, Inc.

Amazon Web Services (AWS)

Many services
Available under your Amazon retail account
Vast amounts of unused space and compute capacity
Simple Storage Service (S3)
Elastic Compute Cloud (EC2) !IAAS
! Provision a running Linux or Windows server
! Variety of memory and CPU sizes
Simple Queue Service (SQS)
Simple DB
Relational Database Service (RDS)
Flexible Payments Service (FPS)
Copyright 2010, Blue Gecko, Inc.

US
East
EC2: the Elastic Compute Cloud
US West
EU West
More
Soon
Copyright 2010, Blue Gecko, Inc.

EC2s value to IT
Scaling, automation, management features
Completely flexible run anything
Common use cases
! Development / testing
! Disaster recovery site
! Student sandboxes 64-bit OEL for every student
! Prototyping / proof of concept
! Production
Pay as you go
Terminate host to stop paying
Copyright 2010, Blue Gecko, Inc.

Caveat emptor
I/O Performance is limited
! Fine for 80% of deployments at peak
! Requires testing
You need a reliable Internet connection
Not always cheaper than self-hosted
Oracle supportability question
Copyright 2010, Blue Gecko, Inc.

Will Oracle support it?

Oracle fully supports deploying Oracle Database,
Middleware, and Enterprise Manager products on the
Amazon EC2 platform. However, it may be noted that
Amazon EC2 is a virtualized environment and uses a
virtualization engine that is not supported by Oracle.
Users will therefore be directed to Amazon for any
virtualization-related issues.*
This may change (for the better) in 2010
*From http://www.oracle.com/technology/tech/cloud/faq.html#support
accessed Feb. 11, 2010
Copyright 2010, Blue Gecko, Inc.

Instances: The heart of EC2

Virtual host managed by API calls to AWS
Windows, Linux or OpenSolaris
! Many versions and distributions available
! Or roll your own
Broad range of sizes / scales
! 32-bit or 64-bit
! 1-CPU to 26-CPU (equivalent)
! 1.7G RAM 68.4GB RAM
! 8.5/hour - $2.88/hour
Copyright 2010, Blue Gecko, Inc.

Instance types and prices, Q1 2010

!"#$%"&'
()*'
+,- ./0 1$23%4' ,5%$6237 !89 :8;2<3
>-1 ?%#$@
AB"<C8DB"E2F#
!"#$!%&& " ()*+ "#,-. "/0-. 123456 789:;%6: <0#0=> ? <0#"2++
!"#&%;@: A ()* ,#>-. =>0-. /A3456 B5@C <0#1A ? <0#A=
!"#D&%;@: = ()* ">-. "/E0-. /A3456 B5@C <0#/= ? <0#E/
F"#!:95G! > ()* "#,-. 1>0-. 123456 789:;%6: <0#", ? <0#2E
F"#D&%;@: 20 ()* ,-. "/E0-. /A3456 B5@C <0#/= ? <"#"/
!2#2D&%;@: "1 ()* 1A#2-. =>0-. /A3456 B5@C <"#20 ? <"#AA
!2#AD&%;@: 2/ ()* /=#A-. "/E0-. /A3456 B5@C <2#A0 ? <2#==
*1 ECU provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.
**Long-term reserved instance pricing at reduced price per unit time also available.
Copyright 2010, Blue Gecko, Inc.

EC2 Regions and availability zones

Region
US-West
Region
US-East
Zone us-west-1a
Zone us-west-1b
Zone us-west-1c
Zone us-east-1a
Zone us-east-1b
Zone us-east-1c
Asia soon Region
EU-West
Copyright 2010, Blue Gecko, Inc.

Amazon Machine Image (AMI):

Your bootable operating system
Intel Linux any distribution
Windows, even OpenSolaris
Amazon-created
Community-created
Roll your own (Amazon provides tools)
! Can be based on your own physical server
Oracles OEL AMIs have Oracle 11g, required
packages and other Oracle software pre-installed
Copyright 2010, Blue Gecko, Inc.

Basic security features

No ingress on any port by default
You configure a security group
! Allows ingress on specific ports (SSH/RDC)
! Restrict to IP address or IP address range
You generate an RSA key pair
! Amazon installs the public key on your instance
! You keep the private key for your SSH client
Copyright 2010, Blue Gecko, Inc.

Management
All AWS services are controlled by Web API calls
HTTP SOAP requests, XML responses
Requires a signature with key data
Therefore tools are required
Two approaches:
! Manually manage services using admin tools
! Write software to call the API using libraries
! Amazon supplies Java, Ruby & Perl libraries
Depends on what you are doing
Copyright 2010, Blue Gecko, Inc.

Management tools: AWS Management Console

https://console.aws.amazon.com/ec2
Copyright 2010, Blue Gecko, Inc.

Management tools: ElasticFox

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609
Copyright 2010, Blue Gecko, Inc.

Management tools: Command line API tools

Allow scripting, logging, repeatability Unix/Linux, Windows and Mac
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351
ec2-add-group
ec2-add-keypair
ec2-allocate-address
ec2-associate-address
ec2-associate-dhcp-options
ec2-attach-volume
ec2-attach-vpn-gateway
ec2-authorize
ec2-bundle-instance
ec2-cancel-bundle-task
ec2-cancel-spot-instance-requests
ec2-cmd
ec2-confirm-product-instance
ec2-create-customer-gateway
ec2-create-dhcp-options
ec2-create-image
ec2-create-snapshot
ec2-create-spot-datafeed-subscription
ec2-create-subnet
ec2-create-volume
ec2-create-vpc
ec2-create-vpn-connection
ec2-create-vpn-gateway
ec2-delete-customer-gateway
ec2-delete-dhcp-options
ec2-delete-group
ec2-delete-keypair
ec2-delete-snapshot
ec2-delete-spot-datafeed-subscription
ec2-delete-subnet
ec2-delete-volume
ec2-delete-vpc
ec2-delete-vpn-connection
ec2-delete-vpn-gateway
ec2-deregister
ec2-describe-addresses
ec2-describe-availability-zones
ec2-describe-bundle-tasks
ec2-describe-customer-gateways
ec2-describe-dhcp-options
ec2-describe-group
ec2-describe-image-attribute
ec2-describe-images
ec2-describe-instance-attribute
ec2-describe-instances
ec2-describe-keypairs
ec2-describe-regions
ec2-describe-reserved-instances
ec2-describe-reserved-instances-offerings
ec2-describe-snapshot-attribute
ec2-describe-snapshots
ec2-describe-spot-datafeed-subscription
ec2-describe-spot-instance-requests
ec2-describe-spot-price-history
ec2-describe-subnets
ec2-describe-volumes
ec2-describe-vpcs
ec2-describe-vpn-connections
ec2-describe-vpn-gateways
ec2-detach-volume
ec2-detach-vpn-gateway
ec2-disassociate-address
ec2-fingerprint-key
ec2-get-console-output
ec2-get-password
ec2-migrate-image
ec2-modify-image-attribute
ec2-modify-instance-attribute
ec2-modify-snapshot-attribute
ec2-monitor-instances
ec2-purchase-reserved-instances-offering
ec2-reboot-instances
ec2-register
ec2-release-address
ec2-request-spot-instances
ec2-reset-image-attribute
ec2-reset-instance-attribute
ec2-reset-snapshot-attribute
ec2-revoke
ec2-run-instances
ec2-start-instances
ec2-stop-instances
ec2-terminate-instances
ec2-unmonitor-instances
ec2-version
Copyright 2010, Blue Gecko, Inc.

EC2 is Feature rich

Elastic Block Store (EBS)
Elastic IP Addresses
Virtual Private Cloud (VPC)
CloudWatch
Auto Scaling
Elastic Load Balancing
Boot from EBS
EBS Volume Snapshot (and sharing)
AMI Bundling and Sharing
Copyright 2010, Blue Gecko, Inc.

Advanced Security: Virtual Private Cloud

Internet
Amazon EC2
P
r
i
v
a
t
e

R
e
s
o
u
r
c
e
s

VPN
Gateway
VPN
Gateway
Copyright 2010, Blue Gecko, Inc.

The Security Question

The database is the core of Gracenotes business, and for
that reason, it will never run on anyone elses
infrastructure.
Steven Winter, Senior DB Engineer, Gracenote
SAS70 Type II compliant
To date no demonstrated vulnerabilities
Security is largely up to the consumer
VPC separates hosts from the Internet
Hypervisor separates environments from each other
Physical security in datacenters is exceptional
Deploy DB servers with no Internet ingress
O/S and Oracle security (plus encryption)
Test, dev and training can use obfuscated data
Copyright 2010, Blue Gecko, Inc.

Running Oracle on EC2

1. Choose an AMI (O/S)
2. Set up external access (security)
3. Start up and connect to an EC2 host
4. Determine an I/O strategy
5. Determine an O/S persistence strategy
6. Design a backup strategy
7. Create and attach EBS volumes
8. Start ASM and add volumes
9. Create or clone a database to EC2
Copyright 2010, Blue Gecko, Inc.

EC2 First Steps

Demo: Choosing an AMI in ElasticFox
! Search string oracle_11107
! ami-47c5252e
Demo: Setting up key pair and a security group in the AWS
Management Console
! Key pair rmoug
! Remote management ingress group remote
! SSH (TCP/22), VNC (TCP/5900-5910), SQL Net (TCP/1521)
Demo: Start up an EC2 instance with the EC2 API (command
line) tools
! #$%&'()&*)+,-)$#+ -.*&/0$1%1%# &&2'3(4 '#.3,# &&5#6 '.3(2
! Note in which availability zone the instance starts
Copyright 2010, Blue Gecko, Inc.

AWS Storage Options

S3: The Simple Storage Service independent of EC2
! Stores objects (files) in a namespace
! Objects up to 5G larger objects can be split
! Stored/retrieved via HTTP or physical import/export
! Access from inside or outside AWS perfect for web media
! Access control make objects public or private
! Home of Amazon Machine Images (AMIs)
! 15/GB per month, data transfer in currently free
! Replicated to three availability zones within a region
Ephemeral Storage EC2 only
! Lives with the EC2 instance
! Dies with the EC2 instance
! Default root disk and / filesystem
EBS: The Elastic Block Store EC2 only
! Volumes 1G 1T
! Not shared disk no possibility of RAC
! Attach to Linux instances as devices (/dev/sdx) ASM, LVM, filesystem
! Detach and re-attach to other instances
! Persists beyond instance termination
! Snapshot to S3 can be shared with other accounts
Copyright 2010, Blue Gecko, Inc.

EBS, S3 and EC2

s1
s2 v1 v2 v3 v4 v5
i1 i2 i3 i4 i5 i6
o1 o2 o3
o4 o5 o6
S
n
a
p
s
h
o
t
s

A
M
I
s

O
b
j
e
c
t
s

s3
s4
Snapshot volume to S3
Create
volume from
snapshot
Create
instance
from AMI
The Internet
S3
EBS
EC2
Copyright 2010, Blue Gecko, Inc.

I/O Strategy
Persistent data (databases) belong on EBS
EBS is accessed via the EC2 hosts network interface
Striping across volumes is #1 optimization
! ASM, LSR, LVM, etc.
Some recent ORION results:
! 3500 peak 8k reads / second
! 10x 1G EBS volumes with
simulated striping
! m1.large instance
Compare I/O capacity to DB requirements
! ORION to test EBS
! DBA_HIST_IOSTAT_FILETYPE for DB peak
0
500
1000
1500
2000
2500
3000
3500
4000
4500
4
0

5
0

6
0

7
0

8
0

9
0

1
0
0

1
1
0

1
2
0

1
3
0

1
4
0

1
5
0

1
6
0

1
7
0

1
8
0

1
9
0

2
0
0

Copyright 2010, Blue Gecko, Inc.

Other performance strategies

Monitor demand vs. capacity
! Use CloudWatch and Auto Scaling
! Your own monitoring and load management
Adopt a DB Time-based performance method
! No chasing ghosts
! Only address real problems
Reduce round-trips over the Internet or VPN
! Array fetch, bulk load, etc.
Frequently evaluate Internet connection options
Copyright 2010, Blue Gecko, Inc.

Configuring Storage
Demo: Create and attach EBS Volumes (on EC2 tools machine)
! #$%&$'#()#&*+,-.# &&/01# % &&(*(0,(20,0)3&1+4# -/&#(/)&52
! #$%&$'#()#&*+,-.# &&/01# % &&(*(0,(20,0)3&1+4# -/&#(/)&52
! #$%&$'#()#&*+,-.# &&/01# % &&(*(0,(20,0)3&1+4# -/&#(/)&52
! #$%&$'#()#&*+,-.# &&/01# % &&(*(0,(20,0)3&1+4# -/&#(/)&52
! #$%&$'#()#&*+,-.# &&/01# % &&(*(0,(20,0)3&1+4# -/&#(/)&52
! Take note of volume names
! #$%&())($6&*+,-.# *+,&%7$$889: &&04/)(4$# 0&;(7<=>2% &&;#*0$# ?;#*?/;:
! #$%&())($6&*+,-.# *+,&<($$88@< &&04/)(4$# 0&;(7<=>2% &&;#*0$# ?;#*?/;A
! #$%&())($6&*+,-.# *+,&8$$$887@ &&04/)(4$# 0&;(7<=>2% &&;#*0$# ?;#*?/;6
! #$%&())($6&*+,-.# *+,&88$$887B &&04/)(4$# 0&;(7<=>2% &&;#*0$# ?;#*?/;0
! #$%&())($6&*+,-.# *+,&8%$$8872 &&04/)(4$# 0&;(7<=>2% &&;#*0$# ?;#*?/;C
Demo: Start ASM and configure for EBS (on EC2 instance)
D ,+$(,$+4:0A (;;
D #$6+ EFFGE H /:;0/I &J ?;#*?/;:K +'($,#(/. $'#()#;0/I L85 ?;#*?/;:5
D #$6+ EFFGE H /:;0/I &J ?;#*?/;AK +'($,#(/. $'#()#;0/I L8% ?;#*?/;A5
D #$6+ EFFGE H /:;0/I &J ?;#*?/;6K +'($,#(/. $'#()#;0/I L8< ?;#*?/;65
D #$6+ EFFGE H /:;0/I &J ?;#*?/;0K +'($,#(/. $'#()#;0/I L89 ?;#*?/;05
D #$6+ EFFGE H /:;0/I &J ?;#*?/;CK +'($,#(/. $'#()#;0/I L8@ ?;#*?/;C5
D /- & +'($,#
M N +'(#4*
OPQRGSTUOVS W X?6+.#?+'($,#Y Z )/)<%
M ;2$( &/0,#4) &$+4:0A-'#(/. &;0/I,0/) OPRG[L85FOPRG[L8%FOPRG[L8<FOPRG[L89 \
&;0/IA'+-]4(.# ;()( &'#;-4;(4$3 #^)#'4(, &'#$+*#'3;0/I,0/) OPRG[L8@ \
&'#$+*#'3A'+-]4(.# '#$+ &'#$+*#'3A'+-]'#;-4;(4$3 #^)#'4(,
Copyright 2010, Blue Gecko, Inc.

O/S Persistence
Start with the original AMI each time
! Re-bundle every time you make a change
! /etc/oratab, ORACLE_HOME, spfile all revert
! Store them on an EBS volume filesystem
! Create symbolic links each time you start up
Boot from EBS
! AMI initially copied to an EBS volume filesystem
! Thereafter, snapshots to S3 on termination
! Snapshot is basis for EBS volume on restart
! O/S and root disk evolves over lifetime of server
Copyright 2010, Blue Gecko, Inc.

Backups
Oracle Secure Backup (OSB) Cloud Module
! RMAN Media Management Library (MML)
! Compelling alternative to tape vaulting
! Instant off-site
! Always online, geographically replicated
Demo: Back up a traditionally-hosted DB to S3
789:; '() <-==3$-,# $>-))#= +?- ,64# +@,A
@-$5(4 *)$'#.#),-= =#B#= C
-+ $3.4'#++#D @-$5(4+#,
D-,-@-+# 4=(+ -'$>*B#=32
D#=#,# *)4(,AE
789:; =*+, @-$5(4 3F D-,-@-+# @6 F*=#A
Copyright 2010, Blue Gecko, Inc.

Demo: Clone to EC2 using the S3 backup

Set up a clone Oracle instance on EC2
! /etc/oratab
! Basic init<sid>.ora
! db_name = jw02
! db_recovery_file_dest = '+RECO'
! db_recovery_file_dest_size = 2G
! Static listener.ora entry
Install the RMAN OSB Cloud MML on the EC2 instance
C(*( &C(' _?/$'0])/?+/2`/?+/2`/T04/)(,,NC(' &(`/0; aaaa \
&(`/I#3 aaaa &+)4-/#' -/#'b#.(0,N$+. &+)4](// aaaa \
&`(,,#);0' MOPQRGSTUOVS?;2/?+/2`/T`(,,#) \
&,02;0' MOPQRGSTUOVS?,02
Duplicate from S3
! Run RMAN on target (source)
Copyright 2010, Blue Gecko, Inc.

Disaster recovery use case: AWS as DR only

DB Host
DB Volumes
Traditional
Hosting Facility
Amazon Web Services
Simple Storage Service
(S3)
Flash
Recovery
Area
RMAN Backup
(OSB Cloud
Module)
RMAN Backup
15/GB
per month
256GB
under
$40/month
Also replaces
tape vaulting
service
Copyright 2010, Blue Gecko, Inc.

Disaster recovery use case: AWS as DR only

Amazon Web Services
Simple Storage Service
(S3)
Traditional
Hosting Facility
EC2 Instance
EBS Volumes
RMAN Restore
(OSB Cloud
Module)
Copyright 2010, Blue Gecko, Inc.

Disaster recovery use case: DR within AWS

EC2 Instance
DB Volumes
Region
us-east-1a
Region
us-east-1b
Region
us-east-1c
Simple Storage Service (S3)
RMAN Backup
(OSB Cloud
Module)
Copyright 2010, Blue Gecko, Inc.

Disaster recovery use case: DR within AWS

EC2 Instance
DB Volumes
Region
us-east-1a
Region
us-east-1b
Region
us-east-1c
Simple Storage Service
(S3)
RMAN Restore
(OSB Cloud
Module)
Copyright 2010, Blue Gecko, Inc.

Disaster recovery use case: AWS as DataGuard

DB Host
DB Volumes
Traditional
Hosting Facility
Amazon Web Services
Simple Storage Service
(S3)
Log Transport
VPC / LGWR
EC2 Instance
DB Volumes
RMAN Backup
(OSB Cloud
Module)
Copyright 2010, Blue Gecko, Inc.

Summary
AWS is a viable Oracle platform for many businesses,
organizations and use cases
Like all platforms, AWS requires performance testing for
the load and use profile intended
Optimizations are possible on AWS to maximize Oracle
performance
For more Oracle on AWS information:
! OTN Cloud Center
! http://www.oracle.com/technology/tech/cloud
! AWS Oracle Page
! http://aws.amazon.com/solutions/featured-partners/oracle
Copyright 2010, Blue Gecko, Inc.

Q & A