DIGITAL AGE ENTERPRISE

YouTube, LLC

“The Future of Online Video”

- from
o Concept
Co cept to Hyper-Growth!
ype G o t

Presented by: Gopalakrishnan

March 06th 2008 Information Technology System | University of Sussex

 - an Overview

… FFounded
d d in
i February
F b 14th 2005,
2005 by
b Chad
Ch d Hurley
H l (CEO),
(CEO) Steve
S Ch
Chen (CTO) &
Jawed Karim as an Video Sharing Website.
… Acquired by Google Inc., for $1.65 Billion on October 16th 2006
… 2ndd most popular website in the World. [Sourced: Alexa.com]
… Video Sharing Website, users can Upload, View and Share Video Clips.
… Uses Adobe Flash technology to display wide variety of video content.
… Available in 19Language (au, br, ca, fr, de, hk, ie, it, jp, kr, mx, nl, nz, pl, ru, es, tw, uk, us)
… Headquarters in San Bruno, California, US.
… Revenue $15 - $50 Million (approx).
… YouTube Mobile launched on 15th June 2007
… YouTube Remixer launched on 18th June 2007

ICT Policy & Strategy Presentation | University of Sussex

Source: www.youtube.com | www.alexa.com | www.wikipedia.org| www.usatoday.com
Photo Credits: Martin E. Klimek (USA Today)
 - Concept

… Evolution of YouTube Concept

† Killer Applications – LiveJournal (1999), HotOrNot (2000), Wikipedia (2001), Friendster
(2002), del.icio.us (2003), Flickr (2004).
† State of Art in Video Sharing in 2004 – Directory Listing Video (search term)
term), Upload &
Download via HTTP. Issues with Codec in Playing.
† Video Viewing Experience – Windoes Media Player (buffering..10%...15%...20%....
Wrong Codec... Downloading codec..10%...20%....Can’t decode Video...!! Audio Only!!).
R l Pl
Real Player – Takes
T k over your system
t (Resource).
(R )
† “Clip Cultures” emerges. – Feb 1st 2004 (Wardrobe Malfunction) and October 15th
2004 (Stewart on Crossfire)
† “BitTorrent
BitTorrent Effect
Effect” – Video Solutions over Internet. Large audience is Over the Internet.
† Video Sharing Site: Upload & Watch Video!
„ Broadband at Home
„ Macromdiea (now Adobe) Flash7
„ Digital Camera & Cellphones

Source: video.google.com | www.youtube.com

 - Concept

… Y T b Project
YouTube P j KiKickk Starts!
S !
† February 14th 2005: Work Begins
† April 23rd 2005: First Video
† Promote through Wired Reporters (no one replied!)
† Talk to VC’s (Venture Capitalist) – “Cute product! So what?”
† Marketing 101: E-Mail Your Friends
† May 2005: Slow Start!
† Promote via Craiglist (Upload Video & Earn $)
† June 2005 Revamp – Increase views through “Related Videos”,
“Refer
Refer Friends
Friends”, “External
External Player”
Player .
† August 2005: Catching On! Pitch In to VC’s.
† Sequoia Funds YouTube $3.5 Million.
† Acquired by Google for $1
$1.6
6 Billion
Billion.

Source: video.google.com | www.youtube.com

 - in Numbers

… 100 Million Video views per day

… 70,000 videos uploaded per sec (~1/second)
… 30 Million Visitors per month
… 58% of Internet Videos watched on YouTube
… Time spent / user / day: ~30 minutes
… 6 1 million
6.1 illi videos
d (requiring
( 45 TB off storage space)) in August
A 2006
… 74,200,000 Videos available in March 2008 (March 6th 2008)
… 500, 000 user accounts in August 2006
… 20% Growing rate every year
… Super computer – 400 Nodes and a 10 Gigabit Ethernet connection

Source: www.wsj.com | www.wikipedia.org | www.newsweek.com | www.alexa.com

 - in Statistics

Source: Alexa Traffic Details | www.alexa.com Statistics of YouTube.com vs. MySpace.com vs. Wikipedia.org vs. Yahoo.com
 - Architecture

… Pl f
Platform
† Apache
† Python
† Linux (SuSe)
† MySQL
† psyco, a dynamic python -> C Compiler
† lighttpd for video instead of Apache

… The Team
† 2 System Administrators, 2 Software Architect
† 2 Feature Developers, 2 Network Engineers, 1 DBA

Source: video.google.com | www.youtube.com | www.newsweek.com

 - Architecture

… R i for
Recipe f handling
h dli rapid
id growth
th
while (true)
{
identify_and_fix_bottlenecks();
d i k()
drink();
sleep();
notice_new_bottleneck();
ti b ttl k()
This loop runs many times a day!
}

Source: YouTube Scalability [URL: http://video.google.com/videoplay?docid=-6304964351441328559]

by Cuong Do (Engineering Manager –YouTube). Photo snapshot from Video.
 - Architecture

… W b Servers
Web S
† NetScalar is used for load balancing and caching static content.
† Run Apache with mod_fast_cgi
† Python allows rapid flexible development and deployment.
† For high CPU intensive activities like encryption, they use C extensions
† Row level caching in the database.

… Video Serving
† Each video hosted by a mini-cluster.
† Most popular content is moved to a CDN (Content Delivery Network)
† Less popular content (1-20 views per day) uses YouTube servers in various colo sites.
† Servers use the lighttpd web server for video

Source: video.google.com | www.youtube.com

 - Architecture

… S i Video
Serving Vid K
Key Points
P i
† Keep it simple and cheap.
† Keep a simple network path.
† Use commodity hardware.
† Use simple common tools.
† Handle random seeks well (SATA, tweaks).

… Serving Thumbnails
† 4 thumbnails for each video so there are a lot more thumbnails than videos.
† Thumbnails are hosted on just a few machines.
† Saw problems associated with serving a lot of small objects.
† Started using Google's BigTable.

Source: video.google.com | www.youtube.com

 - Architecture

… D b
Databases
† Use MySQL to store meta data like users, tags, and descriptions.
† Suffered from replica lag.
† Went to Database Partitioning.
† Better cache locality which means less IO
† Can now scale database almost arbitrarily.

… Data Center Strategy

† Used Managed Hosting Providers. (“Living Off Credit Cards” - Cuong Do)
† Went to a colocation arrangement - customize everything and negotiate
† Use 5 or 6 data centers plus the CDN.
† For images latency matters, especially when you have 60 images on a page.
† Images are replicated to different data centers using Google’s BigTable.

Source: video.google.com | www.youtube.com

 - Architecture

… O
Outcome
† Stall for time. Creative and risky tricks can help you cope in the short term while you work
out longer term solutions.
† Prioritize.
P i iti KnowK what's
h t' essential
ti l tto your service
i and
d prioritize
i iti your resources and
d efforts
ff t
around those priorities..
† Pick your battles. Don't be afraid to outsource some essential services. YouTube uses a
CDN to distribute their most popular content. Creating their own network would have taken
too long and cost too much.
† Keep it simple! Simplicity allows you to rearchitect more quickly so you can respond to
problems.
† Shard. Sharding helps to isolate and constrain storage,
Shard storage CPU
CPU, memory
memory, and IO
IO. ItIt'ss not just
about getting more writes performance.
† You succeed as a team! Have a good cross discipline team that understands the whole
system and what's underneath the system. “With a good team all things are possible.”

Source: video.google.com | www.youtube.com

 - Recap

… Overview of YouTube
… Concept of YouTube
… S i i off YouTube
Statistics Y T b
… Architecture of YouTube
… Scalability of YouTube

“Just because ‘experts’ reject it doesn’t mean its a bad idea. In certain areas there no
expert, if there were experts why didn’t they develop this? If you are doing something
new – You are an expert!” – Jawed Karim (October 2006 at University of Illinois)

Quote Source: YouTube Video by Jawed Karim http://www.youtube.com/watch?v=nssfmTo7SZg

 - Questions (or) Comments?

Thank You for your Time! Questions & Comments are welcome!

Photo Credits: Martin E. Klimek (USA Today)