Multimedia

Digital Audio & Video

Outline

Audio / Video on the Web

Basic Digital Audio Concepts
Streaming Audio
Web Audio Formats

Basic Digital Video Concepts

Video compression/decompression methods.
Video File Formats

Other Multi-media Formats

Summary
CSC1720

All copyrights reserved by

AV on the Web?

It is part of the Internet.

Radio Stations
Music, sound clips
Streaming Audio / Video
Video conferencing
Digital Cameras
Animation on the Web

Shockwave, Flash

CSC1720

All copyrights reserved by

Basic Terms
Term

Definition

Bit

Binary value 0 or 1

Kilobit (Kb)

1,000 bits (approx.)

Byte

8 bits

Kilobyte (KB)

1,000 bytes

Megabyte
(MB)

1,000,000 bytes

Gigabyte
(GB)

1,000,000,000 bytes

Kbps

Kilobits per second (1,000 bits in a

second)

KB/sec

Kilobytes per second (1,000 bytes in a

second)
4
All copyrights reserved by

CSC1720

Analog vs. Digital

Analog: A signal that has a

continuous nature rather than a
pulsed or discrete nature.
Note: Electrical or physical analogies,
such as continuously varying voltages,
frequencies, or phases, may be used
as analog signals.

Digital: A signal in which discrete

steps are used to represent
information.
CSC1720

All copyrights reserved by

Digital Signal

Use Pulse
Code
Modulation
(PCM) to
represent an
audio signal
by digital
data.

CSC1720

All copyrights reserved by

ADC & DAC

Figure 4.3 Conversion from Analog to Digital requires an Analog-to-Digital Converter

Figure 4.4 Conversion from Digital to Analog requires a Digital-to-Analog Converter

CSC1720

All copyrights reserved by

Basic Digital Audio

Concepts

Sampling rate
Number of sample taken of a signal in a
given time (usually one second)

Bit depth
Describes the accuracy of the audio data

Channels
E.g. Stereo has two channels

Bit rate
Measured in kilobits per second (Kbps) is a
function of the bit depth and sampling rate
CSC1720

All copyrights reserved by

Sampling rate

The more sample taken per second, the

higher the accuracy.
Typically measured in kilohertz (KHz).
CD audio has 44,100 samples per second
(44.1KHz).
8 KHz produces lower quality radio sound.
Standard sampling rates include 8KHz,
11.025KHz,
The high-end 96K is used in DVD, but is
not applicable to the Web.
CSC1720

All copyrights reserved by

Sampling Rate

demo

CSC1720

10

All copyrights reserved by

Bit depth

Also called sampling resolution or word

length.
The more bits, the better is the quality of
the audio (and a larger file of course).
Common bit depths are 8-bit (telephone
like), 16-bit (CD quality), and 20, 24, 32, 48bit depths.
How many signal can a 8-bit and a 16-bit
data represent?
0000 0000 1111 1111
0000 0000 0000 0000 1111 1111 1111 1111

CSC1720

11

All copyrights reserved by

Quantization

demo

CSC1720

12

All copyrights reserved by

Channels

Audio file can support one to six

channels of audio formats.
Mono one channel
Stereo two channels
Some others three, four channels.
Six channels 5.1-channel surround
sound.
More multi-channel formats announce
in the coming years.
CSC1720

13

All copyrights reserved by

Channel Examples

CSC1720

14

All copyrights reserved by

Bit rate

Audio files are measured in terms of

bit rate which is measured in kilobits
per second (Kbps).
It can be calculated by dividing the
file size by the time (in second) to
play the audio clip.
E.g. 3Mb file play in 30 seconds
3000k / 30 = 100kbps.

Quality at different compression rates

CSC1720

15

All copyrights reserved by

Optimizing for the Web

Length of the audio clip

Keep the audio clip as short as
possible.
E.g. just keep the most sweetest part
of your greeting.

Number of channels
A mono audio file is halved the space
of a stereo file.
Depends on your actual needs.
CSC1720

16

All copyrights reserved by

Optimizing for the Web

Bit depth
Audio file on the Web are usually 8-bits.
Half the size of a 16-bit file.

Sampling rate
Half the sampling rate will also halve the
space needed.
Voice only audio file can be reduced to 8KHz.
22 KHz music clips are acceptable.

Putting all things together: Mono, 8-bit,

22KHz, MP3 compression.
CSC1720

17

All copyrights reserved by

Calculate audio size

8-bit mono: seconds x KHz

16-bit mono: (seconds x KHz) x 2
8-bit stereo: (seconds x KHz) x 2
16-bit stereo: ([seconds x KHz] x
2)x2
E.g. the file size of 30 seconds of
16-bit, 11KHz mono audio:
(30 x 11) x 2 = 660K.
CSC1720

18

All copyrights reserved by

Streaming Audio

What is it?
Play almost immediately after the request,
continues playing the transferring data.

Advantages:
Address the problem of long download time.
Control distribution and protect copyright,
because the user cannot get a copy of the file.

Disadvantages:
Sound quality may be be affected by low
speed or unstable Internet connection.

Reference
CSC1720

19

All copyrights reserved by

What is Streaming?

CSC1720

20

All copyrights reserved by

How does it work?

Streaming audio

Packet are sent to a buffer on the receivin

Computer, the RealPlayer will play the sou
File when buffer full

Web browser

Web browser request

a RealAudio from
the web server

RealAudio
Server

Web server
CSC1720

21

All copyrights reserved by

Quality Comparison

http://www.cit.cornell.edu/atc/itsupport/streamcompare.shtml

CSC1720

22

All copyrights reserved by

Web Audio Formats

WAV/AIFF (.wav/.aif/.aiff)
The Waveform Audio File format (.wav)
was developed by Microsoft, supports
arbitrary sampling rates and bit depths.
The Audio Interchange File format
(.aif, .aiff) was developed for Macintosh
platform.
They are less likely used on the Web,
because people use mp3 or
streaming.
CSC1720

23

All copyrights reserved by

http://www.nuance.com/prodserv/demo_vocalizer.h

WAV/AIFF

Try the Bell-lab synthesis link.

http://www.belllabs.com/project/tts/voices.html

Select the
Audio format

CSC1720

24

All copyrights reserved by

MP3 (.mp3)

Able to maintain excellent sound

quality at very small file sizes.
The compression reduces an audio
file to one-tenth of its original size.
E.g. 40MB file 3.5MB

MP3 is actually MPEG-1 Layer-III

Good for distribution of HQ audio.
Demo: www.mp3.com
CSC1720

25

All copyrights reserved by

What is MP3 digital

encoding?

MP3 is actually the file extension for MPEG,

audio layer 3. Layer 3 is one of three coding
schemes (layer 1, layer 2, and layer 3) for the
compression of audio signals. Layer 3 uses
perceptual audio coding and psychoacoustic
compression to remove all superfluous
information. (More specifically, it removes the
redundant and irrelevant parts of a sound
signal--the stuff the human ear doesn't hear
anyway). It also adds a MDCT (Modified
Discrete Cosine Transform) that implements a
filter bank, increasing the frequency resolution
18 times higher than that of layer 2.
CSC1720

26

All copyrights reserved by

MP3 Players

CSC1720

27

All copyrights reserved by

Apple QuickTime Audio

(.mov)

QuickTime is a well-known video

format, but it can create audio-only
movies.
QuickTime is a container format, which
is able to store still images, movie
formats,
Excellent compression, true streaming

Netscape and IE have Plug-in now.

Quicktime : demo
CSC1720

28

All copyrights reserved by

MIDI (.mid/.midi)

MIDI stands for Musical Instrument Digital

Interface which is developed for
electronic musical instruments.
MIDI files are very compact and very good
for low-bandwidth delivery.
Instruments are piano, drums, bass,
orchestral strings,
It is very attractive for adding MIDI file to
your website with very little download
time.
Demo: www.findmidis.com
CSC1720

29

All copyrights reserved by

http://www.findmidis.com/

MIDI

CSC1720

30

All copyrights reserved by

RealMedia/RealAudio
(.rm/.ra)

RealAudio is a server-based streaming

audio format.
The RealServer responses to the requests
and delivers the streaming packets,
including the bandwidth negotiation.
A robust RealServer can support
thousands of simultaneous listeners.
Good for continuous-playing audio and
live broadcasts to a large group of people.
Example: RTHK Radio
CSC1720

31

All copyrights reserved by

RealMedia/RealAudio

CSC1720

32

All copyrights reserved by

Windows Media
(.wma/.asf)

Windows Media is a streaming

system.
It wraps all media elements into a
Active Streaming File (.asf).
Audio may be saved as non-streaming
Windows Media Audio format (.wma).
Good for continuous-play audio.
The encoder and player is Free,
shipped with Windows OS.
CSC1720

33

All copyrights reserved by

Windows Media

CSC1720

34

All copyrights reserved by

Choosing an audio
format
Audio Needs

Suggested formats

Short voice greetings

WAV, AIFF, MP3

News broadcasts

Streaming solutions
(RealAudio, Windows Media)

Background music

MIDI, WAV

Music samples for some

audience

MP3 or QuickTime

Radio-style or Live
broadcasting

RealMedia System

Musical E-greeting card

MIDI, WAV

CSC1720

35

All copyrights reserved by

Add Audio to your Web

Page

A simple link
<A HREF=audio/song.wav>Play the song </A>
<A HREF=music.mp3><IMG
SRC=buttons/play.gif></A>

Background Sound
<BGSOUND SRC=audio/song.mid
LOOP=3>

Link to RealMedia
<A HREF=song.ram>Link to the song</A>
pnm://domainname.com/song.rm
CSC1720

36

All copyrights reserved by

Break Time 15
minutes

CSC1720

37

All copyrights reserved by

Basic Digital Video

Concepts

Movie length
Frame size
Frame rate
Quality
Color bit depth
Data rate (bit rate)

CSC1720

38

All copyrights reserved by

Movie length

File size is proportional to the

movie length.
Videos longer than 1 or 2 minutes
cause long download times.
If it is a long video, consider to
use streaming video.

CSC1720

39

All copyrights reserved by

Frame size

Full-screen video is 640x480 pixels.

The most common frame size for
web video is 160x120 pixels.
Not recommend to use a frame size
larger than 320x240.
The size depends on the CPU power
and the Internet connection
bandwidth.
CSC1720

40

All copyrights reserved by

Image and Video?

CSC1720

41

All copyrights reserved by

Frame rate

Frame rate is measured in number of

frames per second (fps).
Standard TV-quality video uses 30
fps.
For the web, 15 or even 10 fps is
more appropriate and produces fair
smooth quality for the user.
Commercial Internet Broadcasts are
using 0.5, 0.25 frames per second.
CSC1720

42

All copyrights reserved by

Quality

Many video-editors allow you to

set the overall quality of the video.
The degree of compression
controls the target quality.
The low or medium setting results
a fairly high compression which is
appropriate for web delivery.
Frame rate and quality are usually
tradeoff in different applications.
CSC1720

43

All copyrights reserved by

Color bit depth

The number of pixel colors in each

frame affects the size of the video.
The file size of the video will be
greatly reduced by changing the
number of colors from 24-bit to 8bit.
It sacrifices the image quality of
the video.
CSC1720

44

All copyrights reserved by

Data rate (bit rate)

This is the rate that the data must be

transferred in order to ensure the video can
play smoothly without interruption.
It is measured in kilobytes per second (K/sec
or Kbps).
It can be calculated by dividing the size of
the file (in K) by the movie length (in
seconds).

E.g. the video file size is 1.9MB 1900K

Play 40 seconds long, Data rate = 47.5K/sec

Consider the Internet bandwidth!

CSC1720

45

All copyrights reserved by

Calculate space
requirements of Video

NTSC video (640 x 480 and 29.97

fps)
Frame size = ([Pixel width x pixel height
x bit depth]/8)/1024
E.g. 200KB/Frame : 6.0 MB/sec
200KB x 30 fps = 6000KB/s, 6 MB/sec

PAL video (768 x 576 and 25 fps)

E.g. 200KB/Frame : 5.0 MB/sec
200KB x 25 fps = 5000KB/s, 5 MB/sec
CSC1720

46

All copyrights reserved by

Video CODECs

CODEC is
Compression/Decompression
algorithms.
The sound and frame images of a
digital video must be compressed.
The vast amount of data

Compressed in a number of ways

Lossless and Lossy compression

Spatial and Temporal compression

CSC1720

47

All copyrights reserved by

Video Clip Demo

CSC1720

48

reference

All copyrights reserved by

Lossless and Lossy

compression

Lossless compression means no

information is lost and the final file is
the same as the original.
Most compression methods are lossy.
Sacrifices some data from the file in
order to achieve higher compression
rates.
Use complicated algorithm to toss out
some data that is not discernible to the
human eye or ear.
CSC1720

49

All copyrights reserved by

Spatial and Temporal

compression

Spatial (Intraframe) compression takes

place on each individual frame of the
video.
Temporal (Interframe) compression
applies on a series of frames, it takes
advantages of areas of the image remain
unchanged from frame to frame.
Relies on the key frames and delta frames.
A key frame is placed once every second.
E.g. 15 fps, a key frame once 15 frames.
CSC1720

50

All copyrights reserved by

Delta frame

CSC1720

51

All copyrights reserved by

Video File Formats

QuickTime Movie (.mov)

Introduced by Apple Computer in 1991.
First developed for Macintosh, now also
supports the PCs.
Also supports streaming.

How to create?
Most video editor, QuickTime Pro.

How to play?
QuickTime plug-in or QuickTime player.
CSC1720

52

All copyrights reserved by

http://www.apple.com/quicktime/gallery/cubicvr/times_square

Quicktime

CSC1720

53

All copyrights reserved by

RealMedia (.rm)

Industry standard streaming format.

RealPlayer for playback.
RealServer for serving streams.
RealProducer for creating .rm files.
Good for
Long-playing video or broadcast to many people.

How to create?
RealSystem Producer

How to play?
RealPlayer (Free), RealPlayer Plus (Commerical)

CSC1720

54

All copyrights reserved by

Windows Media
(.wmv/ .asf)

Created by Microsoft, closely integrated

with Windows OS.
Support Windows Media Video (.wmv) and
Advanced Streaming Format (.asf) and
other formats (.avi, .mpeg, )
Also support streaming.
How to create?
Windows Media Encoder, Windows Media
Author

How to play?
Media Player in Windows OS
CSC1720

55

All copyrights reserved by

AVI (.avi)

Stands for Audio/Video Interleaved.

Introduced by Microsoft in 1992.
In a AVI file, the audio and video
information are interleaved every frame.
Good for
Short web clips, high-quality video

How to create?
Most video editing tools.

How to play?
Windows Media, QuickTime, etc.
CSC1720

56

All copyrights reserved by

Video Clip Demo

CSC1720

57

reference

All copyrights reserved by

MPEG (.mpg/ .mpeg)

Created by Moving Picture Experts Group

(MPEG).
Supports 1) Video, 2) Audio, 3) Streaming.
Extremely high compression rates with
small quality degradation (lossy).
MPEG-1 : VHS quality
MPEG-2 : HQ standard for TV broadcast
MPEG-4 : Very HQ for AV compression
MPEG can be compressed by using three
schemes: Layer-I, Layer-II, Layer-III.
CSC1720

58

All copyrights reserved by

Official MPEG page reference

CSC1720

59

All copyrights reserved by

MPEG Comparison
MPEG-1

MPEG-2

MPEG-4

Standard since

1992

1995

1999

Default Video resolution

(NTSC)

352 x 288

640 x 480

640 x 480

Max. Audio Frequency range

48 KHz

96 KHz

96 KHz

Max. audio Channel

Regular data rate

1380 kbit/s

6500 kbit/s

880 kbit/s

Frames per sec (NTSC)

30

30

30

Video Quality

Satisfactory

Very good

Very Good

Hardware requirement for

encoding/decoding

Low

Medium

High

CSC1720

60

All copyrights reserved by

Flash & Shockwave

Advantages:

File sizes are small

Image quality is high
It uses streaming technology
It uses high-quality streaming audio
It is scriptable

Disadvantages:
A plug-in player is required
Expensive authoring software
Problems on printing their content
CSC1720

61

All copyrights reserved by

Summary

Discuss the basic digital audio/video

terminology.
Introduction to different formats: WAV,
MP3, QuickTime, RealMedia, Windows
Media, AVI, MPEG.
To deliver long-playing audio/video or
live broadcasts, you should choose one
of the streaming media.
Flash and shockwave are popular and
appropriate format for the Web.
CSC1720

62

All copyrights reserved by

References

Design Web Audio (J. Beggs, D. Thede), Oreilly.

E-Video (H. Peter Alesso), Addison-Wesley.
Audio Video Knowledge Center
Bible in MP3 format
MPEG
Musiq
Luke Video
The End.
Thank you for your patience!
CSC1720

63

All copyrights reserved by