Вы находитесь на странице: 1из 85

RTP: Multimedia Streaming over IP

Colin Perkins
USC Information Sciences Institute
Internet Multimedia

• Internet Multimedia has long history:


– RFC 741, "Network Voice Protocol", 1977
– First video experiments in the early 1980s

• Modern standards development began in 1992:


– Developing from teleconferencing systems
– Audiocast of IETF meetings
• 20 sites on 3 continents
• Precursors to RTP and the present standards
– Standardized RTP in 1996

• Widespread availability of suitable networks in the last


couple of years

• Unusual characteristics, worth exploring


Copyright © 2002 Colin Perkins
Talk Outline

• Use of IP for real-time traffic


• Protocols for real-time multimedia over IP
– Outline of signalling protocols
– Outline of media transport protocols
• RTP: Real-time Transport Protocol
– RTP data transfer protocol
– RTP control protocol
• Robustness
– Playout and timing correction
– Error correction
– Security
– Congestion control
• Security
• Conclusions

Copyright © 2002 Colin Perkins


Talk Outline

• Use of IP for real-time traffic


• Protocols for real-time multimedia over IP
– Outline of signalling protocols
– Outline of media transport protocols
• RTP: Real-time Transport Protocol
– RTP data transfer protocol
– RTP control protocol
• Robustness
– Playout and timing correction
– Error correction
– Security
– Congestion control
• Security
• Conclusions

Copyright © 2002 Colin Perkins


The IP Protocol Stack

Application programs • Applications can't see the link


HTML MIME Media codecs layers
HTTP SMTP RTP – Just see the performance of
FTP SIP RTSP
the IP layer
TCP UDP – Must assume lowest common
IP denominator behaviour
ADSL • Link layer can't tell the needs
Ethernet PPP of the application
Twisted Pair – Just see a series of packets
Optical Fibre Wireless
– Optimisations for particular
traffic classes are risky
• IP forms an abstraction layer
– Is the traffic really what you
– Applications and transport think?
protocols above
• Decoupling applications from
– Assorted link technologies
the network
below

Copyright © 2002 Colin Perkins


IP Service Model

The IP service model is limited


• Best effort packet transport
• Fragmentation
• Routing and addressing

Copyright © 2002 Colin Perkins


Best Effort Packet Transport

• Performance not guaranteed • Many causes of problems:


• Packets can be… – Congestion may cause loss
– lost – Packet corruption may cause loss
– delayed – Route changeover may cause loss
– reordered – Queuing delay
– duplicated – Multi-path IP routing may reorder
– corrupted – Link-layer striping may reorder
…and the transport protocol – Spurious retransmission and
must compensate router bugs cause duplicates

• Checksum to catch bit errors

Assumption: Significant packet loss

Copyright © 2002 Colin Perkins


Packet Loss Patterns

45

40
Packet Loss Rate (percent)

35

30

25

20

15

10

0
8:00 10:00 12:00 14:00 16:00 18:00

Time of day

Copyright © 2002 Colin Perkins


Timing Disruption

45º line: clocks


are synchronized

Discontinuity due
to route change.
Reception time

Queuing jitter causes


variation in inter-packet
arrival time.

Non-45º slope shows


presence of clock skew

Transmission time

Copyright © 2002 Colin Perkins


Best Effort Transport

• Performance can be bad


– Applications should be prepared to compensate
– Doesn't have to be!

• Loss and jitter can be made arbitrarily low through


careful engineering
– Most backbone networks have very good performance
– Interconnects and customer LANs are currently the main
trouble spots
– Explicit QoS doesn't appear necessary

Copyright © 2002 Colin Perkins


Fragmentation

• IP fragments packets that exceed the MTU


– Often >1500 octets
• This causes an undesirable loss multiplier effect
– Loss of one fragment means the others must also be
discarded
– Better for the application to fragment, and generate small
IP packets

Original

Fragments

Packet loss

Reconstructed

Copyright © 2002 Colin Perkins


Routing and Addressing

• Most communication is unicast • The use of IP with best


– Point-to-point effort transport implies
• Inefficient for many receivers heterogeneity
– The server must generate n • Multicast makes this an
copies of the data, for n clients order of magnitude worse
– Significant scaling bottleneck – Each receiver sees different
• Multicast allows the server to loss characteristics
scale – Hard to get timely feedback
in a scalable manner
– Adds network complexity

Copyright © 2002 Colin Perkins


Properties of IP: Summary

• Best effort, unreliable, packet delivery service


– Much of the network performs well
– Peering points and customer premises often lacking
• Provides a fragmentation service
– Typically best avoided
• Provides a multicast service
– In some parts of the network
• 10% of broadband connections

• At best: simple, effective, service

Copyright © 2002 Colin Perkins


Transport Protocols

• The IP service, by itself, is very limited


– Just (tries to) deliver packets

• Always augmented by a transport protocol


– TCP
– UDP
– (others in development)

Copyright © 2002 Colin Perkins


TCP

• Enhances the raw IP service


– Point-to-point and connection oriented
– Service selection through ports
– Reliable, in-order, delivery
– Rate adaptation to match network capacity

Copyright © 2002 Colin Perkins


TCP

• Rate adaptation matches


throughput to capacity
– High link utilization

Congestion Window Size


– Fair share between flows

• Retransmission ensures
that no data is lost
– Reliable, in-order, delivery
Time
Slow start Slow start Congestion Congestion
avoidance avoidance

Copyright © 2002 Colin Perkins


UDP

• Uses the services of IP


– best effort (unreliable) but timely
– fragmentation
– routing and addressing
– unicast and multicast
• Provides ports, in addition to IP addressing, but no
other services

Copyright © 2002 Colin Perkins


Reliability/Timeliness Tradeoff

Reliable Unreliable

Not timely Timely


TCP UDP
RTP

• Protocols built on unreliable • Multimedia systems choose


packet networks must make their transport carefully:
a fundamental trade-off: – TCP for signalling
– Timely – UDP for media data
– Reliable • Application level protocols
• TCP is at one extreme, UDP can blur the boundary
the other – E.g. RTP for multimedia data
– Essential for performance
Copyright © 2002 Colin Perkins
Talk Outline

• Use of IP for real-time traffic


• Protocols for real-time multimedia over IP
– Outline of signalling protocols
– Outline of media transport protocols
• RTP: Real-time Transport Protocol
– RTP data transfer protocol
– RTP control protocol
• Robustness
– Playout and timing correction
– Error correction
– Security
– Congestion control
• Security
• Conclusions

Copyright © 2002 Colin Perkins


The Multimedia Protocol Framework

Two fundamental components of the framework:


• Signalling protocols
• Media transport protocols

Copyright © 2002 Colin Perkins


Signalling Protocols

The first stage of any multimedia session is signalling


• User location
• Session initiation, call setup and teardown
• Media negotiation
• Conference control

Many signalling protocols exist


• Teleconferencing: H.323
• Telephony: SIP
• Streaming: RTSP, SAP

Copyright © 2002 Colin Perkins


Signalling Protocols: H.323

• Original signalling protocol for IP-based multimedia


– Extension of H.320 ISDN conferencing to IP
– Tightly coupled, small group, video conferencing

• Flexible media negotiation and call control


• Reputation for complexity
– ASN.1 encoding
– Many RTT call setup
• (mostly fixed in later versions)

Copyright © 2002 Colin Perkins


Signalling Protocols: SIP

• RFC 2543 (recently updated)

• SIP is used to invite someone to join a session


– Media negotiation
– User location
– Call setup and teardown

• Considerable overlap with H.323


– More flexible integration with other Internet services
• Email, Web, streaming media, recording, agents, etc.
– More limited media negotiation and call control
• Extensions underway
– Very different style
• Protocol operation is based on HTTP
• Reuses much existing infrastructure

Copyright © 2002 Colin Perkins


Signalling Protocols: RTSP

• RFC 2326

• Designed for control of a media on demand server


– Point-to-point VCR-style remote control
– Record/play/rewind/fast-forward
• Widespread commercial use…
– RealAudio, QuickTime

• May also be useful for controlling other devices


– Voice mail
– Interactive voice response

• Leverages HTTP and SIP infrastructure

Copyright © 2002 Colin Perkins


Signalling Protocols: SAP

• RFC 2974

• A multicast announce-listen protocol for wide area


announcement of multimedia sessions
– Announcers periodically multicast SDP descriptions to a
well known group
– Inter-announcement interval is 10+ minutes
– Listeners slowly build up a cache of sessions
• Suitable for announcing long-lived public sessions
– E.g. radio/TV station, event coverage

• Mostly used with IP multicast


– Talk of use with cable networks

Copyright © 2002 Colin Perkins


Media Transport Protocols

Once the session has been setup, media flows

Convergence on a single media transport protocol for:


• Voice over IP
• Teleconferencing
• Streaming media

Real-time Transport Protocol, RTP


• Flexible, supports many codecs and media types
• Extensible to new scenarios

Copyright © 2002 Colin Perkins


The Multimedia Protocol Framework

Call control Light weight Media Media Call control


RAS
Media negotiation sessions codecs codecs Media negotiation

RTSP SIP SAP RTP RTP H225.0 H.245

TCP UDP UDP TCP


IP IP
IETF Multimedia Protocol Stack ITU Teleconferencing Protocols

Copyright © 2002 Colin Perkins


Design Choices

• Depending on the scenario, implement:


– An appropriate signalling protocol
• RTSP
• SIP/H.323
• SAP
– Media transport using RTP
• One or more codecs
– MPEG
– H.263
• Error correction and concealment
• Congestion Control

Copyright © 2002 Colin Perkins


Talk Outline

• Use of IP for real-time traffic


• Protocols for real-time multimedia over IP
– Outline of signalling protocols
– Outline of media transport protocols
• RTP: Real-time Transport Protocol
– RTP data transfer protocol
– RTP control protocol
• Robustness
– Playout and timing correction
– Error correction
– Congestion control
• Security
• Conclusions

Copyright © 2002 Colin Perkins


RTP: Real-time Transport Protocol

• The standard for real-time transport over IP networks


– Streaming audio and video
– Voice over IP

• Published as an IETF proposed standard


– RFCs 1889 and 1890 in January 1986
– Adopted by ITU as part of H.323
– Adopted by 3GPP for next generation cellular telephony
– Widespread use in streaming: QuickTime, Real, Microsoft
• (HTTP streaming still common)

• Recently revised for draft standard status


– Work complete, awaiting publication
– Changes include:
• Clarifications and bug-fixes based on experience
• Scalability improvements
• Support for new codecs
– 100% backwards compatible
Copyright © 2002 Colin Perkins
Philosophy of RTP

• The challenge:
– build a mechanism for robust, real-time media delivery
above an unreliable and unpredictable transport layer
– without changing the transport layer

Push responsibility for media Make the system robust to


delivery onto the end-points network problems; media
where possible data should be loss tolerant

The end-to-end argument Application level framing

Copyright © 2002 Colin Perkins


The End-to-end Argument

• Two options for ensuring reliability


– Pass responsibility hop-by-hop, along with the data
• Email
– Responsibility remains with the end points, which ensure
delivery even if the intermediate steps are unreliable
• Both TCP and RTP take the second approach

• Consequences:
– Intelligence tends to "bubble-up" the protocol stack to the
end points
– The intermediate systems can be simple, and need not be
robust
• They can simply discard data they cannot deliver, since it will
be recovered end-to-end

• The network is dumb, but end-points are smart


Copyright © 2002 Colin Perkins
Application Level Framing

• Only the application has sufficient knowledge of its data


to make an informed decision about how that data should
be transported
• Implications:
– The transport protocol should accept data in application
meaningful chunks ("ADUs")
• The application must understand the data,
• The application must be able to process ADUs independently, in
arbitrary order, and in the presence of loss
– The transport protocol should expose details of delivery,
allowing the applications to react intelligently if there are
problems
• Blind retransmission is not always appropriate
• Maybe the data is stable, and an updated version can be sent
• Maybe the data is obsolete, and doesn't need to be resent
• Maybe an alternate representation of the data can be sent

Copyright © 2002 Colin Perkins


Philosophy of RTP

• The philosophy of RTP implies smart, network-aware,


applications that are capable of reacting to problems
end-to-end.
– Both sender and receiver are intelligent
– The network is dumb and can be unreliable
• Similar principles apply to the signalling protocols
– Mostly end-to-end operation, limited support for network
state

• Fits well with the IP service


• Contrast with traditional applications:
– Telephone network is smart, end-points are dumb
– MPEG sender is smart, receiver relatively dumb

Copyright © 2002 Colin Perkins


Protocol Components

Payload
Payload
Payload
Payload
Format RTP Profile
Format
Format
Format

RTP Data Transfer Protocol RTP Control Protocol

UDP

IP

Copyright © 2002 Colin Perkins


Protocol Components

RTP Data Transfer Protocol

• Transports application data • Provides:


units – Source identification
– Audio – Payload identification
– Video – Media sequencing
• One RTP stream transports – Timing recovery
each media type • Extensions allow for error
correction
Copyright © 2002 Colin Perkins
Protocol Components

RTP Control Protocol

• Reception quality feedback • Mapping from media clock to


– Packet loss fraction external time-base
– Average timing jitter – E.g. for lip synchronization
• Optional source description • Loosely coupled membership
– Name, location, email management
address, phone number

Copyright © 2002 Colin Perkins


Protocol Components

Payload
Payload
Payload
Payload
Format
Format
Format
Format

• Provide the adaptation layer • Many payload formats exist,


between a particular codec with more being developed:
and RTP – H.261, H.263, M-JPEG,
• Optimised for robustness to MPEG-2, MPEG-4, BT.656,
packet loss SMPTE-292

Copyright © 2002 Colin Perkins


Protocol Components

RTP Profile

• Define use of RTP in particular • Provides a namespace


application scenarios for payload formats
– "Reasonable defaults"
– Adaptation to unusual conditions
• Single source multicast
• Operation without back channel
• Authenticated and secure operation
Copyright © 2002 Colin Perkins
Combining the Pieces

• A multimedia session comprises several RTP sessions


– One for each media type
• Each RTP session:
– Implements a particular RTP profile
– Includes an RTP data flow
• Transporting a single media type according to one or more
payload formats
– E.g. Audio switching between G.729 and Fax
– E.g. Video using MPEG
– Includes an RTP control protocol flow
• Providing reception quality feedback, user information, etc.
– Is defined by:
• Source and destination IP addresses
• A pair of UDP ports: one for RTP, one for RTCP

Copyright © 2002 Colin Perkins


RTP Data Transfer Protocol

• The RTP Data Transfer RTP provides:


Protocol delivers a single • Source identification
media stream from sender • Media identification
to one, or more, receivers
• Media transport
– Few assumptions about the
underlying transport – Padding, if necessary
– Usually runs over UDP/IP – Marking of significant
events
• Sequencing
• Typically implemented in an
application or as a library • Timing recovery
– User level, not part of the
kernel

Copyright © 2002 Colin Perkins


Packet Format

V PX CC M PT Sequence Number
Timestamp
Synchronization source (SSRC) identifier

Contributing source (CSRC) identifiers

Payload data

Padding

Copyright © 2002 Colin Perkins


Source Identification

• Each packet carries a 32 bit V P X CC M PT Sequence Number

synchronization source Timestamp

– Randomly chosen at start- Synchronization source (SSRC) identifier

up, with collision detection Contributing source (CSRC) identifiers

• Provides a transport layer


independent identifier Payload data
– Supports gateways
Padding
– IPv4, IPv6, ATM
• Identifies a single time-
synchronized media flow
– Mapped to a persistent
identifier using RTCP

Copyright © 2002 Colin Perkins


Source Identification

• Each packet may include a V P X CC M PT Sequence Number

list of contributing sources Timestamp

– Allows data from up to 16 Synchronization source (SSRC) identifier

sources to be identified Contributing source (CSRC) identifiers

– Each CSRC is the SSRC of a


mixed participant
Payload data

Padding
• Allows RTP to support mixers
and translators
– Mixers combine several flows
into one
• E.g. Conferencing MCU
– Translators change the
format of a flow, or gateway
between different networks
• Transcode to a lower bit-rate
• Gateway between unicast
and multicast

Copyright © 2002 Colin Perkins


Communication Models

• Mixers and translators greatly expand RTP end point

the range of communications models RTP Translator or mixer

available to RTP Multicast group, the


network replicates
data as necessary, with
no translation or mixing.

Point-to-point
communication
via unicast

Four participants
communicating via
a multicast group

Translated: multicast to
unicast. Two participants
communicating via a
multicast group, with a
Replicated unicast: a third linked to the session
group of three using an by an RTP translator.
RTP translator/mixer to
mediate communications.
Copyright © 2002 Colin Perkins
Media Identification

• Each packet carries a 7 bit V P X CC M PT Sequence Number

payload type field Timestamp

• Mapped to a payload format Synchronization source (SSRC) identifier

during session setup Contributing source (CSRC) identifiers

– Allows flexible signalling of


codec type and parameters Payload data
– Mapping can be static, if the
profile allows Padding

• Each flow carries only one


type of media
– Only audio, or only video
• The payload type allows the
sender to switch between a
set of payload formats
– E.g. a flow carries only audio
but may switch between fax
and voice at any time

Copyright © 2002 Colin Perkins


Media Transport and Payload Formats

• Packets contain a block of V P X CC M PT Sequence Number

payload data, described by Timestamp

a payload format Synchronization source (SSRC) identifier

• Payload formats describe the Contributing source (CSRC) identifiers

mapping between codec output


and RTP packets Payload data

– Chosen so that each packet is


Padding
independently decodable
– Application level framing
• The payload data typically
includes a payload header to
ease parsing
– E.g. The H.261 payload format
copies some information from
the GOB header so each set of
macro-blocks can be decoded
independently

Copyright © 2002 Colin Perkins


Media Transport: Marker

• Each packet includes a bit to V P X CC M PT Sequence Number

mark significant events Timestamp

– Start of talk spurt for audio Synchronization source (SSRC) identifier

– Last packet of frame for video Contributing source (CSRC) identifiers

• A hint that special processing


may be required Payload data

Padding

Copyright © 2002 Colin Perkins


Media Transport: Padding

• Each packet may be padded V P X CC M PT Sequence Number

beyond its natural size Timestamp

• Rarely used, but needed by Synchronization source (SSRC) identifier

some encryption algorithms Contributing source (CSRC) identifiers

– DES in CBC modes operates


on 64 bit blocks Payload data

Padding

• The SRTP profile provides a


better security solution

Copyright © 2002 Colin Perkins


Sequencing

• Each packet contains a 16 V P X CC M PT Sequence Number

bit sequence number Timestamp

– Random initial value Synchronization source (SSRC) identifier

– Increments monotonically Contributing source (CSRC) identifiers

with each packet sent


– Wraps around to zero when
Payload data
the limit is reached
• Used to detect packet loss Padding

– Is not used to determine


playout order

• Basic RTP does not provide


error correction
– The receiver is expected to
conceal the error, and to
continue processing
– Extensions provide forward
error correction and limited
retransmission
Copyright © 2002 Colin Perkins
Timing Recovery

• Each packet contains a 32 V P X CC M PT Sequence Number


bit timestamp Timestamp

• Indicates the sampling Synchronization source (SSRC) identifier

instant of the oldest Contributing source (CSRC) identifiers


payload data
– Determines playout order
Payload data

• The clock rate is defined by Padding

the payload format:


– Audio clock is sampling rate • Time code not carried
– Video clock is 90kHz, directly, but mapping to
indicating the frame time
wall clock time via RTCP
– Mapping to codec time-base
is also defined sender reports

• No requirements on stability
or accuracy of clock
– Implies receiver adaptation

Copyright © 2002 Colin Perkins


RTP Control Protocol (RTCP)

• Each RTP data flow has an associated control flow


• The control flow provides:
– Time-base management
– Quality of service feedback
– Member identification and management

Copyright © 2002 Colin Perkins


Time-base Management

• Timestamps map between the RTP timeline and NTP


“wall-clock” time
– If a common NTP clock is used for multiple streams, a
receiver can synchronize them

• No explicit transport of SMPTE (or similar) time-codes


– Can be derived from NTP timestamps
• Accuracy limited by NTP resolution, unless external clock
provided
• RTSP provides a mapping function

• Also allows receivers to estimate data/packet rate and


possibly clock skew

Copyright © 2002 Colin Perkins


Reception Quality Reporting

• Quality of service feedback from each receiver:


– Loss fraction
– Cumulative number of packets lost
– Highest sequence number received
– Inter-arrival jitter
– Round-trip time

• Many uses:
– Loss rate can be used to select amount of FEC to employ
– Jitter gives estimate of playout buffer delay at receiver

Copyright © 2002 Colin Perkins


Membership Management

• RTCP provides a canonical • Augments the membership


name, mapping SSRC to a management provided by
persistent identifier the signalling protocol
– Used to associate streams – Primarily using the explicit
for synchronisation leave indication

• RTCP can optionally deliver


source description data:
– Name
– Email address
– Phone number
– Location
– (extend with metadata)

• Provides loosely coupled


presence information
– Explicit leave message

Copyright © 2002 Colin Perkins


RTCP reporting interval

• RTCP is a low-rate reporting protocol


– Not intended for uses that require instant feedback
– Scalable to very large sessions
• Statistical summary of group conditions

• Packets are sent periodically


– The interval between packets is adjusted to limit RTCP to
once per 5 seconds, or 5% of the data rate
– Randomized to avoid synchronization

Copyright © 2002 Colin Perkins


RTP: Summary

• Flexible and extensible media transfer protocol


– Supports a range of codecs
– Allows detection of network problems
– Allows recovery of media timing

• Associated, low rate, reporting of reception quality,


time-base, and presence information

Copyright © 2002 Colin Perkins


Talk Outline

• Use of IP for real-time traffic


• Protocols for real-time multimedia over IP
– Outline of signalling protocols
– Outline of media transport protocols
• RTP: Real-time Transport Protocol
– RTP data transfer protocol
– RTP control protocol
• Robustness
– Playout and timing correction
– Error correction
– Congestion control
• Security
• Conclusions

Copyright © 2002 Colin Perkins


Robustness

• RTP operates over UDP/IP


– Best effort delivery
– Packets can be lost, delayed, reordered, duplicated, etc.

• Applications are responsible for correct playout


– Timing recovery
– Error concealment
– Congestion control

Copyright © 2002 Colin Perkins


Timing Recovery

• The network can seriously


disrupt media timing
• Receivers must include a
jitter compensation buffer
to reconstruct the media for
playout
Original media stream

Sender Router

Constant inter-packet spacing

Internet

Network induces
timing jitter into
the media stream

Received media stream

Router Receiver

Variable inter-packet spacing

Copyright © 2002 Colin Perkins


Playout and Timing Correction

1st talk spurt 2nd talk spurt

Transmission
Jitter affects inter-packet timing
Network
Transit

Reception

Playout
Buffer
Network transit delay

Playout

Packet discarded
due to late arrival
Playout buffering delay added
to compensate for jitter.

Copyright © 2002 Colin Perkins


Playout and Timing Correction

• RTP does not specify a • Many trade-offs to consider:


standard playout buffer – latency versus quality
or timing reconstruction – speed of reaction to change
algorithm – buffering ability
– Provides the necessary
information; allowing
product differentiation • Typical design:
– Streaming applications use
large delay (10+ seconds)
• Compare with MPEG, where – Interactive applications try
the buffer model is closely to keep delay low (tens of
defined milliseconds)

Copyright © 2002 Colin Perkins


Error Correction

• Limited Retransmission • Unequal error protection


– RTCP feedback profile • Interleaving
• Forward Error Correction
– Media specific
– Media independent

Reliable Unreliable

Not timely Timely


TCP UDP
RTP

⇒ Add limited reliability to RTP

Copyright © 2002 Colin Perkins


Retransmission in RTP

• Why must retransmission be limited?


– Timely versus reliable
– We don't want to re-invent TCP

• How to implement retransmission?


– RTCP provides a back channel
– Modify the RTCP timing rules to allow early feedback
• Keep the fundamental scaling rules to avoid potential for
implosion

Copyright © 2002 Colin Perkins


RTCP Feedback Profile

• RTCP reports sent as usual • Transport layer feedback


• Feedback can be sent early – NACK, ACK sequence number
– Ignore 5 second rule • Payload specific feedback
– Borrow bandwidth from the – Reference picture selection
next reporting interval – MPEG-4 NEWPRED
– Delays next report
– Send minimal report packet
• Under development in IETF

Immediate Early RTCP Regular


FB mode mode RTCP mode

Group
Report every Report many Just regular size
2 relevant event of the events RTCP packets
immediately but not all

Send feedback + regular RTCP packets


Copyright © 2002 Colin Perkins
Forward Error Correction

• Retransmission relies on • Alternative:


feedback from receivers – Sender adds redundant data
– Must request that lost to the media stream
packets are resent – Receivers use this to correct
• Works well in many cases errors, without contacting
sender
• Two scenarios where it is
inefficient: • Forward Error Correction
– Round-trip time is large – Well known technique at the
link layer
– Many receivers, independent
loss events – Also be applicable at the IP
level and above

RTP RTP RTP FEC RTP RTP RTP FEC RTP

Copyright © 2002 Colin Perkins


Media Specific FEC

• Some codecs may naturally


be loss tolerant
• Design payload format to
take advantage of this

Copyright © 2002 Colin Perkins


Media Specific FEC

• Some codecs may naturally


be loss tolerant
• Design payload format to
take advantage of this
– Audio/video redundancy
– RFC 2198, AMR, H.263+

1 2 3 4 Original Stream

1 2 3 4 Redundant data added

1 2 4 Packet lost in transport

1 2 3 4 Reconstructed stream

Copyright © 2002 Colin Perkins


Media Independent FEC

• Media specific FEC needs to • Better if the FEC can be derived


be produced by the encoder from pre-compressed media
– Part of the payload format – Less load on sender
– Either off-line or on-line – Perhaps less network efficient
⇒ Compression performed at
time of transmission
⇒ To pick appropriate FEC
⇒ Undesirable
⇒ Too much load on sender to
support many streams

Compress Packetize

Source image RTP RTP RTP RTP FEC

FEC

Copyright © 2002 Colin Perkins


Parity FEC

• Parity codes to protect serial • Standard for parity FEC:


communication well known – RFC 2733
• Can apply same technique to – Flexible parity operation
packet networks • Standards for Reed-Solomon
– Generate parity packet coding under development

Bit stream A 1 1 1 0 0 0 1 1 1 1 0 0 0 1

Calculate parity to recover


0 0 1 1 0 1 0
B = A XOR (A XOR B)

Parity code: A XOR B 1 1 0 1 0 1 1 1 1 0 1 0 1 1

Bit stream B 0 0 1 1 0 1 0 Transmission loses B


Copyright © 2002 Colin Perkins
Unequal Error Protection

• Not all data in the packets • Some links have high bit
is equally important error rate
– Headers and codec state – Causes packet corruption
updates are vital – Detected by UDP checksum
– Media data is of variable – Packet discarded
importance
• Seriously impacts wireless
• Data used for predication
• Data used in a single frame
link performance
– Cellular, especially

Partial Checksum
at the UDP level

Copyright © 2002 Colin Perkins


Interleaving

• Packet loss concealment • Can interleave, to make


and correction work best bursts of loss appear as
when loss is isolated random loss
– Single packet losses – Adds considerable delay
• Packet loss on the Internet • Popular with streaming apps
is bursty – Part of many audio payload
formats

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Original stream

1 5 9 13 2 6 10 14 3 7 11 15 4 8 12 16 Interleaved stream

1 5 9 13 2 6 10 14 4 8 12 16 Packet loss

1 2 4 5 6 8 9 10 12 13 14 16 Reconstructed stream

Copyright © 2002 Colin Perkins


Error Correction: Summary

• Extensive, and ongoing, ⇒ Network doesn't have to be


research and standards perfect
work developing error • If it's okay for data traffic,
correction for RTP it's probably okay for video
– Retransmission • Unless you have very strict
– Media/codec specific FEC requirements
– Media independent FEC
– Partial checksum ⇒ Acceptable to overprovision
– Interleaving for QoS
• No real need for RSVP or
differentiated services

⇒ Well designed applications


can tolerate significant loss
• Often, 5% loss acceptable

Copyright © 2002 Colin Perkins


Congestion Control

But… … the preceding assumed


traffic is well behaved
… assumed flows respond to
congestion in the network

… or, QoS and flow admission


control employed

Copyright © 2002 Colin Perkins


Congestion Control

• An IP network provides a Normal operation Congestion Collapse


best-effort packet-switched

Packets delivered
service.
– No admission control
– The network accepts all
packets and tries to deliver
them.
Packets sent
• However, no guarantee of
delivery provided
– Excess packets discarded if
links become congested.

• The transport protocol must


detect loss, and reduce its
rate to allow the congestion
to clear
– TCP does this automatically
– RTP does not

Copyright © 2002 Colin Perkins


Congestion Control

• Adaptation must be done by Normal operation Congestion Collapse


the application

Packets delivered
– Possible rate changes
depend on the codec
– Complex feedback loop
between codec and network

Packets sent
• For RTP, implies senders
should observe receiver
feedback
– If loss fraction is non-zero,
consider sending less
– As loss decreases, consider
sending faster

Copyright © 2002 Colin Perkins


TCP Friendly Rate Control

• Possible to predict the long- • The "best current practice"


term average throughput of congestion control scheme
TCP for multimedia flows

s • With appropriate parameters:


T=
2p 3p – Fair to TCP on average
R + 3 p(1 + 32 p 2 ) ⋅ Trto
3 8 – Slowly changing rate

• Derive throughput in terms


of observed loss rate, RTT
and packet size
– Measurable qualities in RTP
• Adapt sending rate to match
– Driven by reception of RTCP
– Drives codec operation

Copyright © 2002 Colin Perkins


Limitations of Congestion Control

• TCP friendly algorithms are • Human factors aspects play a


new, and evolving key role
– Very limited deployment – Congestion control implies
– Not clear that they have variable quality
reached their final form – Subjectively very annoying,
unless the rate of change is
slow
• Interactions between codec
– Can have a significant impact
and network are not well on congestion control
defined
– Unclear how slow response,
or limited adaptability, will
impact fairness

Copyright © 2002 Colin Perkins


Talk Outline

• Use of IP for real-time traffic


• Protocols for real-time multimedia over IP
– Outline of signalling protocols
– Outline of media transport protocols
• RTP: Real-time Transport Protocol
– RTP data transfer protocol
– RTP control protocol
• Robustness
– Playout and timing correction
– Error correction
– Congestion control
• Security
• Conclusions

Copyright © 2002 Colin Perkins


Security

Several aspects to multimedia security


• Confidentiality of the media
RTP can help here
• Authentication of the sender
• Watermarking
• Storage

Copyright © 2002 Colin Perkins


Security in RTP

• Basic RTP provides limited • A secure RTP profile is under


security development
– Packets may be encrypted – Encryption for confidentiality
• DES is specified; algorithm • Encrypts only the payload data,
may be negotiated during not the headers
session setup • AES in counter or F8 mode
– Does not support sender • Robust to bit errors, allows
authentication header compression, suitable
for cellular wireless
– Sender authentication
• Adds a trailer to each packet,
containing authentication code
• HMAC-SHA1

Copyright © 2002 Colin Perkins


Watermarking, Storage and DRM

• RTP is concerned only with • Implications:


the transmission of media – Receivers may store the payload
• Does not consider: • RTP cannot influence the process
– The contents of the video • The copy may not be perfect
image – Watermarks or other embedded
– How the image is captured, data can be inserted at source
generated and stored • But must be robust to packet loss

Copyright © 2002 Colin Perkins


Talk Outline

• Use of IP for real-time traffic


• Protocols for real-time multimedia over IP
– Outline of signalling protocols
– Outline of media transport protocols
• RTP: Real-time Transport Protocol
– RTP data transfer protocol
– RTP control protocol
• Robustness
– Playout and timing correction
– Error correction
– Congestion control
• Security
• Conclusions

Copyright © 2002 Colin Perkins


Conclusions

• IP provides a non-optimal service for video transport


– Careful network engineering can solve many problems

• Multimedia protocol framework comprises:


– Signalling: H.323, SIP, RTSP, SAP
– Media Transport: RTP + codecs

• RTP provides:
– Robust, flexible and extensible media transport
– Range of error correction schemes
– Range of security solutions

• Limitations:
– Congestion control

Copyright © 2002 Colin Perkins


For More Information

IETF Audio/Video Transport Working Group


http://www.ietf.org/html.charters/avt-charter.html

Colin Perkins
http://www.east.isi.edu/~csp/

Copyright © 2002 Colin Perkins

Вам также может понравиться