Android Multimedia

Framework
on Jelly Bean
Author: Picker

Introduction to OpenMAX

Resource from Khronos::OpenMAX

Resource from Khronos::OpenMAX

from
Multimedia Framework
to
OpenMAX

Resource from Khronos::OpenMAX

Resource from Khronos::OpenMAX

Operating System Operating System
Operating System Operating System
Multimedia Framework
Multimedia Framework
OpenMAX IL
OpenMAX IL
Software & Hardware Codec
Software & Hardware Codec
Go through the OpenMAX


e!eloper A"I #Media"layer$ e!eloper A"I #Media"layer$ e!eloper A"I #Media"layer$ e!eloper A"I #Media"layer$
%!entHandler
%!entHandler SurfaceHolder
SurfaceHolder
StageFrig&t
StageFrig&t
OpenMAX Interface
OpenMAX Interface
OpenMAX IL
OpenMAX IL
Software & Hardware Codec
Software & Hardware Codec
The simple stack architecture

Big picture of the StageFright

The Simple Workflow

The StageFright Workflow

Path of All the Components

frameworks/av/media/libmedia
Path of the Source Files
frameworks/base/media/java/android/media
frameworks/av/media/libmediaplayerservice
frameworks/base/media/jni
frameworks/av/media/libstagefright

Big Picture
Class Diagram


High-Level Applications Framework


StageFright Framework


OpenMAX Interface

Four Steps for Playing the Media File
Step 1: MediaPlayer mp = new MediaPlayer();
Step 2: mp.
setDataSource
setDataSource(mediaPath);
Step 3: mp.
prepare
prepare();
Step 4: mp.
start
start();


How to set the data source?

The workflow of the setDataSource

The sequence flow of the setDataSource


How to prepare to play?

The workflow of the prepare

The sequence flow of the prepare


How to start the playing?

The workflow of the start

The sequence flow of the start

Synchronization Architecture of StageFright

Synchronization Flow of StageFright

The Synchronization Formula of StageFright
mPositionTimeRealsUs=(
mNumFramesPlayed+
sizedone
mFrameSize
mSampleRate
)1000000
mTimeSourceDeltaUs=mPositionTimeRealUsmPositionTimeMediaUs
nowUs=RealTimeUsmTimeSourceDeltaUs
latenessUs=nowUstimeUs
mPositionTimeRealsUs: the time in real playing
mPositionTimeMediaUs: the time which is defined in the media source

The Synchronization Conditions
Conditions of latenessUs
1. > 500000ll
2. > 40000
3. < -10000 1750 if (latenessUs 500000ll
1751 !! m"udioPlayer #$ %U&&
175' !! m"udioPlayer(getMediaTimeMapping(
175) !realTimeUs* !mediaTimeUs++ ,

1757 m-ideo.uffer(release(+/
1750 m-ideo.uffer $ %U&&/

171) post-ideo23ent4l(+/
1715 return/
1715 6
1717 if (latenessUs 50000+ ,
1710 77 8e9re more than 50ms late:
1700 m-ideo.uffer(release(+/
1701 m-ideo.uffer $ %U&&/

1707
1700 post-ideo23ent4l(+/
170; return/
17;1 6
17;) if (latenessUs < (10000+ ,
17;5 77 8e9re more than 10ms early:
17;5 post-ideo23ent4l(10000+/
17;1 return/
17;7 6

Conclusion

Easy for maintaining rather than OpenCore

Support Software/Hardware Codecs

Unstable Product

Rough Mechanisms

Create the NuPlayer for playing the

multimedia streaming