Programming with JMF (Advanced)

Learning Objectives
The purpose of this material is to teach you more programming skills in JMF. After completing this material, you would be able to write codes for setting up the voice communication channel of the simplified Internet phone system.

1. Programming with JMF

In this section, we describe various programming skills in JMF.

1.1 How to Capture Audio Stream from a Capture Device?

i) Locate the capture device by querying the CaptureDeviceManager. e.g., Vector cds = CaptureDeviceManager.getDeviceList( new AudioFormat(AudioFormat.LINEAR)); Get a CaptureDeviceInfo object for the device. This object contains the properties of the device. e.g., CaptureDeviceInfo cdi = null; if (cds.size()>0) cdi = (CaptureDeviceInfo)cds.elementAt(0);

ii)

iii) Get MediaLocator by calling getLocator method from the CaptureDeviceInfo object (MediaLocator stores the identifier of the capture device). e.g., MediaLocator mediaLocator = cdi.getLocator(); The code segment of how to capture an audio device is as follows.
/** * Filename: SendProcessor.java */ ... // Constructor public SendProcessor() { // Locate the capture device by querying the CaptureDeviceManager Vector cds = CaptureDeviceManager.getDeviceList(new AudioFormat(AudioFormat.LINEAR)); // Get a CaptureDeviceInto object CaptureDeviceInfo cdi = null; if (cds.size()>0) cdi = (CaptureDeviceInfo)cds.elementAt(0) else System.exit(-1); // Get MediaLocator by calling getLocator method from the CaptureDeviceInfo object; MediaLocator mediaLocator = cdi.getLocator(); ... } ...

Prepared by Tony K. C. Chan.

1.2 How to Setup a RTP Session for Communication?

i) ii) Create RTPManager object. e.g., rtpManager = RTPManager.newInstance(); Initialize RTPManager with the local hosts address. e.g., rtpManager.initialize(new SessionAddress( InetAddress.getLocalHost(), port));

iii) Create the target address as a SessionAddress. e.g., targetAddr = new SessionAddress( InetAddress.getByName("10.10.10.1"), port); iv) Add the SessionAddress as a target of the RTPManager. e.g., rtpManager.addTarget(targetAddr); v) Add listeners (ReceiveStreamListener must be added to receive incoming stream). e.g., rtpManager.addReceiveStreamListener(this);

vi) Implement the Update method for the ReceiveStreamEvent as follows. e.g., public void update(ReceiveStreamEvent e) { if (e instanceof NewReceiveStreamEvent) { ... } } The code segment of how to setup a RTP session is as follows.
/** * File name: ChattingManager.java */ ... public class ChattingManager implements ReceiveStreamListener { // Declare variables int rtpPort; // Declare a RTPManager RTPManager rtpManager; ... public ChattingManager(int aMediaPort) { rtpPort = aMediaPort; ... } public void setupConnection(InetAddress aTargetIp, int aTargetPort) { // Create a new instance of RTPManager ... // Initialize the RTPManager with the local SessionAddress ... // Add ReceiveStreamListener ... // Add the target SessionAddress ... ... } public void update(ReceiveStreamEvent e) { ... } }

Prepared by Tony K. C. Chan.

1.3 How to Terminate a RTP Session?

i) ii) Remove all the target addresses. e.g., rtpManager.removeTargets("Session end"); Release the resources for garbage collection. e.g., rtpManager.dispose(); The code segment of how to terminate a RTP session is as follows.
/** * File name: ChattingManager.java */ ... public void destroy() { // Stop and release resources of other controllers ... // Remove all added targets ... // Dispose the RTPManager ... } ...

1.4 How to Transmit a Captured Audio Stream?

i) Create DataSource of the MediaLocator that stores the identifier of the capture device. e.g., DataSource capDataSource = Manager.createDataSource( mediaLocator) Create Processor of the captured DataSource from Manager. e.g., processor = Manager.createProcessor(capDataSource);

ii)

iii) Add ControllerListener (if appropriate). e.g., processor.addControllerListener(this); iv) Implement the controllerUpdate method as follows (if appropriate). e.g., public synchronized void controllerUpdate( ControllerEvent e) { ... } v) Configure the Processor. e.g., processor.configure();

vi) Set the output content descriptor to an RTP-specific format (e.g., RAW_RTP). e.g., processor.setContentDescriptor( new ContentDescriptor(ContentDescriptor.RAW_RTP)); vii) Processing the media (e.g., convert between formats). e.g., TrackControl[] tracks = processor.getTrackControls(); for (int i=0; i<tracks.length; i++) { if (tracks[i].setFormat( new AudioFormat( AudioFormat.G723_RTP)) != null) break; tracks[i].setEnabled(false); } viii) Realize the Processor 3 Prepared by Tony K. C. Chan.

e.g., processor.realize(); ix) Get output DataSource from the Processor. e.g., outDataSoure = processor.getDataOutput(); x) Create SendStream of the output DataSource from the RTPManager. e.g., sendStream = rtpManager.createSendStream( sendProcessor.outDataSource, 0);

xi) Start the SendStream. e.g., sendStream.start(); xii) Start the Processor. e.g., processor.start(); The code segment of how to transmit the captured audio stream is as follows.
/** * File name: SendProcessor.java */ ... public class SendProcessor implements ControllerListener { // Declare a Processor & a Output DataSource. Processor processor; DataSource outDataSource = null; public SendProcessor() { // Capture the audio device ... // Create a processor ... // Configure the processor ... } public synchronized void controllerUpdate(ControllerEvent e) { if (e instanceof ConfigureCompleteEvent) { // Set the ContentDescriptor ... TrackControl[] tracks = processor.getTrackControls(); // Go through the tracks and try to program one of them to output G723_RTP data. for (int i=0; i<tracks.length; i++) { if (tracks[i].setFormat(new AudioFormat(AudioFormat.G723_RTP)) != null) break; // if tracks[i] cannot be set to G723_RTP, disable it tracks[i].setEnabled(false); } // Realize the processor ... } else if (e instanceof RealizeCompleteEvent) { // Get output DataSource and store it into outDataSource; ... } public void start() { // Start the processor ... } }

Prepared by Tony K. C. Chan.

/** * File name: ChattingManager.java */ ... public class ChattingManager implements ReceiveStreamListener { ... SendProcessor sendProcessor; SendStream sendStream; ... public ChattingManager(int aMediaPort) { ... sendProcessor = new SendProcessor(); } public void setupConnection(InetAddress aTargetIp, int aTargetPort) { // Setup the RTPManager ... // Create the send stream from the output DataSource of the SendProcessor ... } public void start() { // Start the send stream ... // Start the SendProcessor ... } ... }

1.5 How to Receive and Play an Audio Stream?

i) ii) Get the ReceiveStream from ReceiveStreamEvent. e.g., recStream = e.getReceiveStream(); Get DataSource from the ReceiveStream. e.g., recDataSource = recStream.getDataSource(); The code segment of how to receive and play the audio stream is as follows.
/** * File name: ChattingManager.java */ ... public class ChattingManager implements ReceiveStreamListener { // Declare variables ... Player player = null; public void update(ReceiveStreamEvent e) { if (e instanceof NewReceiveStreamEvent) { // Create audio stream form the ReceiveStreamEvent ... // Create DataSource form the created ... // Create player from the created DataSource ... // Start the player ... } } ... }

iii) Create Player to play the DataSource (see Laboratory 4 for details).

Prepared by Tony K. C. Chan.

2. Summary
The main steps of how to establish a voice communication channel are as follows. i) ii) Create an RTPManager object. Initialize the RTPManager with the local hosts address.

iii) Set up the listener (e.g., ReceiveStreamListener). iv) ReceiveStreamListener should be added to receive incoming stream. v) For the outgoing stream, perform the following: Obtain the output DataSource Create a SendStream object Start the SendStream.

vi) For the incoming stream, perform the following: Get the ReceiveStream Get its DataSource Handle the stream by a Player object vii) If disconnected, release all the created objects.

Prepared by Tony K. C. Chan.

Appendix
Some useful APIs are listed as follows. Class: AudioFormat javax.media.format.* Encapsulates format information for audio data. The attributes of an Description AudioFormat include the sample rate, bits per sample, and number of channels. Constructor AudioFormat(java.lang.String encoding) Constructs an AudioFormat with the specified encoding type (e.g., AudioFormat.LINEAR and AudoFormat.G723_RTP). Class: CaptureDeviceInfo javax.media.* Description A CaptureDeviceInfo object contains information about a particular capture device. Methods MediaLocator getLocator() Gets the MediaLocator needed to create a DataSource for this device through the Manager. The MediaLocator is unique and no two devices can use the same locator. Class: CaptureDeviceManager javax.media.* It is a manager class that provides access to a list of the capture devices Description available on a system. Methods static Vector getDeviceList(Format format)() Gets a list of CaptureDeviceInfo objects that corresponds to devices that can capture data in the specified Format. If no Format is specified, this method returns a list of CaptureDeviceInfo objects for all the available capture devices. Class: ContentDescriptor javax.media.protocol.* Description A ContentDescriptor identifies media data containers. Constructors ContentDescriptor(String cdName) Creates a ContentDescriptor with the specified name (e.g., ContenDescriptor.RAW_RTP). Class: Manager javax.media.* Description It is an intermediary object used to create Player, Processor and DataSource. static processor createProcessor(DataSource Methods dataSource) Creates a Processor for the DataSource. Throws: IOException if there was a problem connecting with the source. Class: Player javax.media.* It is a media handler for rendering and controlling time-based media data. Description Methods void close() Releases all resources and ceases all activity. The close method indicates that the Controller will no longer be used and can shut itself down. A ControllerClosedEvent is posted. Methods invoked on a closed Controller might throw errors.

Prepared by Tony K. C. Chan.

Class: Processor javax.media.* This interface defines a module for processing and controlling time-based Description media data. Processor extends the Player interface. Unlike a Player, which processes data as a "black box" and only renders data to preset destinations, a Processor supports a programmatic interface that enables control over the media data processing and access to output data streams. void addControllerListener(ControllerListener Methods listener) Specifies a ControllerListener to which this Controller will send events. A Controller can have multiple ControllerListeners. void configure() Prepares the Processor to be programmed. The Processor gathers information about the data it is going to process. Calling configure puts the Processor into the Configuring state and returns immediately. When this process is complete and the Processor is in the Configured state, the Processor posts a ConfigureCompleteEvent. DataSource getDataOutput() Gets the output DataSource form this Multiplexer. The DataSource is returned in the connected state. Throws: NotRealizedError If the Processor is has not yet been realized. TrackControl[] getTrackControls() Gets a TrackControl for each track in the media stream. This method can only be called once the Processor has been configured. void realize() Constructs the media dependent portions of the Controller. This may include examining media data and might take some time to complete. The realize method puts the Controller into the Realizing state and returns immediately. When realize is complete and the Controller is in the Realized state, the Controller posts a RealizeComleteEvent. ContentDescriptor setContentDescriptor( ContentDescriptor outputContentDescriptor) Sets the output content-type for this Processor. If setContentDescriptor is not called, the output DataSource is set to raw output by default: (new ContentDescriptor(ContentDescriptor.RAW)). The source streams from the DataSource are the demultiplexed tracks from the input source. void start() Starts the Processor as soon as possible. The start method attempts to transition the Processor to the Started state. If the Processor has not been Realized or Prefetched, start automatically performs those actions. The appropriate events are posted as the Player moves through each state. void stop() Stops the processing media.

Prepared by Tony K. C. Chan.

Class: ReceiveStream javax.media.rtp.* This interface representing a receiving stream within an RTP session. Description ReceiveStreams are created by the RTP subsystem, as necessary, for each independent stream of data arriving from remote participants on a session. Methods DataSource getDataSource() Returns the DataSource of the stream. Class: ReceiveStreamEvent javax.media.rtp.event.* Description The ReceiveStreamEvent will notify a listener of all events that are received on a particular ReceiveStream. This allows the user to get details on all the ReceiveStreams as they transition through various states. Methods ReceiveStream getReceiveStream() Gets the ReceiveStream from ReceiveStreamEvent. Class: RTPManager javax.media.rtp.* Description The interface implemented by the RTPManager. This is the starting point for creating, maintaining and closing an RTP session. void addReceiveStreamListener(ReceiveStreamListener Methods listener) Adds a ReceiveStreamListener. This listener listens to all the events that notify state transitions for a particular ReceiveStream. void addTarget(SessionAddress targetAddr) This method opens the session, causing RTCP reports to be generated and callbacks to be made through the SessionListener interface. This method must be called after session initialization and prior to the creation of any streams on a session. SendStream createSendStream (DataSource dataSource, int StreamIndex) This method is used to create a sending stream within the RTP session. For each time the call is made, a new sending stream will be created. void despose() Releases all objects allocated in the course of the session and prepares the RTPManager to be garbage-collected. This method should be called at the end of any RTP session. void initialize(SessionAddress localAddr) Initializes the session. Once this method has been called, the session is initialized and this method cannot be called again. static RTPManager newInstance() Create an RTPManager object for the underlying implementation class. void removeTargets(String reason) Closes the open streams associated with all remote endports. Class: SendStream javax.media.rtp.* This interface representing a sending stream within an RTP session. Description Methods void close() Removes the stream from the session. void start() It will resume data transmission over the network on this SendStream. void stop() It will temporarily stop the SendStream

Prepared by Tony K. C. Chan.

Class: SessionAddress javax.media.rtp.* Class to encapsulate a pair of internet address and a pair of ports for use in Description RTPSM methods. Constructor SessionAddress(InetAddress ipAddress, int port) Creates a SessionAddress of the given internet address and port. Class: TrackControl javax.media.control.* Description The TrackControl interface is provided by the Processor to query, control and manipulate the data of individual media tracks. Methods Format setFormat(Format format) Sets the data format. The method returns null if the format is not supported. Otherwise, it returns the format that is actually set. void setEnabled(boolean enabled) Enable or disable the track.

References
[1] A. Terrazas, J. Ostuni, and M. Barlow, Java Media APIs: Cross-Platform Imaging, Media, and Visualization, SAMS, 2002. [2] Java Media Framework API Guide. [Online] Available: http://java.sun.com/products/java-media/jmf/2.1.1/guide/index.html. [3] Java Media Framework API Specification. [Online] Available: http://java.sun.com/products/java-media/jmf/2.1.1/apidocs/. [4] Java 2 Platform, Standard Edition, v 1.4.2 API Specification. [Online] Available: http://java.sun.com/j2se/1.5.0/docs/api/.

10

Prepared by Tony K. C. Chan.