8

Understanding the JMF RTP API

JMF enables the playback and transmission of RTP streams through the APIs defined in the javax.media.rtp, javax.media.rtp.event, and javax.media.rtp.rtcp packages. JMF can be extended to support additional RTP-specific formats and dynamic payloads through the standard JMF plug-in mechanism.

Note: JMF-compliant implementations are not required to support the RTP APIs in javax.media.rtp, javax.media.rtp.event, and javax.media.rtp.rtcp. The reference implementations of JMF provided by Sun Microsystems, Inc. and IBM Corporation fully support these APIs.

You can play incoming RTP streams locally, save them to a file, or both.

Figure 8-1: RTP reception.

For example, the RTP APIs could be used to implement a telephony application that answers calls and records messages like an answering machine.

Similarly, you can use the RTP APIs to transmit captured or stored media streams across the network. Outgoing RTP streams can originate from a file or a capture device. The outgoing streams can also be played locally, saved to a file, or both.

Figure 8-2: RTP transmission.

For example, you could implement a video conferencing application that captures live audio and video and transmits it across the network using a separate RTP session for each media type.

Similarly, you might record a conference for later broadcast or use a prerecorded audio stream as "hold music" in a conferencing application.

RTP Architecture

The JMF RTP APIs are designed to work seamlessly with the capture, presentation, and processing capabilities of JMF. Players and processors are used to present and manipulate RTP media streams just like any other media content. You can transmit media streams that have been captured from a local capture device using a capture DataSource or that have been stored to a file using a DataSink. Similarly, JMF can be extended to support additional RTP formats and payloads through the standard plug-in mechanism.

Figure 8-3: High-level JMF RTP architecture.

Session Manager

In JMF, a SessionManager is used to coordinate an RTP session. The session manager keeps track of the session participants and the streams that are being transmitted.

The session manager maintains the state of the session as viewed from the local participant. In effect, a session manager is a local representation of a distributed entity, the RTP session. The session manager also handles the RTCP control channel, and supports RTCP for both senders and receivers.

The SessionManager interface defines methods that enable an application to initialize and start participating in a session, remove individual streams created by the application, and close the entire session.

Session Statistics

The session manager maintains statistics on all of the RTP and RTCP packets sent and received in the session. Statistics are tracked for the entire session on a per-stream basis. The session manager provides access to global reception and transmission statistics:

• GlobalReceptionStats: Maintains global reception statistics for the session.
• GlobalTransmissionStats: Maintains cumulative transmission statistics for all local senders.

Statistics for a particular recipient or outgoing stream are available from the stream:

• ReceptionStats: Maintains source reception statistics for an individual participant.
• TransmissionStats: Maintains transmission statistics for an individual send stream.

Session Participants

The session manager keeps track of all of the participants in a session. Each participant is represented by an instance of a class that implements the Participant interface. SessionManagers create a Participant whenever an RTCP packet arrives that contains a source description (SDES) with a canonical name(CNAME) that has not been seen before in the session (or has timed-out since its last use). Participants can be passive (sending control packets only) or active (also sending one or more RTP data streams).

There is exactly one local participant that represents the local client/server participant. A local participant indicates that it will begin sending RTCP control messages or data and maintain state on incoming data and control messages by starting a session.

A participant can own more than one stream, each of which is identified by the synchronization source identifier (SSRC) used by the source of the stream.

Session Streams

The SessionManager maintains an RTPStream object for each stream of RTP data packets in the session. There are two types of RTP streams:

• ReceiveStream represents a stream that's being received from a remote participant.
• SendStream represents a stream of data coming from the Processor or input DataSource that is being sent over the network.

A ReceiveStream is constructed automatically whenever the session manager detects a new source of RTP data. To create a new SendStream, you call the SessionManager createSendStream method.

RTP Events

Several RTP-specific events are defined in javax.media.rtp.event. These events are used to report on the state of the RTP session and streams.

Figure 8-4: RTP events.

To receive notification of RTP events, you implement the appropriate RTP listener and register it with the session manager:

• SessionListener: Receives notification of changes in the state of the session.
• SendStreamListener: Receives notification of changes in the state of an RTP stream that's being transmitted.
• ReceiveStreamListener: Receives notification of changes in the state of an RTP stream that's being received.
• RemoteListener: Receives notification of events or RTP control messages received from a remote participant.

Session Listener

You can implement SessionListener to receive notification about events that pertain to the RTP session as a whole, such as the addition of new participants.

There are two types of session-wide events:

• NewParticipantEvent: Indicates that a new participant has joined the session.
• LocalCollisionEvent: Indicates that the participant's synchronization source is already in use.

Send Stream Listener

You can implement SendStreamListener to receive notification whenever:

• New send streams are created by the local participant.
• The transfer of data from the DataSource used to create the send stream has started or stopped.
• The send stream's format or payload changes.

There are five types of events associated with a SendStream:

• NewSendStreamEvent: Indicates that a new send stream has been created by the local participant.
• ActiveSendStreamEvent: Indicates that the transfer of data from the DataSource used to create the send stream has started.
• InactiveSendStreamEvent: Indicates that the transfer of data from the DataSource used to create the send stream has stopped.
• LocalPayloadChangeEvent: Indicates that the stream's format or payload has changed.
• StreamClosedEvent: Indicates that the stream has been closed.

Receive Stream Listener

You can implement ReceiveStreamListener to receive notification whenever:

• New receive streams are created.
• The transfer of data starts or stops.
• The data transfer times out.
• A previously orphaned ReceiveStream has been associated with a Participant.
• An RTCP APP packet is received.
• The receive stream's format or payload changes.

You can also use this interface to get a handle on the stream and access the RTP DataSource so that you can create a MediaHandler.

There are seven types of events associated with a ReceiveStream:

• NewReceiveStreamEvent: Indicates that the session manager has created a new receive stream for a newly-detected source.
• ActiveReceiveStreamEvent: Indicates that the transfer of data has started.
• InactiveReceiveStreamEvent: Indicates that the transfer of data has stopped.
• TimeoutEvent: Indicates that the data transfer has timed out.
• RemotePayloadChangeEvent: Indicates that the format or payload of the receive stream has changed.
• StreamMappedEvent: Indicates that a previously orphaned receive stream has been associated with a participant.
• ApplicationEvent: Indicates that an RTCP APP packet has been received.

Remote Listener

You can implement RemoteListener to receive notification of events or RTP control messages received from a remote participants. You might want to implement RemoteListener in an application used to monitor the session--it enables you to receive RTCP reports and monitor the quality of the session reception without having to receive data or information on each stream.

There are three types of events associated with a remote participant:

• ReceiverReportEvent: Indicates that an RTP receiver report has been received.
• SenderReportEvent: Indicates that an RTP sender report has been received.
• RemoteCollisionEvent: Indicates that two remote participants are using the same synchronization source ID (SSRC).

RTP Data

The streams within an RTP session are represented by RTPStream objects. There are two types of RTPStreams: ReceiveStream and SendStream. Each RTP stream has a buffer data source associated with it. For ReceiveStreams, this DataSource is always a PushBufferDataSource.

The session manager automatically constructs new receive streams as it detects additional streams arriving from remote participants. You construct new send streams by calling createSendStream on the session manager.

Data Handlers

The JMF RTP APIs are designed to be transport-protocol independent. A custom RTP data handler can be created to enable JMF to work over a specific transport protocol. The data handler is a DataSource that can be used as the media source for a Player.

The abstract class RTPPushDataSource defines the basic elements of a JMF RTP data handler. A data handler has both an input data stream (PushSourceStream) and an output data stream (OuputDataStream). A data handler can be used for either the data channel or the control channel of an RTP session. If it is used for the data channel, the data handler implements the DataChannel interface.

An RTPSocket is an RTPPushDataSource has both a data and control channel. Each channel has an input and output stream to stream data to and from the underlying network. An RTPSocket can export RTPControls to add dynamic payload information to the session manager.

Because a custom RTPSocket can be used to construct a Player through the Manager, JMF defines the name and location for custom RTPSocket implementations:

 <protocol package-prefix>.media.protocol.rtpraw.DataSource

RTP Data Formats

All RTP-specific data uses an RTP-specific format encoding as defined in the AudioFormat and VideoFormat classes. For example, gsm RTP encapsulated packets have the encoding set to AudioFormat.GSM_RTP, while jpeg-encoded video formats have the encoding set to VideoFormat.JPEG_RTP.

AudioFormat defines four standard RTP-specific encoding strings:

 public static final String ULAW_RTP = "JAUDIO_G711_ULAW/rtp";
public static final String DVI_RTP = "dvi/rtp";
public static final String G723_RTP = "g723/rtp";
public static final String GSM_RTP = "gsm/rtp";

VideoFormat defines three standard RTP-specific encoding strings:

 public static final String JPEG_RTP = "jpeg/rtp";
public static final String H261_RTP = "h261/rtp";
public static final String H263_RTP = "h263/rtp";

RTP Controls

The RTP API defines one RTP-specific control, RTPControl. RTPControl is typically implemented by RTP-specific DataSources. It provides a mechanism to add a mapping between a dynamic payload and a Format. RTPControl also provides methods for accessing session statistics and getting the current payload Format.

SessionManager also extends the Controls interface, enabling a session manager to export additional Controls through the getControl and getControls methods. For example, the session manager can export a BufferControl to enable you to specify the buffer length and threshold.

Reception

The presentation of an incoming RTP stream is handled by a Player. To receive and present a single stream from an RTP session, you can use a MediaLocator that describes the session to construct a Player. A media locator for an RTP session is of the form:

 rtp://address:port[:ssrc]/content-type/[ttl]

The Player is constructed and connected to the first stream in the session.

If there are multiple streams in the session that you want to present, you need to use a session manager. You can receive notification from the session manager whenever a stream is added to the session and construct a Player for each new stream. Using a session manager also enables you to directly monitor and control the session.

Transmission

A session manager can also be used to initialize and control a session so that you can stream data across the network. The data to be streamed is acquired from a Processor.

For example, to create a send stream to transmit data from a live capture source, you would:

1. Create, initialize, and start a SessionManager for the session.
2. Construct a Processor using the appropriate capture DataSource.
3. Set the output format of the Processor to an RTP-specific format. An appropriate RTP packetizer codec must be available for the data format you want to transmit.
4. Retrieve the output DataSource from the Processor.
5. Call createSendStream on the session manager and pass in the DataSource.

You control the transmission through the SendStream start and stop methods.

When it is first started, the SessionManager behaves as a receiver (sends out RTCP receiver reports). As soon as a SendStream is created, it begins to send out RTCP sender reports and behaves as a sender host as long as one or more send streams exist. If all SendStreams are closed (not just stopped), the session manager reverts to being a passive receiver.

Extensibility

Like the other parts of JMF, the RTP capabilities can be enhanced and extended. The RTP APIs support a basic set of RTP formats and payloads. Advanced developers and technology providers can implement JMF plug-ins to support dynamic payloads and additional RTP formats.

Implementing Custom Packetizers and Depacketizers

To implement a custom packetizer or depacketizer, you implement the JMF Codec interface. (For general information about JMF plug-ins, see Implementing JMF Plug-Ins.)