2

JMFについての理解

Java^TMメディア・フレームワーク(JMF)は、時系列メディア・データの獲得、処理および配信を管理するために一体になったアーキテクチャーおよび通信プロトコルを提供します。JMFは、AIFF、AU、AVI、GSM、MIDI、MPEG、QuickTime、RMFおよびWAVのような最も標準のメディア内容タイプをサポートすることを目指しています。

Javaプラットフォーム開発の利点によって、JMFはJavaプログラムの中でオーディオとビデオのようなメディアを使用したい開発者に「一度ロジックを書けば、どどんな環境でも実行できる(Write Once, Run Anywhere^TM)」を約束します。 JMFは基礎をなすメディア・フレームワークにアクセスするために共通のクロス・プラットフォームJava APIを提供します。 開発者がJMF APIで書くことにより時系列メディアを特色とするポータブルのJavaプログラムを容易に作成することができる一方、JMFの実装は基礎をなすオペレーティング・システムの能力に影響力を行使することができます。

JMFで、時系列メディアを表示、キャプチャ、操作、格納するアプリケーションを容易に作成することができます。 フレームワークによって上級開発者および技術供給者は、生のメディア・データのカスタム処理の実行、追加コンテンツタイプおよびフォーマットを支援するJMFのシームレスな拡張、支援されたフォーマットの取り扱いの最適化、新しいプレゼンテーション・メカニズムの作成ができます。

高度な構成(High-Level Architecture)

テープデッキとVCRのような装置は、時系列メディアの記録・処理・表示のよく知られているモデルを供給します。 VCRを使用して映像を鑑賞するときは、ビデオテープの挿入によりVCRにメディアストリームを供給します。 VCRは、テープ上のデータを読み解釈し、あなたのテレビおよびスピーカーに適切な信号を送ります。

図 2-1: 時系列メディアの記録・処理・表示

JMF もこれと同じ基本的なモデルを使用しています。データソース(data source) ビデオテープのようにメディアストリームをカプセルに包みプレーヤ(player)は処理とVCRと同様のコントロールメカニズムを提供します。JMFを使っての音声と映像の再生・キャプチャはマイクロフォン、カメラ、スピーカ、モニタのような適切な入出力装置を必要とします。

データソース(Data sources)とプレーヤ(players) は時系列メディアをキャプチャ、表示、処理の為のJMFの高度APIに不可欠な部分です 。 JMFは、さらにカスタム処理コンポーネントおよび拡張のシームレスの統合を支援する、より低いレベルのAPIを提供します。 この層状化は、Javaプログラムに柔軟性を維持しつつ時系列メディアを組み入れられるようにJava開発者に使いやすいAPIを供給します。また、支援するのに必要な伸長性はメディア・アプリケーションおよび将来のメディア技術を進めました。

図 2-2: 高度な JMF 構成

タイムモデル(Time Model)

JMFは、時間をナノセカンド(10億分の１秒)の正確さで維持します。同じくナノセコンド仕様の時間をサポートするクラスがいくつかありますが、時間についての特徴は典型的にTimeオブジェクトにより象徴されます。

JMFタイムモデルをサポートしているクラスは、独立したメディアストリームのためのトラックの時間を保つためにClockを実装しています。 Clock インターフェースはメディアデータのプレゼンテーションを制御するのに必要な基本的なタイミングと同期作業を定義します。

図 2-3: JMF タイムモデル

Clock はメディアストリームが演じられてる時間の経過したトラックを保つためにTimeBaseを使います。 TimeBase はまるで腕時計の水晶振動子のように継続的にカチカチと音を刻む時間ソース(time source)を提供します。TimeBaseが提供する唯一の情報は基礎時間(time-base time)と呼ばれる現在の時間です。 基礎時間(time-base)は停止やリセットができません。基礎時間(Time-base time)は頻繁にシステムクロック(system clock)に基づきます。

Clockオブジェクトのメディア時間(media time)はメディアストリーム内の現在の位置を表します--ストリームの始まりがメディア時間ゼロ(media time zero)、 ストリームの終わりがメディアの最長メディア時間(maximum media time for the stream)です。メディアストリームのデュレーション(duration)は始めから終わりまでの経過時間(メディアストリームを表示するのにかかる時間の長さ)を意味します。 (もしメディアストリームのデュレーションを報告できるなら、そのメディアオブジェクトはDurationインターフェースを実装しています)

メディア時間のトラックを最新にしておくために、Clockは次のものを使います:

• 基礎時間の開始時間(The time-base start-time)--プレゼンテーションが始まった時にそのTimeBaseが報告する時間。
• メディアの開始時間(The media start-time)--プレゼンテーションを開始したメディアストリーム内の位置。
• 再生レート(The playback rate)--基礎時間(TimeBase)に対してどれくらい速くClockが進むか。レート(rate)は基礎時間(TimeBase)に対しての尺度です。 例えば、 1.0 の レートは メディアストリームを普通の速さで再生するレートを表し、一方 2.0のレートはプレゼンテーションが通常の２倍の速さで再生される事を示します。 レートが負の値の時はClockが基礎時間(TimeBase)と逆方向に進む事を意味します。--例えば、負のレートはメディアストリームの巻き戻しに使用されます。

プレゼンテーションが始まる時、メディアの時間が基礎時間の時間上にマッピング（写像）され、 基礎時間により時間の経過を計ることになります。プレゼンテーション中に、現在のメディア時間は次の定式を使用して計算されます:

本文

 MediaTime = MediaStartTime + Rate(TimeBaseTime - TimeBaseStartTime)

訳文

 メディア時間 = メディア時間の開始時間 + レート(基礎時間 - 基礎時間の開始時間)

プレゼンテーションが止まる時は、メディア時間は止まりますが、時間基礎の時間は進み続けます。 プレゼンテーションが再開される時に、メディア時間が現在の時間基礎の時間に再マッピング(再写像)されます。

マネージャ(Managers)

JMF APIは、主としてキャプチャに使われるオブジェクトの振る舞いと相互作用、時系列メディアの処理と表示を定義するインターフェースから構成されています。 これらのインターフェースの実装はフレームワークの構造内で処理します。マネージャ(managers)と呼ばれるオブジェクトを仲介に使って、 JMFは、既存のクラスと共にシームレスに使用することができるキー・インターフェースの新しい実装を統合することを容易にします。

JMFは次の4つのマネージャーを使用します:

• Manager--Players、Processors、DataSourcesおよびDataSinksの構造を扱います。 間接のこのレベルは、新しい実装がシームレスにJMFへ統合されることを可能にします。クライアント展望から、これらのオブジェクトは、要求されたオブジェクトが初期の実装あるいはカスタムの実装のどちらで構築されていても常に同じ方法で作成されます。
• PackageManager--カスタムのPlayers、Processors、DataSources、 および DataSinksのようなJMFクラスを含んでいるパッケージの登録を維持します。
• CaptureDeviceManager--利用可能なキャプチャデバイス(装置)の登録を維持します。
• PlugInManager--Multiplexers、Demultiplexers、Codecs、Effects、およびRenderersのような利用可能なJMFプラグ・イン処理コンポーネントのレジストリを維持します。

JMFに基づいたプログラムを書く為には、アプリケーションのためにPlayers、Processors、DataSources、およびDataSinksを構築するManagerのcreateメソッドを使う必要があるでしょう。もし、入力装置からメディアデータをキャプチャしているなら、利用可能な装置とその接続情報を探すのにCaptureDeviceManagerを使います。また、どの処理がデータ上で実行されるかコントロールすることに興味を持っている場合は、登録されているプラグインが何であるか断定する為にPlugInManager に問い合わせる事もできます。

新しいプラグインを実装することによりJMFを機能的に拡張したい場合、プラグインAPIをサポートするProcessorsがそのプラグインを利用できるようにPlugInManagerを使ってプラグイン登録ができます。 カスタムのPlayer、Processor、DataSource、DataSinkをJMFで使う為に、PackageManagerを使ってユニークなパッケージ接頭辞を登録します。

イベントモデル(Event Model)

JMF は JMF準拠プログラムが現状メディアシステムの通知を維持する為 または データ・資源が利用不可の場合のようなメディアで発生するエラー処理を可能にする為に 構造化されたイベントレポーティングメカニズムを使います。オブジェクトが現在の条件を通知する必要がある場合は常に、 それはMediaEventを通知します。MediaEvent はたくさんの特別な種類のイベントを識別する為にサブクラス化されます。これらのオブジェクトは確立されたイベントの為の Java Beans パターンに従います。

MediaEventsを通知できるJMFオブジェクトのタイプについては 、 JMF が対応するリスナーインターフェースを定義します。MediaEventが通知されたときにその通知を受信するためには、適切なリスナーインターフェースを実装し、addListener メソッドをコールすればイベントを通知するオブジェクトのクラスを登録します。

(PlayersとProcessorsのような)ControllerオブジェクトとGainControlのようないくらかのControl オブジェクトはメディアイベントを通知します。

図 2-4: JMF イベントモデル.

RTPSessionManagerもまたイベントを通知します。詳細についてはRTP Eventsをご覧下さい。

データモデル(Data Model)

JMF メディアプレーヤは メディアコンテンツの転送を管理する為に、通常はDataSourcesを使用します。DataSource はメディアのロケーションとプロトコルを隠蔽し(カプセル化し)、ソフトウェアがメディアを転送する為に使います。一度ソースを獲得してしまうと、他のメディアを転送するための再利用はできなくなります。

DataSourceはJMFのMediaLocatorまたはURL (universal resource locator)のどちらか一方で識別されます。MediaLocatorはURLと似ていてURLから構築することができますが、対応するプロトコルハンドラがシステムにインストールされていなくても構築することができます。( Javaでは、対応するプロトコルハンドラがシステムにインストールされている場合のみURLを構築できます。)

DataSourceはSourceStreamオブジェクトのセットを管理します。標準的なデータソースは転送のユニットとしてバイト配列を使います。バッファーデータソース(buffer data source)はその転送のユニットにBuffer オブジェクトを使います。 JMF は次のさまざまなを DataSourceオブジェクトの種類を定義します:

図 2-5: JMF データモデル.

プッシュデータソースとプルデータソース

メディアデータはローカルファイルまたはネットワークファイルと生放送のようなさまざまなソースから得ることが出来ます。 JMFデータソースはデータ転送がどのように始められるかによって分類することができます:

• プルデータソース(Pull Data-Source)--クライアントがデータ転送を始めてプルデータソースからデータのフローを制御します。この種のデータ用の確立しているプロトコルはハイパーテキスト・トランスファー・プロトコル(HTTP)およびFILEを含んでいます。 JMFは2種類のプルデータソースを定義します: PullDataSource と転送のユニットとしてBufferオブジェクトを使うPullBufferDataSourceです。
• プッシュデータソース(Push Data-Source)--サーバがデータ転送を始めてプッシュデータソースからデータのフローを制御します。プッシュデータソースは放送メディア、マルチキャスト・メディアおよびビデオ・オン・デマンド(VOD)を含んでいます。 放送データについての、1つのプロトコルはインターネット技術特別調査委員会(IETF)に開発中のリアル・タイム・トランスポート・プロトコル(RTP)です。 SGIによって開発されたMediaBaseプロトコルの一つはVODの為に使われます。JMF は2種類のプッシュデータソースを定義します: PushDataSource と転送のユニットとしてBufferオブジェクトを使うPushBufferDataSourceです。

クライアント・プログラムがユーザに施すことができるコントロールの程度は、示されているデータソースのタイプに依存します。 例えば、MPEGファイルの再生位置は変えることができます。また、クライアント・プログラムは、ユーザがビデオクリップを再生したりビデオ中の新しい再生位置を探すことを許します。 対照的に、放送メディアはサーバー管理の下にあり、再生位置を変えることができません。 いくつかのVODプロトコルは制限のあるユーザ・コントロールを支援するでしょう--例えば、クライアントプログラムはユーザが新しい再生位置を探しす事を許し、早送りまたは巻き戻しを許さないかもしれません。

特別なデータソース

JMF は複製可能データソース(cloneable data sources)と融合データソース(merging data sources)の２種類の特別なタイプのデータソースを定義します。

複製可能データソース(cloneable data sources)はプル(pull)またはプッシュ(push)どちらのDataSourceの複製(クローン)を作成するのにも使われます。 複製可能データソース(cloneable DataSource)を作成するには、ManagerのcreateCloneableDataSource メソッドをコールし引数に複製したいDataSourceを渡します。一旦DataSourceがcreateCloneableDataSourceに渡されたならば、複製可能DataSourceおよびその複製(クローン)とだけ対話するべきです;オリジナルのDataSourceはもはや直接使用されてはなりません。

複製可能データソース(cloneable data sources)はcreateCloneというひとつのメソッドを定義するSourceCloneableインターフェースを実装しています。createCloneをコールすることで、cloneable DataSourceを構築するために使われるDataSourceをいくつでも作りだす事ができます。クローンはそれらを作成できる複製可能データソース(cloneable DataSource)によって制御することができます。 -- 接続(connect)、切断(disconnect)、開始(start)、停止(stop)のメソッドが複製可能データソース(cloneable DataSource)でコールされた時、そのクローンへメソッドのコールが伝播されます。

クローンはそれらを作成する為に使われた複製可能データソース(cloneable data source)あるいはオリジナルのデータソース(DataSource)と同じプロパティを持つ必要がありません。例えば、キャプチャデバイス用に作成された複製可能なデータソースは、そのクローンのためのマスター・データソースとして機能するかもしれません--例えば、キャプチャデバイス用に作成された複製可能なデータソースは、そのクローンのためのマスター・データソースとして機能するかもしれません--この場合、もし複製可能なデータソースが使用されなければ、クローンはデータを生産しないでしょう。あなたがクローンを作ることができるデータソースおよび1つ以上のクローンの両方を引っ掛ければ、クローンはマスターと同じレートでデータを生産するでしょう。

MergingDataSourceはいくつかの DataSourcesから単一のDataSourceへSourceStreams を組み合わせる為に使用できます. これによりマージされた１セットのDataSourcesを一つのポイントから管理することができます--MergingDataSourceで接続(connect), 切断(disconnect), 開始(start), または停止(stop)メソッドが呼び出されたとき、そのメソッド呼び出しはマージされたDataSourcesに伝播されます。

MergingDataSourceを構築するには、ManagerクラスのcreateMergingDataSourceメソッドをコールし、組み合わせたいデータソースを含んだ配列を渡します。組み合わせる為には、すべてのDataSourcesが同じ型でなくてはなりません。例えば、PullDataSourceとPushDataSourceは組み合わせる事ができません。組み合わされた DataSource オブジェクトらのいずれかが存在する間、 組み合わされたDataSourceは存在します。 その ContentTypeはapplication/mixed-mediaです。

データフォーマット(Data Formats)

オブジェクトの厳密なメディアフォーマットは Formatオブジェクトによって表されます。 formatオブジェクトはそれ自身ではエンコード化する特定のパラメータまたはグローバルな時間情報を備えていません。エンコード名とフォーマットが要求するデータ型を備えます。

JMF は音声と映像に特有のフォーマットを定義する為に Formatを派生します。

図 2-6: JMF メディアフォーマット.

AudioFormatサンプルレート、サンプルビット数、チャンネルの数などの音声形式を属性の仕様を記述します。VideoFormat は映像データに関連する情報を含みます。一般的な映像形式の属性を記述する VideoFormat から得られるいくつかの形式は次のものを含みます。

• IndexedColorFormat
• RGBFormat
• YUVFormat
• JPEGFormat
• H261Format
• H263Format

Controllerからのフォーマット変更の通知を得る為に、ControllerListener インターフェースを実装し、FormatChangeEventsイベントを監視します。 (詳細は、Responding to Media Eventsの項を参照してください。)

Controls

JMF Control はオブジェクトの設定のメカニズムと属性を取得する機能を提供します。オブジェクトの属性を制御できるユーザインターフェースのコンポーネントに対応したControl オブジェクトもあります。 Controller オブジェクト、 DataSource オブジェクト、 DataSink オブジェクト、JMF プラグイン(JMF plug-ins)などの多くの JMF オブジェクトはControlsをみせます。

対応する Control オブジェクトに接続の提供を要求するどのJMFオブジェクトもControlsインターフェースを実装することができます。 Controls インタフェースは関連するControlオブジェクトを検索するメソッドを定義します。DataSource と PlugIn は Control オブジェクトへの接続をできるようにする為、Controls インタフェースを使います 。

標準 Controls

JMF defines the standard Control interfaces shown in Figure 2-8:, "JMF controls."

CachingControl enables download progress to be monitored and displayed. If a Player or Processor can report its download progress, it implements this interface so that a progress bar can be displayed to the user.

GainControl enables audio volume adjustments such as setting the level and muting the output of a Player or Processor. It also supports a listener mechanism for volume changes.

Figure 2-7: Gain control.

Figure 2-8: JMF controls.

DataSink or Multiplexer objects that read media from a DataSource and write it out to a destination such as a file can implement the StreamWriterControl interface. This Control enables the user to limit the size of the stream that is created.

FramePositioningControl and FrameGrabbingControl export frame-based capabilities for Players and Processors. FramePositioningControl enables precise frame positioning within a Player or Processor object's media stream. FrameGrabbingControl provides a mechanism for grabbing a still video frame from the video stream. The FrameGrabbingControl can also be supported at the Renderer level.

Objects that have a Format can implement the FormatControl interface to provide access to the Format. FormatControl also provides methods for querying and setting the format.

A TrackControl is a type of FormatControl that provides the mechanism for controlling what processing a Processor object performs on a particular track of media data. With the TrackControl methods, you can specify what format conversions are performed on individual tracks and select the Effect, Codec, or Renderer plug-ins that are used by the Processor. (For more information about processing media data, see Processing Time-Based Media with JMF.)

Two controls, PortControl and MonitorControl enable user control over the capture process. PortControl defines methods for controlling the output of a capture device. MonitorControl enables media data to be previewed as it is captured or encoded.

BufferControl enables user-level control over the buffering done by a particular object.

JMF also defines several codec controls to enable control over hardware or software encoders and decoders:

• BitRateControl--used to export the bit rate information for an incoming stream or to control the encoding bit rate. Enables specification of the bit rate in bits per second.
• FrameProcessingControl--enables the specification of frame processing parameters that allow the codec to perform minimal processing when it is falling behind on processing the incoming data.
• FrameRateControl--enables modification of the frame rate.
• H261Control--enables control over the H.261 video codec still-image transmission mode.
• H263Control--enables control over the H.263 video-codec parameters, including support for the unrestricted vector, arithmetic coding, advanced prediction, PB Frames, and error compensation extensions.
• KeyFrameControl--enables the specification of the desired interval between key frames. (The encoder can override the specified key-frame interval if necessary.)
• MpegAudioControl--exports an MPEG audio codec's capabilities and enables the specification of selected MPEG encoding parameters.
• QualityControl--enables specification of a preference in the trade-off between quality and CPU usage in the processing performed by a codec. This quality hint can have different effects depending on the type of compression. A higher quality setting will result in better quality of the resulting bits, for example better image quality for video.
• SilenceSuppressionControl--enables specification of silence suppression parameters for audio codecs. When silence suppression mode is on, an audio encoder does not output any data if it detects silence at its input.

User Interface Components

A Control can provide access to a user interface Component that exposes its control behavior to the end user. To get the default user interface component for a particular Control, you call getControlComponent. This method returns an AWT Component that you can add to your applet's presentation space or application window.

A Controller might also provide access to user interface Components. For example, a Player provides access to both a visual component and a control panel component--to retrieve these components, you call the Player methods getVisualComponent and getControlPanelComponent.

If you don't want to use the default control components provided by a particular implementation, you can implement your own and use the event listener mechanism to determine when they need to be updated. For example, you might implement your own GUI components that support user interaction with a Player. Actions on your GUI components would trigger calls to the appropriate Player methods, such as start and stop. By registering your custom GUI components as ControllerListeners for the Player, you can also update your GUI in response to changes in the Player object's state.

Extensibility

Advanced developers and technology providers can extend JMF functionality in two ways:

• By implementing custom processing components (plug-ins) that can be interchanged with the standard processing components used by a JMF Processor
• By directly implementing the Controller, Player, Processor, DataSource, or DataSink interfaces

Implementing a JMF plug-in enables you to customize or extend the capabilities of a Processor without having to implement one from scratch. Once a plug-in is registered with JMF, it can be selected as a processing option for any Processor that supports the plug-in API. JMF plug-ins can be used to:

• Extend or replace a Processor object's processing capability piecewise by selecting the individual plug-ins to be used.
• Access the media data at specific points in the data flow. For example, different Effect plug-ins can be used for pre- and post-processing of the media data associated with a Processor.
• Process media data outside of a Player or Processor. For example, you might use a Demultiplexer plug-in to get individual audio tracks from a multiplexed media-stream so you could play the tracks through Java Sound.

In situations where an even greater degree of flexibility and control is required, custom implementations of the JMF Controller, Player, Processor, DataSource, or DataSink interfaces can be developed and used seamlessly with existing implementations. For example, if you have a hardware MPEG decoder, you might want to implement a Player that takes input from a DataSource and uses the decoder to perform the parsing, decoding, and rendering all in one step. Custom Players and Processors can also be implemented to integrate media engines such as Microsoft's Media Player, Real Network's RealPlayer, and IBM's HotMedia with JMF.

Note: JMF Players and Processors are not required to support plug-ins. Plug-ins won't work with JMF 1.0-based Players and some Processor implementations might choose not to support them. The reference implementation of JMF 2.0 provided by Sun Microsystems, Inc. and IBM Corporation fully supports the plug-in API.

Presentation

In JMF, the presentation process is modeled by the Controller interface. Controller defines the basic state and control mechanism for an object that controls, presents, or captures time-based media. It defines the phases that a media controller goes through and provides a mechanism for controlling the transitions between those phases. A number of the operations that must be performed before media data can be presented can be time consuming, so JMF allows programmatic control over when they occur.

A Controller posts a variety of controller-specific MediaEvents to provide notification of changes in its status. To receive events from a Controller such as a Player, you implement the ControllerListener interface. For more information about the events posted by a Controller, see Controller Events.

The JMF API defines two types of Controllers: Players and Processors. A Player or Processor is constructed for a particular data source and is normally not re-used to present other media data.

Figure 2-9: JMF controllers.

Players

A Player processes an input stream of media data and renders it at a precise time. A DataSource is used to deliver the input media-stream to the Player.The rendering destination depends on the type of media being presented.

Figure 2-10: JMF player model.

A Player does not provide any control over the processing that it performs or how it renders the media data.

Player supports standardized user control and relaxes some of the operational restrictions imposed by Clock and Controller.

Figure 2-11: JMF players.

Player States

A Player can be in one of six states. The Clock interface defines the two primary states: Stopped and Started. To facilitate resource management, Controller breaks the Stopped state down into five standby states: Unrealized, Realizing, Realized, Prefetching, and Prefetched.

Figure 2-12: Player states.

In normal operation, a Player steps through each state until it reaches the Started state:

• A Player in the Unrealized state has been instantiated, but does not yet know anything about its media. When a media Player is first created, it is Unrealized.
• When realize is called, a Player moves from the Unrealized state into the Realizing state. A Realizing Player is in the process of determining its resource requirements. During realization, a Player acquires the resources that it only needs to acquire once. These might include rendering resources other than exclusive-use resources. (Exclusive-use resources are limited resources such as particular hardware devices that can only be used by one Player at a time; such resources are acquired during Prefetching.) A Realizing Player often downloads assets over the network.
• When a Player finishes Realizing, it moves into the Realized state. A Realized Player knows what resources it needs and information about the type of media it is to present. Because a Realized Player knows how to render its data, it can provide visual components and controls. Its connections to other objects in the system are in place, but it does not own any resources that would prevent another Player from starting.
• When prefetch is called, a Player moves from the Realized state into the Prefetching state. A Prefetching Player is preparing to present its media. During this phase, the Player preloads its media data, obtains exclusive-use resources, and does whatever else it needs to do to prepare itself to play. Prefetching might have to recur if a Player object's media presentation is repositioned, or if a change in the Player object's rate requires that additional buffers be acquired or alternate processing take place.
• When a Player finishes Prefetching, it moves into the Prefetched state. A Prefetched Player is ready to be started.
• Calling start puts a Player into the Started state. A Started Player object's time-base time and media time are mapped and its clock is running, though the Player might be waiting for a particular time to begin presenting its media data.

A Player posts TransitionEvents as it moves from one state to another. The ControllerListener interface provides a way for your program to determine what state a Player is in and to respond appropriately. For example, when your program calls an asynchronous method on a Player or Processor, it needs to listen for the appropriate event to determine when the operation is complete.

Using this event reporting mechanism, you can manage a Player object's start latency by controlling when it begins Realizing and Prefetching. It also enables you to determine whether or not the Player is in an appropriate state before calling methods on the Player.

Methods Available in Each Player State

To prevent race conditions, not all methods can be called on a Player in every state. The following table identifies the restrictions imposed by JMF. If you call a method that is illegal in a Player object's current state, the Player throws an error or exception.

Method	Unrealized Player	Realized Player	Prefetched Player	Started Player
addController	NotRealizedError	legal	legal	ClockStartedError
deallocate	legal	legal	legal	ClockStartedError
getControlPanelComponent	NotRealizedError	legal	legal	legal
getGainControl	NotRealizedError	legal	legal	legal
getStartLatency	NotRealizedError	legal	legal	legal
getTimeBase	NotRealizedError	legal	legal	legal
getVisualComponent	NotRealizedError	legal	legal	legal
mapToTimeBase	ClockStoppedException	ClockStoppedException	ClockStoppedException	legal
removeController	NotRealizedError	legal	legal	ClockStartedError
setMediaTime	NotRealizedError	legal	legal	legal
setRate	NotRealizedError	legal	legal	legal
setStopTime	NotRealizedError	legal	legal	StopTimeSetError if previously set
setTimeBase	NotRealizedError	legal	legal	ClockStartedError
syncStart	NotPrefetchedError	NotPrefetchedError	legal	ClockStartedError

Table 2-1: Method restrictions for players.

Processors

Processors can also be used to present media data. A Processor is just a specialized type of Player that provides control over what processing is performed on the input media stream. A Processor supports all of the same presentation controls as a Player.

Figure 2-13: JMF processor model.

In addition to rendering media data to presentation devices, a Processor can output media data through a DataSource so that it can be presented by another Player or Processor, further manipulated by another Processor, or delivered to some other destination, such as a file.

For more information about Processors, see Processing.

Presentation Controls

In addition to the standard presentation controls defined by Controller, a Player or Processor might also provide a way to adjust the playback volume. If so, you can retrieve its GainControl by calling getGainControl. A GainControl object posts a GainChangeEvent whenever the gain is modified. By implementing the GainChangeListener interface, you can respond to gain changes. For example, you might want to update a custom gain control Component.

Additional custom Control types might be supported by a particular Player or Processor implementation to provide other control behaviors and expose custom user interface components. You access these controls through the getControls method.

For example, the CachingControl interface extends Control to provide a mechanism for displaying a download progress bar. If a Player can report its download progress, it implements this interface. To find out if a Player supports CachingControl, you can call getControl(CachingControl) or use getControls to get a list of all the supported Controls.

Standard User Interface Components

A Player or Processor generally provides two standard user interface components, a visual component and a control-panel component.You can access these Components directly through the getVisualComponent and getControlPanelComponent methods.

You can also implement custom user interface components, and use the event listener mechanism to determine when they need to be updated.

Controller Events

The ControllerEvents posted by a Controller such as a Player or Processor fall into three categories: change notifications, closed events, and transition events:

• Change notification events such as RateChangeEvent, DurationUpdateEvent, and FormatChangeEvent indicate that some attribute of the Controller has changed, often in response to a method call. For example, a Player posts a RateChangeEvent when its rate is changed by a call to setRate.
• TransitionEvents allow your program to respond to changes in a Controller object's state. A Player posts transition events whenever it moves from one state to another. (See Player States for more information about the states and transitions.)
• ControllerClosedEvents are posted by a Controller when it shuts down. When a Controller posts a ControllerClosedEvent, it is no longer usable. A ControllerErrorEvent is a special case of ControllerClosedEvent. You can listen for ControllerErrorEvents so that your program can respond to Controller malfunctions to minimize the impact on the user.

Figure 2-14: JMF events.

Processing

A Processor is a Player that takes a DataSource as input, performs some user-defined processing on the media data, and then outputs the processed media data.

Figure 2-15: JMF processors.

A Processor can send the output data to a presentation device or to a DataSource. If the data is sent to a DataSource, that DataSource can be used as the input to another Player or Processor, or as the input to a DataSink.

While the processing performed by a Player is predefined by the implementor, a Processor allows the application developer to define the type of processing that is applied to the media data. This enables the application of effects, mixing, and compositing in real-time.

The processing of the media data is split into several stages:

Figure 2-16: Processor stages.

• Demultiplexing is the process of parsing the input stream. If the stream contains multiple tracks, they are extracted and output separately. For example, a QuickTime file might be demultiplexed into separate audio and video tracks. Demultiplexing is performed automatically whenever the input stream contains multiplexed data.
• Pre-Processing is the process of applying effect algorithms to the tracks extracted from the input stream.
• Transcoding is the process of converting each track of media data from one input format to another. When a data stream is converted from a compressed type to an uncompressed type, it is generally referred to as decoding. Conversely, converting from an uncompressed type to a compressed type is referred to as encoding.
• Post-Processing is the process of applying effect algorithms to decoded tracks.
• Multiplexing is the process of interleaving the transcoded media tracks into a single output stream. For example, separate audio and video tracks might be multiplexed into a single MPEG-1 data stream. You can specify the data type of the output stream with the Processor setOutputContentDescriptor method.
• Rendering is the process of presenting the media to the user.

The processing at each stage is performed by a separate processing component. These processing components are JMF plug-ins. If the Processor supports TrackControls, you can select which plug-ins you want to use to process a particular track. There are five types of JMF plug-ins:

• Demultiplexer--parses media streams such as WAV, MPEG or QuickTime. If the stream is multiplexed, the separate tracks are extracted.
• Effect--performs special effects processing on a track of media data.
• Codec--performs data encoding and decoding.
• Multiplexer--combines multiple tracks of input data into a single interleaved output stream and delivers the resulting stream as an output DataSource.
• Renderer--processes the media data in a track and delivers it to a destination such as a screen or speaker.

Processor States

A Processor has two additional standby states, Configuring and Configured, which occur before the Processor enters the Realizing state..

Figure 2-17: Processor states.

• A Processor enters the Configuring state when configure is called. While the Processor is in the Configuring state, it connects to the DataSource, demultiplexes the input stream, and accesses information about the format of the input data.
• The Processor moves into the Configured state when it is connected to the DataSource and data format has been determined. When the Processor reaches the Configured state, a ConfigureCompleteEvent is posted.
• When Realize is called, the Processor is transitioned to the Realized state. Once the Processor is Realized it is fully constructed.

While a Processor is in the Configured state, getTrackControls can be called to get the TrackControl objects for the individual tracks in the media stream. These TrackControl objects enable you specify the media processing operations that you want the Processor to perform.

Calling realize directly on an Unrealized Processor automatically transitions it through the Configuring and Configured states to the Realized state. When you do this, you cannot configure the processing options through the TrackControls--the default Processor settings are used.

Calls to the TrackControl methods once the Processor is in the Realized state will typically fail, though some Processor implementations might support them.

Methods Available in Each Processor State

Since a Processor is a type of Player, the restrictions on when methods can be called on a Player also apply to Processors. Some of the Processor-specific methods also are restricted to particular states. The following table shows the restrictions that apply to a Processor. If you call a method that is illegal in the current state, the Processor throws an error or exception.

Method	Unrealized Processor	Configuring Processor	Configured Processor	Realized Processor
addController	NotRealizedError	NotRealizedError	NotRealizedError	legal
deallocate	legal	legal	legal	legal
getControlPanelComponent	NotRealizedError	NotRealizedError	NotRealizedError	legal
getControls	legal	legal	legal	legal
getDataOutput	NotRealizedError	NotRealizedError	NotRealizedError	legal
getGainControl	NotRealizedError	NotRealizedError	NotRealizedError	legal
getOutputContentDescriptor	NotConfiguredError	NotConfiguredError	legal	legal
getStartLatency	NotRealizedError	NotRealizedError	NotRealizedError	legal
getSupportedContent- Descriptors	legal	legal	legal	legal
getTimeBase	NotRealizedError	NotRealizedError	NotRealizedError	legal
getTrackControls	NotConfiguredError	NotConfiguredError	legal	FormatChange- Exception
getVisualComponent	NotRealizedError	NotRealizedError	NotRealizedError	legal
mapToTimeBase	ClockStoppedException	ClockStoppedException	ClockStoppedException	ClockStopped- Exception
realize	legal	legal	legal	legal
removeController	NotRealizedError	NotRealizedError	NotRealizedError	legal
setOutputContentDescriptor	NotConfiguredError	NotConfiguredError	legal	FormatChange- Exception
setMediaTime	NotRealizedError	NotRealizedError	NotRealizedError	legal
setRate	NotRealizedError	NotRealizedError	NotRealizedError	legal
setStopTime	NotRealizedError	NotRealizedError	NotRealizedError	legal
setTimeBase	NotRealizedError	NotRealizedError	NotRealizedError	legal
syncStart	NotPrefetchedError	NotPrefetchedError	NotPrefetchedError	NotPrefetchedError

Table 2-2: Method restrictions for processors.

Processing Controls

You can control what processing operations the Processor performs on a track through the TrackControl for that track. You call Processor getTrackControls to get the TrackControl objects for all of the tracks in the media stream.

Through a TrackControl, you can explicitly select the Effect, Codec, and Renderer plug-ins you want to use for the track. To find out what options are available, you can query the PlugInManager to find out what plug-ins are installed.

To control the transcoding that's performed on a track by a particular Codec, you can get the Controls associated with the track by calling the TrackControl getControls method. This method returns the codec controls available for the track, such as BitRateControl and QualityControl. (For more information about the codec controls defined by JMF, see Controls.)

If you know the output data format that you want, you can use the setFormat method to specify the Format and let the Processor choose an appropriate codec and renderer. Alternatively, you can specify the output format when the Processor is created by using a ProcessorModel. A ProcessorModel defines the input and output requirements for a Processor. When a ProcessorModel is passed to the appropriate Manager create method, the Manager does its best to create a Processor that meets the specified requirements.

Data Output

The getDataOutput method returns a Processor object's output as a DataSource. This DataSource can be used as the input to another Player or Processor or as the input to a data sink. (For more information about data sinks, see Media Data Storage and Transmission.)

A Processor object's output DataSource can be of any type: PushDataSource, PushBufferDataSource, PullDataSource, or PullBufferDataSource.

Not all Processor objects output data--a Processor can render the processed data instead of outputting the data to a DataSource. A Processor that renders the media data is essentially a configurable Player.

Capture

A multimedia capturing device can act as a source for multimedia data delivery. For example, a microphone can capture raw audio input or a digital video capture board might deliver digital video from a camera. Such capture devices are abstracted as DataSources. For example, a device that provides timely delivery of data can be represented as a PushDataSource. Any type of DataSource can be used as a capture DataSource: PushDataSource, PushBufferDataSource, PullDataSource, or PullBufferDataSource.

Some devices deliver multiple data streams--for example, an audio/video conferencing board might deliver both an audio and a video stream. The corresponding DataSource can contain multiple SourceStreams that map to the data streams provided by the device.

Media Data Storage and Transmission

A DataSink is used to read media data from a DataSource and render the media to some destination--generally a destination other than a presentation device. A particular DataSink might write data to a file, write data across the network, or function as an RTP broadcaster. (For more information about using a DataSink as an RTP broadcaster, see Transmitting RTP Data With a Data Sink.)

Like Players, DataSink objects are constructed through the Manager using a DataSource. A DataSink can use a StreamWriterControl to provide additional control over how data is written to a file. See Writing Media Data to a File for more information about how DataSink objects are used.

Storage Controls

A DataSink posts a DataSinkEvent to report on its status. A DataSinkEvent can be posted with a reason code, or the DataSink can post one of the following DataSinkEvent subtypes:

• DataSinkErrorEvent, which indicates that an error occurred while the DataSink was writing data.
• EndOfStreamEvent, which indicates that the entire stream has successfully been written.

To respond to events posted by a DataSink, you implement the DataSinkListener interface.

Extensibility

You can extend JMF by implementing custom plug-ins, media handlers, and data sources.

Implementing Plug-Ins

By implementing one of the JMF plug-in interfaces, you can directly access and manipulate the media data associated with a Processor:

• Implementing the Demultiplexer interface enables you to control how individual tracks are extracted from a multiplexed media stream.
• Implementing the Codec interface enables you to perform the processing required to decode compressed media data, convert media data from one format to another, and encode raw media data into a compressed format.
• Implementing the Effect interface enables you to perform custom processing on the media data.
• Implementing the Multiplexer interface enables you to specify how individual tracks are combined to form a single interleaved output stream for a Processor.
• Implementing the Renderer interface enables you to control how data is processed and rendered to an output device.

Note: The JMF Plug-In API is part of the official JMF API, but JMF Players and Processors are not required to support plug-ins. Plug-ins won't work with JMF 1.0-based Players and some Processor implementations might choose not to support them. The reference implementation of JMF 2.0 provided by Sun Microsystems, Inc. and IBM Corporation fully supports the plug-in API.

Custom Codec, Effect, and Renderer plug-ins are available to a Processor through the TrackControl interface. To make a plug-in available to a default Processor or a Processor created with a ProcessorModel, you need to register it with the PlugInManager. Once you've registered your plug-in, it is included in the list of plug-ins returned by the PlugInManager getPlugInList method and can be accessed by the Manager when it constructs a Processor object.

Implementing MediaHandlers and DataSources

If the JMF Plug-In API doesn't provide the degree of flexibility that you need, you can directly implement several of the key JMF interfaces: Controller, Player, Processor, DataSource, and DataSink. For example, you might want to implement a high-performance Player that is optimized to present a single media format or a Controller that manages a completely different type of time-based media.

The Manager mechanism used to construct Player, Processor, DataSource, and DataSink objects enables custom implementations of these JMF interfaces to be used seamlessly with JMF. When one of the create methods is called, the Manager uses a well-defined mechanism to locate and construct the requested object. Your custom class can be selected and constructed through this mechanism once you register a unique package prefix with the PackageManager and put your class in the appropriate place in the predefined package hierarchy.

MediaHandler Construction

Players, Processors, and DataSinks are all types of MediaHandlers--they all read data from a DataSource. A MediaHandler is always constructed for a particular DataSource, which can be either identified explicitly or with a MediaLocator. When one of the createMediaHandler methods is called, Manager uses the content-type name obtained from the DataSource to find and create an appropriate MediaHandler object.

Figure 2-18: JMF media handlers.

JMF also supports another type of MediaHandler, MediaProxy. A MediaProxy processes content from one DataSource to create another. Typically, a MediaProxy reads a text configuration file that contains all of the information needed to make a connection to a server and obtain media data. To create a Player from a MediaProxy, Manager:

1. Constructs a DataSource for the protocol described by the MediaLocator
2. Uses the content-type of the DataSource to construct a MediaProxy to read the configuration file.
3. Gets a new DataSource from the MediaProxy.
4. Uses the content-type of the new DataSource to construct a Player.

The mechanism that Manager uses to locate and instantiate an appropriate MediaHandler for a particular DataSource is basically the same for all types of MediaHandlers:

• Using the list of installed content package-prefixes retrieved from PackageManager, Manager generates a search list of available MediaHandler classes.
• Manager steps through each class in the search list until it finds a class named Handler that can be constructed and to which it can attach the DataSource.

When constructing Players and Processors, Manager generates the search list of available handler classes from the list of installed content package-prefixes and the content-type name of the DataSource. To search for Players, Manager looks for classes of the form:

    <content package-prefix>.media.content.<content-type>.Handler

To search for Processors, Manager looks for classes of the form:

    <content package-prefix>.media.processor.<content-type>.Handler

If the located MediaHandler is a MediaProxy, Manager gets a new DataSource from the MediaProxy and repeats the search process.

If no appropriate MediaHandler can be found, the search process is repeated, substituting unknown for the content-type name. The unknown content type is supported by generic Players that are capable of handling a large variety of media types, often in a platform-dependent way.

Because a DataSink renders the data it reads from its DataSource to an output destination, when a DataSink is created the destination must also be taken into account. When constructing DataSinks, Manager uses the list of content package-prefixes and the protocol from the MediaLocator that identifies the destination. For each content package-prefix, Manager adds to the search list a class name of the form:

 <content package-prefix>.media.datasink.protocol.Handler

If the located MediaHandler is a DataSink, Manager instantiates it, sets its DataSource and MediaLocator, and returns the resulting DataSink object. If the handler is a DataSinkProxy, Manager retrieves the content type of the proxy and generates a list of DataSink classes that support the protocol of the destination Medialocator and the content type returned by the proxy:

 <content package-prefix>.media.datasink.protocol.<content-type>.Handler

The process continues until an appropriate DataSink is located or the Manager has iterated through all of the content package-prefixes.

DataSource Construction

Manager uses the same mechanism to construct DataSources that it uses to construct MediaHandlers, except that it generates the search list of DataSource class names from the list of installed protocol package-prefixes.

For each protocol package-prefix, Manager adds to the search list a class name of the form:

    <protocol package-prefix>.media.protocol.<protocol>.DataSource 

Manager steps through each class in the list until it finds a DataSource that it can instantiate and to which it can attach the MediaLocator.