1.00/1.

001 Intro To

Computation and Engineering

Problem Solving

Fall 2002

Media in Java

Topics
The AWT, Java 2D API and the newer Java
Media Framework provide tools for working
with diverse media.
structure of digital media
the AWT Image class and its related methods
the MediaTracker class
the Java Media Framework structure
use of the JMF for video and audio playing
media synchronization

1
 Digital Media
Includes text, video, audio, animation, images
and vector graphics
video and audio have temporal dimension
files are often large and compressed
diverse file formats
latency and network throughput are often
issues

The AWT and JMF

The AWT supports text, limited drawn graphics and
text.
Java 1.2 provides much improved graphics and text
support, and additional image processing features
the Java Media Framework is an additional package
that provides support for temporal media-audio and
video
JMF features include synchronization and
implementation using native platform features

2
 Images in the AWT
Image class supports the "push model". Images are
"pushed" to an ImageConsumer. Works well for
applets and other Web applications.
A digital image (once loaded) is a rectangular
collection of pixels, where each pixel represents a
color
Java AWT has an Image class for loading and
displaying images
AWT can load image from file or URL over the Web
The Java 2D API has BufferedImage class that has
much wider range of capabilities

Image in an Applet
Simplest case is loading an image into an
application
construct an Image object with code such as:
Image im = new Image(myImageFile.gif);
display image in the paintComponent()
method of Component object using
drawImage() method:
boolean b drawImage(Image im, int x, int y,
ImageObserver iob)
returns true if Image is fully loaded, false otherwise

3
 ImageObserver interface
Images take a long time to load, and
may load incrementally over time
ImageObserver is an interface that has
methods for handling notification of
state of image loading. It can use this
for redisplay as needed.
JFrame and Applet both implement
ImageObserver interface

Drawing an Image
import java.awt.*;

import javax.swing.*;

public class ImagePanel extends JPanel {

private Image im;

public ImagePanel(Image i) {

im = i; }

public void paintComponent(Graphics g) {

super.paintComponent(g);

if(im == null)

g.drawString("Image object null", 30,30);

else
g.drawImage(im,10,10,this); }
} // end of ImagePanel class

4
Other Image loading methods
public Image getImage(URL u)
loads image from Web from URL u. URL is a
class in java.net package. Its simplest
constructor is
URL u = new URL(String s)
public Image getImage(URL u, String name)-u
is the base location on a Web server; name is
the file name

MediaTracker class
Sometimes you want to (or have to wait) for
image to load fully before proceeding
MediaTracker class allows you to register
images and wait until they are fully loaded
supports waiting for groups of images
you register images with MediaTracker object
MediaTracker can establish priorities for
image loading

5
 MediaTracker Constructor and Methods

MediaTracker(Component m) - m is the object where

the images are to be drawn
void addImage(Image im , int id)-im is the image to be
tracked, id is the group of objects with same id
void waitForID(int id) throws InterruptedException
void waitForAll() throws InterruptedException
void waitForID(int id, long ms) throws
InterruptedException
void waitForAll(long ms) throws InterruptedException
boolean checkID(int id)
boolean checkAll()

MediaTracker Use
MediaTracker iTrack = new MediaTracker(this);

Image tImage = new Image(cecif2.jpg);

iTrack.addImage(tImage, 1);

try {

iTrack.waitForID(1);

catch(InterruptedException e) {

System.err.println(Error in Image loading);

6
 import java.awt.*;

import javax.swing.*;

public class ImageDisplay extends JFrame {

private Image testImage;

private ImagePanel iPanel;

public static void main(String[] args) {

new ImageDisplay(args[0]); }

public ImageDisplay(String imageFile) {

setTitle("Simple Image Displayer");

MediaTracker iTrack = new MediaTracker(this);

testImage = Toolkit.getDefaultToolkit().getImage(imageFile);

iTrack.addImage(testImage, 1);

try {

iTrack.waitForID(1); }
catch(InterruptedException e) {
System.out.println("Exception in loading image(s).");
System.exit(0);
}

iPanel = new ImagePanel(testImage);

getContentPane().add(iPanel);
setSize(testImage.getWidth(this)+50, testImage.getHeight(this)+50);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setVisible(true);
}
} // end of ImageDisplay

7
 Java 2D API
Java 2D API provide "immediate mode"
model for images. the BufferedImage class is
central to this.
BufferedImage class allows much greater
programmatic control than AWT's Image
class
BufferedImage supports affine
transformations, image filtering, multiple color
models, compositing, multiple compression
methods

import java.awt.*;

import javax.swing.*;

import java.io.*;

import com.sun.image.codec.jpeg.*; // JPEG Decoder

public class BufferedImageDisplay extends JFrame {

private Image testImage;

private ImagePanel iPanel;

public static void main(String[] args)

{
new BufferedImageDisplay(args[0]);
}

8
public BufferedImageDisplay(String imageFile) {
setTitle("Simple Image Displayer");
try {
InputStream inStream = new FileInputStream(imageFile);
JPEGImageDecoder de =
JPEGCodec.createJPEGDecoder(inStream);
testImage = de.decodeAsBufferedImage();
}
catch(IOException e) {
System.out.println("Exception in loading image "+imageFile);
System.exit(0); }
iPanel = new ImagePanel(testImage);
getContentPane().add(iPanel);
setSize(testImage.getWidth(this)+50, testImage.getHeight(this)+50);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setVisible(true);
}
} // end of ImageDisplay

Inside BufferedImage Class

BufferedImage object has two major
parts:
Raster-storage for pixels, each
representing small region of image
ColorModel-interpretation of pixel value
(maps pixel contents to color components)
BufferedImage class provides wide
range of options for packing pixels,
color models, indexed colors

9
 Common Color Spaces
RGB-Red, Green Blue, often with an
Alpha component for opacity
CMYK Cyan, Magenta, Yellow and
Black
GrayScale

There are usually measured with respect

to standard references for color

Buffered Operations
There are standard filters for operating on
BufferedImages that implement the
BufferedImageOp interface
Basic coding pattern:
create source BufferedImage object
Use a "factory method" to create an appropriate
converter that implements BufferedImageOp
interface
run converter object's filter method, as in
BufferedImage result = converter.filter(source, null); // null
can be the destination image

10
 Image Sharpening Example

import java.awt.*;

import java.awt.image.*;

import javax.swing.*;

import java.io.*;

import com.sun.image.codec.jpeg.*; // JPEG Decoder

public class SharpenOp extends JFrame {

private BufferedImage testImage;
private ImagePanel iPanel;
private BufferedImage convImage;

public static void main(String[] args)

{
new SharpenOp(args[0]);
}

public SharpenOp(String imageFile) {

setTitle("Sharpen Image Displayer");
try {
InputStream inStream = new
FileInputStream(imageFile);
JPEGImageDecoder de =
JPEGCodec.createJPEGDecoder(inStream);
testImage = de.decodeAsBufferedImage();
}
catch(IOException e) {
System.out.println("Exception in loading image
"+imageFile);
System.exit(0);
}

11
 float kern[] = {-1f, -1f, -1f, -1f, 9f, -1f,
-1f, -1f, -1f};
Kernel sKernel = new Kernel(3,3,kern);
ConvolveOp sharp = new ConvolveOp(sKernel);

convImage = sharp.filter(testImage, null);

iPanel = new ImagePanel(convImage);

getContentPane().add(iPanel);
setSize(convImage.getWidth(this)+50,
convImage.getHeight(this)+50);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setVisible(true);

}
} // end of SharpenOp

Java Media Framework Goals

make Java applications using media portable
relieve programmer of details of diverse media types
provide means for using native decompression codes
in Java applications
tools for synchronizing two or more temporal media
allow Java programmer to treat different media types
uniformly independent of format and network
provide useful defaults for media players and still
allow customization when desired

12
 JMF Concepts
DataSource - source of media content--file
server, etc.
Two types are PushDataSource and
PullDataSource
Example
URL mURL = new URL(http://www-
ceci.mit.edu/seg6.mpg);
DataSource mySource = new DataSource(mURL)

Player Interface
Player-an interface for a player of a media
type. Classes meeting this interface will
typically involve native code. Implementations
of Player can be plug ins.
Concept is to allow many different groups
build Player implementations.

13
 Manager class
A factory that can create a player
appropriate to particular DataSource.
Manager can locate Player classes if
available and provide them to applications
public static Player createPlayer (DataSource source)
throws IOException, NoPlayerException
public static Player createPlayer(URL sourceURL)
throws IOException, NoPlayerException

Sample Manager use

// assume mediaURL is a valid URL object
try {
player = Manager.createPlayer(mediaURL);
}
catch(NoPlayerException e) {
System.err.println("No player for URL");
return;
}
catch(IOException e) {
System.err.println("IO Exception encountered");
return;
}

14
 Player States
unrealized - Player has been created but doesn't
know anything about its media except the URL
realizing - Player is in process of acquiring whatever
resouces it needs
realized - Player finished realizing and knows about
its resources and media. It now knows how to display
itself.
prefetching - Player preparing to present media
prefetched - Player done preparing to present media
started - clock of Player is running

State Diagram for Player

Source: JavaTM Media Player White Paper, Sun web site

15
 Player Components
public Component getControlPanelComponent() -
returns a panel with controls for media
(start,stop,etc.)
public Component getVisualComponent()-returns a
Component in which media can be displayed
public Component getGainComponent() - returns a
Component with audio volume controls

16
 Player Events

Player can generate a large number of

different Event types specific to JMF
ControllerEvent is most useful
Player can register ControllerListener objects
for notification. Uses AWT event model
ControllerListener interface must have
controllerUpdate(ControllerEvent e) methods

JMF Event Types

17
 Sample JMF Usage
Create Frame for video player to display in
create URL for video file
Use Manager class to create appropriate
Player for media
handle event where Player is fully realized,
and add visual and control Component of
Player to Frame
start the Player by calling player.start()
method

import java.awt.*; // import Java AWT package

import java.awt.event.*; // import Java event package

import java.net.*; // import network package

import java.io.*; // import Java IO package

import javax.media.*; // import JMF package

public class VideoPlayer implements ControllerListener

Player player = null;

Component visualComponent, controlComponent;

Container m_container;

public static void main (String args[]) {

String mediaFile = args[args.length-1];

Frame fr = new DestroyableFrame("Simple Video Player");

fr.setSize(428, 398);

fr.setVisible(true);

VideoPlayer player = new VideoPlayer(mediaFile, fr);

} // end of main method for VideoPlayer class

18
 // constructor method for VideoPlayer class
public VideoPlayer(String mediaFile, Container container) {
URL mediaURL = null;
m_container = container;
m_container.setLayout(new BorderLayout());
try {
mediaURL = new URL(mediaFile);

player = Manager.createPlayer(mediaURL);

player.addControllerListener(this);

player.realize();

catch (MalformedURLException e) {

System.err.println("Invalid media file URL!");

return; }

catch(NoPlayerException e) {

System.err.println("Unable to find a player for "+mediaURL);

return; }

catch(IOException e) {

System.err.println(IO Exception encountered");

return; }

} // end of VideoPlayer constructor method

// When player is realized, get its visual and control Components and
// put them into the container and start() the complete player.
public void controllerUpdate(ControllerEvent event)
{
if(event instanceof RealizeCompleteEvent){
if((visualComponent = player.getVisualComponent()) != null)
m_container.add("Center", visualComponent);
if ((controlComponent = player.getControlPanelComponent())
!= null)
m_container.add("South",controlComponent);
m_container.validate(); // ensure layout is up to date
m_container.repaint(); // ensure that anything drawn painted
player.start(); // start the player
}
else if (event instanceof ControllerErrorEvent){
System.err.println("*** ControllerErrorEvent *** " +
((ControllerErrorEvent)event).getMessage());
}
} // end of controllerUpdate method
} // end of VideoPlayer class

Java is a trademark or registered trademark of Sun Microsystems, Inc. in the United

States and other countries.

19