Print out one of the PDF markers in the NYArToolkit/Data. For my demo, i’m using pattHiro.pdf
Import all projects under NYArToolkit in your Eclipse workspace. You can ignore the ones for JOGL and QuickTime for this specific demo.
In these projects, tweak the build path to point to the JMF / Java3D jars on your system.
Plug in your web camera.
Run jp.nyatla.nyartoolkit.jmf.sample.NyarToolkitLinkTest class to test that JMF has been installed correctly and can display the captured feed.
Run jp.nyatla.nyartoolkit.java3d.sample.NyARJava3D class to test the tracking capabilities of the NYArToolkit. Once the camera feed is showing, point the camera to the printed marker so that it is fully visible. Once you have it on your screen, a colored cube should be shown. Try rotating, moving and tilting the marker printout – all the while keeping it in the frame – to verify that the cube is properly oriented.
The demo from the video above is available as a part of the new Marble project. Here is how to set it up:
Sync the latest SVN tip of Marble.
Create an Eclipse project for Marble. It should have dependencies on NYArToolkit, NYArToolkit.utils.jmf, NYArToolkit.utils.java3d and Trident. It should also have jmf.jar, vecmath.jar, j3dcore.jar and j3dutils.jar in the build path.
Run the org.pushingpixels.marble.MarbleFireworks3D class. Follow the same instructions as above to point the webcam.
There’s not much to the code in this Marble demo. It follows the NyARJava3D class, but instead of static Java3D content (color cube) it has a dynamic scene that is animated by Trident. For the code below note that i’m definitely not an expert in Java3D and NYArToolkit, so there might as well be a simpler way to do these animations. However, they are enough to get you started in exploring animations in Java-powered augmented reality.
Each explosion volley is implemented by a collection of Explosion3D objects. Each such object models a single explosion “particle”. Here is the constructor of the Explosion3D class:
public Explosion3D(float x, float y, float z, Color color) {
this.x = x;
this.y = y;
this.z = z;
this.color = color;
this.alpha = 1.0f;
this.sphere3DTransformGroup = new TransformGroup();
this.sphere3DTransformGroup
.setCapability(TransformGroup.ALLOW_TRANSFORM_WRITE);
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
this.appearance3D = new Appearance();
this.appearance3D
.setCapability(Appearance.ALLOW_TRANSPARENCY_ATTRIBUTES_WRITE);
this.appearance3D
.setCapability(Appearance.ALLOW_COLORING_ATTRIBUTES_WRITE);
this.appearance3D.setColoringAttributes(new ColoringAttributes(color
.getRed() / 255.0f, color.getGreen() / 255.0f,
color.getBlue() / 255.0f, ColoringAttributes.SHADE_FLAT));
this.appearance3D.setTransparencyAttributes(new TransparencyAttributes(
TransparencyAttributes.BLENDED, 0.0f));
this.sphere3D = new Sphere(0.002f, appearance3D);
this.sphere3DTransformGroup.addChild(this.sphere3D);
this.sphere3DBranchGroup = new BranchGroup();
this.sphere3DBranchGroup.setCapability(BranchGroup.ALLOW_DETACH);
this.sphere3DBranchGroup.addChild(this.sphere3DTransformGroup);
}
Here, we have a bunch of Java3D code to create a group that can be dynamically changed at runtime. This group has only one leaf – the Sphere object.
As we’ll see later, the timeline behind this object changes its coordinates and the alpha (fading it out). Here is the relevant public setter for the alpha property:
Not much here – just updating the transparency of the underlying Java3D Sphere object. The setters for the coordinates are quite similar:
public void setX(float x) {
this.x = x;
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
}
public void setY(float y) {
this.y = y;
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
}
public void setZ(float z) {
this.z = z;
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
}
The main class is MarbleFireworks3D. Its constructor is rather lengthy and has a few major parts. The first part initializes the camera and marker data for the NYArToolkit core:
NyARCode ar_code = new NyARCode(16, 16);
ar_code.loadARPattFromFile(CARCODE_FILE);
ar_param = new J3dNyARParam();
ar_param.loadARParamFromFile(PARAM_FILE);
ar_param.changeScreenSize(WIDTH, HEIGHT);
Following that, there’s a bunch of Java3D code that initializes the universe, locale, platform, body and environment, and creates the main transformation group. The interesting code is the one that creates the main scene group that will hold the dynamic collection of Explosion3D groups:
mainSceneGroup = new TransformGroup();
mainSceneGroup.setCapability(TransformGroup.ALLOW_TRANSFORM_WRITE);
mainSceneGroup.setCapability(Group.ALLOW_CHILDREN_EXTEND);
mainSceneGroup.setCapability(Group.ALLOW_CHILDREN_WRITE);
root.addChild(mainSceneGroup);
nya_behavior = new NyARSingleMarkerBehaviorHolder(ar_param, 30f,
ar_code, 0.08);
nya_behavior.setTransformGroup(mainSceneGroup);
nya_behavior.setBackGround(background);
root.addChild(nya_behavior.getBehavior());
nya_behavior.setUpdateListener(this);
locale.addBranchGraph(root);
The NyARSingleMarkerBehaviorHolder is a helper class from the NYArToolkit.utils.java3d project. It tracks the transformation matrix computed by NYArToolkit based on the current location of the marker and updates the transformation set on the main scene group. As you will see later, there is no explicit handling of the marker tracking in the demo code – only creation, update and deletion of the Explosion3D objects.
Finally, we create a looping thread that adds random firework explosions:
// start adding random explosions
new Thread() {
@Override
public void run() {
while (true) {
try {
Thread.sleep(500 + (int) (Math.random() * 1000));
float x = -1.0f + 2.0f * (float) Math.random();
float y = -1.0f + 2.0f * (float) Math.random();
float z = (float) Math.random();
addFireworkNew(x * 5.0f, y * 5.0f, 5.0f + z * 12.0f);
} catch (Exception exc) {
}
}
}
}.start();
The code in this method computes a 3D uniform distribution of small spheres that originate at the specific location (explosion center) and move outwards. Each Explosion3D object is animated with the matching timeline. The timeline interpolates the alpha property, as well as the coordinates. As you can see, while x and y are interpolated linearly, the interpolation of z takes the gravity into the account – making the explosion particles fall downwards. All the timelines are added to a parallel timeline scenario. Once a timeline starts playing, the matching branch group is added to the main scene graph. Once the timeline scenario is done, all the branch groups are removed from the main scene graph:
private void addFireworkNew(float x, final float y, final float z) {
final TimelineScenario scenario = new TimelineScenario.Parallel();
Set scenarioExplosions = new HashSet();
float R = 6;
int NUMBER = 20;
int r = (int) (255 * Math.random());
int g = (int) (100 + 155 * Math.random());
int b = (int) (50 + 205 * Math.random());
Color color = new Color(r, g, b);
for (double alpha = -Math.PI / 2; alpha <= Math.PI / 2; alpha += 2
* Math.PI / NUMBER) {
final float dy = (float) (R * Math.sin(alpha));
final float yFinal = y + dy;
float rSection = (float) Math.abs(R * Math.cos(alpha));
int expCount = Math.max(0, (int) (NUMBER * rSection / R));
for (int i = 0; i < expCount; i++) {
float xFinal = (float) (x + rSection
* Math.cos(i * 2.0 * Math.PI / expCount));
final float dz = (float)(rSection
* Math.sin(i * 2.0 * Math.PI / expCount));
float zFinal = z + dz;
final Explosion3D explosion = new Explosion3D(x * SCALE, y * SCALE,
z * SCALE, color);
scenarioExplosions.add(explosion);
final Timeline expTimeline = new Timeline(explosion);
expTimeline.addPropertyToInterpolate("alpha", 1.0f, 0.0f);
expTimeline.addPropertyToInterpolate("x", x * SCALE, xFinal
* SCALE);
expTimeline.addPropertyToInterpolate("y", y * SCALE, yFinal
* SCALE);
expTimeline.addCallback(new TimelineCallbackAdapter() {
@Override
public void onTimelinePulse(float durationFraction,
float timelinePosition) {
float t = expTimeline.getTimelinePosition();
float zCurr = (z + dz * t - 10 * t * t) * SCALE;
explosion.setZ(zCurr);
}
});
expTimeline.setDuration(3000);
expTimeline.addCallback(new TimelineCallbackAdapter() {
@Override
public void onTimelineStateChanged(TimelineState oldState,
TimelineState newState, float durationFraction,
float timelinePosition) {
if (newState == TimelineState.PLAYING_FORWARD) {
mainSceneGroup.addChild(explosion
.getSphere3DBranchGroup());
}
}
});
scenario.addScenarioActor(expTimeline);
}
}
synchronized (explosions) {
explosions.put(scenario, scenarioExplosions);
}
scenario.addCallback(new TimelineScenarioCallback() {
@Override
public void onTimelineScenarioDone() {
synchronized (explosions) {
Set ended = explosions.remove(scenario);
for (Explosion3D end : ended) {
mainSceneGroup
.removeChild(end.getSphere3DBranchGroup());
}
}
}
});
scenario.play();
}
The dependent libraries that are used here do the following:
JMF captures the webcam stream and provides the pixels to put on the screen
NYArToolkit processes the webcam stream, locates the marker and computes the matching transformation matrix
Java3D tracks the scene graph objects, handles the depth sorting and the painting of the scene
Trident animates the location and alpha of the explosion particles
Release 5.2 of Substance look-and-feel made a few visual changes to the Raven Graphite skins, but these did not address the overall usability of these skins – especially the contrast between the background and the controls, and the background / foreground contrast of text components.
The latest drop of version 5.3dev (code named Reykjavik) features significant overhaul of both Raven Graphite skins, aiming to address the contrast usability issues raised by the users.
Here is a screenshot of a sample application under the Raven Graphite skin in the latest stable 5.2 release:
And here is the same application under the 5.3dev drop:
Here is another screenshot of the same application under the old Raven Graphite visuals:
and the new visuals under the latest 5.3dev drop:
The main changes are:
Removing the watermark that contributed significant visual noise
Darker border color for controls, bringing more delineation to check boxes and radio buttons
Darker background color for text components, resulting in better readability
The same changes were made for the Raven Graphite Glass skin. Here is the sample application under the stable 5.2 release:
and here is the same application under the latest 5.3dev drop:
In addition to the visual changes above, the Raven Graphite Glass skin has removed the glass arc gradient from the toolbars and added a two-tone separator to delineate the title bar / menu bar from the rest of the application content.
To illustrate the visual difference in a larger content, here is a screenshot of a big UI under the stable 5.2 release (click to see the full-size view):
and the same application under the 5.3dev branch:
If you want to take the new visuals for a spin, click on the WebStart button below and change the skin to Raven Graphite and Raven Graphite Glass from the “Skins” menu:
You’re more than welcome to take the latest 5.3dev drop for a spin and leave your comments.
Concluding the series on adding animations to enable rich interactivity expected from modern Swing applications, here is what we have seen so far:
Part 1 – adding simple animation behavior to such scenarios as component appearance (fade in), rollovers and window disposal (fade-out) using built in and custom class attributes and setters.
Part 2 – adding animated load progress indication while the application is loading data.
Part 3 – loading the album art matching the specific search string and asynchronously displaying the associated images.
Part 4 – scrolling the album covers showed in the container and adding animations to the scrolling.
Run the org.pushingpixels.onyx.DemoApp class, passing your Amazon key as the only parameter to this class, adding the Amazon, Trident and Onyx classes to the classpath
If all went right, you should see the main application running and displaying Sarah McLachlan albums as in this video:
I hope you enjoyed this series. If you’re interested in adding rich animations to your Swing applications, you’re more than welcome to explore Trident and Onyx and report any bugs and missing features in the project forums and mailing lists.
After adding such animation effects as fading, translucency, load progress, asynchronous load of images and smooth scrolling in the application window connected to the Amazon backend, it’s time to talk about more complex transition scenarios. In this entry i’m going to talk about displaying larger album art and scrollable track listing when the specific album is selected, along with a complex transition between selected albums. This code is part of the Onyx project which aims to provide blueprints for animated Swing applications powered by the Trident animation library.
Here is a screenshot that illustrates the detailed view of the selected album (and you can view the videos in the first part of this series):
The full sources of this view are in the SVN repository, and i’m going to talk about the full transition scenario that is played when the user selects a specific album. This scenario has six steps:
(Relevant when the details window already shows album art) – collapse the album art component and track listing component to fully overlap.
(In parallel with step 1) – load the new album art from the Internet (based on the URL returned from the original Amazon E-commerce request).
(After steps 1 and 2 have both completed) – set the loaded album art on the album art component. This may also cause resizing the album art if it cannot fully fit in the available space.
(In parallel with step 3) – set the list of album tracks on the track listing component.
(After steps 3 and 4 have both completed) – cross fade the old album art to the new album art.
(After step 5 has been completed) – move the album art component (that displays the new album art) and the track listing to be displayed side by side.
To implement this complex timeline scenario, the code uses the rendezvous timeline scenario provided by Trident. Timeline.RendezvousSequence allows simple branch-and-wait ordering. The rendezvous scenario has a stage-like approach. All actors belonging to the same stage run in parallel, while actors in stage N+1 wait for all actors in stage N to be finished. The RendezvousSequence.rendezvous marks the end of one stage and the beginning of another.
Here is how the code looks like:
/**
* Returns the timeline scenario that implements a transition from the
* currently shown album item (which may be null) to the
* specified album item.
*
* @param albumItem
* The new album item to be shown in this window.
* @return The timeline scenario that implements a transition from the
* currently shown album item (which may be null) to
* the specified album item.
*/
private TimelineScenario getShowAlbumDetailsScenario(final Item albumItem) {
TimelineScenario.RendezvousSequence scenario = new TimelineScenario.RendezvousSequence();
// step 1 - move album art and track listing to the same location
Timeline collapseArtAndTracks = new Timeline(this);
collapseArtAndTracks.addPropertyToInterpolate("overlayPosition",
this.overlayPosition, 0.0f);
collapseArtAndTracks.setDuration((int) (500 * this.overlayPosition));
scenario.addScenarioActor(collapseArtAndTracks);
// step 2 (in parallel) - load the new album art
final BufferedImage[] albumArtHolder = new BufferedImage[1];
TimelineSwingWorker loadNewAlbumArt = new TimelineSwingWorker() {
@Override
protected Void doInBackground() throws Exception {
URL url = new URL(albumItem.getLargeImage().getURL());
albumArtHolder[0] = ImageIO.read(url);
return null;
}
};
scenario.addScenarioActor(loadNewAlbumArt);
scenario.rendezvous();
// step 3 (wait for steps 1 and 2) - replace album art
TimelineRunnable replaceAlbumArt = new TimelineRunnable() {
@Override
public void run() {
albumArt.setAlbumArtImage(albumArtHolder[0]);
}
};
scenario.addScenarioActor(replaceAlbumArt);
// step 4 (in parallel) - replace the track listing
TimelineRunnable replaceTrackListing = new TimelineRunnable() {
@Override
public void run() {
trackListingScroller.setAlbumItem(albumItem);
}
};
scenario.addScenarioActor(replaceTrackListing);
scenario.rendezvous();
// step 5 (wait for steps 3 and 4) - cross fade album art from old to
// new
Timeline albumArtCrossfadeTimeline = new Timeline(this.albumArt);
albumArtCrossfadeTimeline.addPropertyToInterpolate("oldImageAlpha",
1.0f, 0.0f);
albumArtCrossfadeTimeline.addPropertyToInterpolate("imageAlpha", 0.0f,
1.0f);
albumArtCrossfadeTimeline.addCallback(new Repaint(this.albumArt));
albumArtCrossfadeTimeline.setDuration(400);
scenario.addScenarioActor(albumArtCrossfadeTimeline);
scenario.rendezvous();
// step 6 (wait for step 5) - move new album art and track listing to
// be side by side.
Timeline separateArtAndTracks = new Timeline(this);
separateArtAndTracks.addPropertyToInterpolate("overlayPosition", 0.0f,
1.0f);
separateArtAndTracks.setDuration(500);
scenario.addScenarioActor(separateArtAndTracks);
return scenario;
}
This scenario uses the full capabilities offered by the Trident timeline scenarios which allow combining multiple timeline scenario actors in a parallel, sequential or custom order. There are three core types of timeline scenario actors, all used in this code:
To create a custom timeline scenario, use the following APIs of the TimelineScenario class:
public void addScenarioActor(TimelineScenarioActor actor) adds the specified actor
public void addDependency(TimelineScenarioActor actor, TimelineScenarioActor... waitFor) specifies the dependencies between the actors
The rest of the code is pretty straightforward. It defines the components for album art and track listing, as well as the float position of the overlay between them (during the collapse / expand steps):
/**
* Component that shows the album art.
*/
private BigAlbumArt albumArt;
/**
* Component that shows the scrollable list of album tracks.
*/
private TrackListingScroller trackListingScroller;
/**
* 0.0f - the album art and track listing are completely overlayed, 1.0f -
* the album art and track listing are completely separate. Is updated in
* the {@link #currentShowAlbumDetailsScenario}.
*/
private float overlayPosition;
When these components are added, we make sure that the album art is displayed on top of the track listing (during the collapse stage). In addition, we install a custom layout manager that respects the current value of the overlayPosition field:
Container contentPane = this.getContentPane();
contentPane.setLayout(new LayoutManager() {
@Override
public void addLayoutComponent(String name, Component comp) {
}
@Override
public void removeLayoutComponent(Component comp) {
}
@Override
public Dimension minimumLayoutSize(Container parent) {
return null;
}
@Override
public Dimension preferredLayoutSize(Container parent) {
return null;
}
@Override
public void layoutContainer(Container parent) {
int w = parent.getWidth();
int h = parent.getHeight();
// respect the current overlay position to implement the sliding
// effect in steps 1 and 6 of currentShowAlbumDetailsScenario
int dim = BigAlbumArt.TOTAL_DIM;
int dx = (int) (overlayPosition * dim / 2);
albumArt.setBounds((w - dim) / 2 - dx, (h - dim) / 2, dim, dim);
trackListingScroller.setBounds((w - dim) / 2 + dx,
(h - dim) / 2 + 2, dim, dim - 4);
}
});
contentPane.add(albumArt);
contentPane.add(trackListingScroller);
contentPane.setComponentZOrder(trackListingScroller, 1);
contentPane.setComponentZOrder(albumArt, 0);
The overlayPosition is changed in steps 1 and 6 of the main transition scenario, and the public setter revalidates the container causing the layout:
/**
* Sets the new overlay position of the album art and track listing. This
* method will also cause revalidation of the main window content pane.
*
* @param overlayPosition
* The new overlay position of the album art and track listing.
*/
public void setOverlayPosition(float overlayPosition) {
this.overlayPosition = overlayPosition;
this.getContentPane().invalidate();
this.getContentPane().validate();
}
Finally, the scenario itself is created and played when the mouse listener installed on the album overview component detects a mouse click and calls the setAlbumItem API:
/**
* Signals that details of the specified album item should be displayed in
* this window. Note that this window can already display another album item
* when this method is called.
*
* @param albumItem
* New album item to show in this window.
*/
public void setAlbumItem(Item albumItem) {
if (this.currentShowAlbumDetailsScenario != null)
this.currentShowAlbumDetailsScenario.cancel();
this.currentShowAlbumDetailsScenario = this
.getShowAlbumDetailsScenario(albumItem);
this.currentShowAlbumDetailsScenario.play();
}
Note how we first cancel the currently playing scenario – this handles quick subsequent selections by the user, reversing the currently playing scenario in the middle.
One last thing to note in the transition scenario:
// step 5 (wait for steps 3 and 4) - cross fade album art from old to
// new
Timeline albumArtCrossfadeTimeline = new Timeline(this.albumArt);
albumArtCrossfadeTimeline.addPropertyToInterpolate("oldImageAlpha",
1.0f, 0.0f);
albumArtCrossfadeTimeline.addPropertyToInterpolate("imageAlpha", 0.0f,
1.0f);
albumArtCrossfadeTimeline.addCallback(new Repaint(this.albumArt));
albumArtCrossfadeTimeline.setDuration(400);
Note that this timeline is created on the child album art component. After the new album art has been loaded and scaled (in step 3), we initiate the cross-fading timeline on another object – which is fully supported by Trident timelines.
The rest of the code in this package is very similar to the code examples showed earlier, including custom painting that respects the alpha values, fading out on dispose, translucent window etc.
The final code sample shows how the album details panel is shown. Here, we use a separate translucent Window placed alongside the bottom edge of the main application window:
currentlyShownWindow = new DetailsWindow();
// place the details window centered along the bottom edge of the
// main application window
Point mainWindowLoc = mainWindow.getLocation();
Dimension mainWindowDim = mainWindow.getSize();
int x = mainWindowLoc.x + mainWindowDim.width / 2
- currentlyShownWindow.getWidth() / 2;
int y = mainWindowLoc.y + mainWindowDim.height
- currentlyShownWindow.getHeight() / 2;
currentlyShownWindow.setLocation(x, y);
currentlyShownWindow.setOpacity(0.0f);
currentlyShownWindow.setBackground(new Color(0, 0, 0, 0));
currentlyShownWindow.setVisible(true);
currentlyShownWindow.setAlbumItem(albumItem);
Timeline showWindow = new Timeline(currentlyShownWindow);
showWindow.addPropertyToInterpolate("opacity", 0.0f, 1.0f);
showWindow.setDuration(500);
showWindow.play();
What happens here?
Create a new window and position it in the required location.
Set its opacity to 0.0 (it will be gradually faded in).
Set its background to a fully transparent color – allowing the collapse / expand stage to show the underlying window.
Sets the album item, initiating the transition scenario described above.
Creates and plays the timeline that fades in this window.
