A prolific blogger, a frequent presenter at conferences and a tireless advocate of Groovy. Meet Andres Almiray as the interview series on Swing, RIA, JavaFX and related topics that started with Amy Fowler, Mikael Grev, David Qiao and Chet Haase continues.
Tell us a little bit about yourself. 
I’m a software geek who prefers Java/Groovy to do his work, I’ve been hooked into computer graphics since college (being able to use both SGI workstations and PowerMacs helped a lot), which also explains why I picked Java since the early beginnings. But life has a way to move you to places you didn’t expected at first, so I have my share of years behind server development and web applications (PHP and Perl included, yikes!), I even managed to get some ActionScript somewhere in between. These days I’m back to desktop, my day job being mostly related to Swing development at Oracle, my free time devoted to Groovy related projects like GraphicsBuilder (a Java2D builder) that complements SwingBuilder, and other open source related initiatives.
What do you think about Sun’s new direction of client Java towards RIA space with JavaFX?
I think it is about time Sun really cared about the desktop, I’ve been waiting for something like this since Swing came out. The good news is that some AWT/Swing APIs will be updated, as we have seen lately non-rectangular, translucent windows are finally available, while at the same time new APIs have been added like project SceneGraph and JWebPane, smaller JRE and faster applets too. I gladly welcome all those upgrades. But I don’t buy it on JavaFX Script just yet, I have to admit I’m biased on this one being a Groovy developer, as almost everything that JavaFX Script offers I can do with Groovy, the only thing not possible (today) is binding to any expression, which I consider the sole benefit to switch over JavaFX Script, it is really that powerful. But who knows, nothing is set on stone, I might code real applications with it at some point in the future.
Over the last year we have seen a growing number of projects using Swing as a backbone for new UI scripting languages. This includes Groovy, Scala, Jython, JRuby and, of course, JavaFX. Should this be the future of Swing – relegated to be just a foundation layer?
Let’s face it, compared to those other languages Swing in plain Java is too verbose, which is perhaps why so many developers would rather tango with visual tools and code generators in the first place. I believe in polyglot programming, thankfully I’m not restricted to code in just one language to rule over all my applications. Swing as a foundation layer makes sense, but considering that some of those languages provide a way to enhance existing Java classes (even in runtime) you may bend the foundation to your needs.
What are Swing’s weak points, and how are they addressed in Groovy’s SwingBuilder?
Besides it being verbose because of the Java language itself, I would say threading, it is so easy to shoot yourself in the foot with threading issues, you have to constantly remind yourself in which thread your code will be executed. Groovy’s SwingBuilder addresses the first point with Groovy itself. With native syntax for List and Maps, property access on beans, safe reference navigation and GPath, closures and metaprogramming, you have a full set of new tools to code less while retaining the same behavior. The second one is explicit to the builder as it exposes a set of methods that deal with threading in a very simple way. Say you write an ActionListener, as you now all listeners are called on the EDT, in this case the handling code will perform a long task and update the ui when it finishes. The usual solution in plain Java would be to call SwingUtilities.[invokeLater|invokeAndWait] with a Runnable as parameter, which means you have to create a new class, even if it is an inner anonymous one. With SwingBuilder you could do something like this
Do you feel constrained by the internals of Swing?
I consider myself a regular Swing developer, so far I’ve never had the need to meddle with a RepaintManager nor other scary/gory Swing internals, I leave that to the experts like you.
Both JavaFX and Groovy’s SwingBuilder provide access to Swing components and full APIs. Is this a desired feature for migrating existing applications, or perhaps this access should be restricted to enforce a cleaner design and not add yet another way on top of all that Swing carries with it over the past ten years?
There is a slight difference, JavaFX Script relies on Swing wrappers, those wrappers give JavaFX Script better control on Swing components. On the other hand Groovy’s SwingBuilder uses Swing classes as is. Groovy is able to dynamically enhance (both at compile time and runtime) said Swing classes, if needed be, thanks to its metaprogramming facilities. In the end JavaFX has a thin layer of abstraction that may be used as an extension point (in the future) without needing to actually change any Swing class, while Groovy itself can be used at any time to extend those classes. Back on your question both approaches already add something on top of Swing in some way but they clearly do not restrict access in any way, I believe design in Swing can be clean, it just takes too much lines of code to get to that point.
What do you think about visual designers? Should we encourage their use for beginner developers, even at the price of hiding the API internals and creating possible maintenance problems?
Perhaps I’m old-school as I have always preferred VI/command line tools over IDEs and designers, but I would say yes. Let beginners use a visual tool to feel more confident while starting out, but at the same time encourage them to look under the hood and get acquantied with the generated code, so that they may understand it, and later be able to tweak it by hand when and where it makes sense.
What do you think about blurring the lines between business and consumer applications? Should a business application look sleek just because everybody else is doing it?
Should we all fall from a cliff as lemmings do? of course not =). I’m with Chet Haase on this one, snazzy and springy effects do not belong on a business application in most cases, but using animation as a feedback is a powerful tool, perhaps it is a glow effect on a button or a translucent dialog/blurred background combination to focus the user’s attention into a recent change or a part of the UI that requires interaction at that precise moment.
What are your thoughts on breaking compatibility to correct old mistakes and provide a better designed UI toolkit?
If it makes development and a reasonable (by reasonable understanding perhaps a ratio of lines of code to change mixed with the amount of testing that has to be done) migration path for existing applications for which source code is available I would say yes, go for it. I’m sorry for those people that would like to use newer versions of the JRE but remain forever in compatibility mode, compromises have to be made if you are willing to play with new toys.
Is there anything else you would like to add?
Don’t write off desktop apps just yet given that many have jumped into the web bandwagon. Webapps have been copying desktop apps since the very beginning, and now desktop apps are catching up, after all Ben Galbraith said a year ago “Web is becoming ‘desktopy’, Desktop is becoming ‘weby'” (or something close to that) and I believe he is right.
Chet Haase is a well-known figure in the graphics community. Join him as he talks about UI development, collaboration between designers and developers, desktop vs. web and more.
Tell us a little bit about yourself.
I’m a software geek, with an interest and focus in computer graphics. I’ve been working on graphics software ever since I got bored in my first job out of college, where I was doing “network analysis.” I raced off to graduate school and have been doing graphics ever since.
I’m now working at Adobe Systems on the Flex SDK, where my current project is (surprise, surprise) to add additional animation support to the library.
I also enjoy writing and speaking about technology, from my blogs to articles to the book that I wrote with Romain Guy, to presenting at technical conferences. Implementing technology is fun, but it’s also fun to articulate to the users of the technology how things work and how they can take advantage of the platform for their applications.
What are the three most important things for a good user experience with a business UI application?
I don’t know about ‘most important’, but here are three that I think are very important:
Understandable: Can the user understand the UI? Is it obvious what they’re supposed to do at any point during the application?
Efficient: Can the user get their job done quickly? This includes power-users: you need to make sure that people that use the application all the time have shortcuts or very fast means of accomplishing common tasks. I saw this in action watching users of Autocad. As a graphics person, I would expect all of the drawing to be via the mouse and display, but the power-users that ran the program all day long performed many of the operations by typing in shortcut commands instead.
Robust: This one is obvious: Can the application avoid errors and crashes? No matter what the application looks and behaves like, if the user can’t get work done because it’s crashing left and right, then they’ll abandon it for anything else that gets the job done.
Can business applications (especially internal) justify being “sexy”? Is the current trend of bringing sleek visuals, animated content, 3D effects and media more than a fad?
Well, I sure hope so, or else I’ve been wasting my life for the past few years…
I think that it’s easy to write off graphical and animated effects as just being eye-candy. But I believe that any application can benefit from some level of effects in order to make the application more powerful, effective, and useable. In the case of a business application, you might not want to put throbbing buttons and spinning sprites all over the screen, or to make the application look more like a media player than an HR forms database front-end. But the same techniques can be used to make this kind of business application more useable; you just need to know how and when to apply them. For example, animation can also be used to provide a subtle pulsation of the default button, or transitions between states of the application. It’s this last one that has been of particular interest to me for some time. Why give your users new GUI pages to figure out whenever the application screens change? Keep them connected to the application, and more productive, by animating between the states so that they can more quickly determine what’s going on and what they have to do.
Animation and graphical effects can be totally gratuitous, but they can also be helpful and powerful tools for any application, as long as they are used properly.
Rich client development tools place great emphasis on allowing smooth collaboration between developers and designers. How important is this for developing business applications?
There should probably be more use of designers in business applications than there is today. “Design” is not just about creating media-rich consumer experiences; it is also about making applications more useable by the target users. The ability for designers and developers to collaborate more closely should help in developing applications that incorporate this design input.
Most of your career has been spent at the pixel level, but your last few efforts (such as timing framework and animated transitions) were on higher concepts. Does this reflect on leaving pixels in the hands of designers?
Actually, that’s more a picture of my recent career at Sun, working on Java, than of my career overall. Prior to that experience, I had worked at both lower levels (e.g., implementing drivers for graphics chips) and higher levels (e.g., writing 3D modeling applications) over an uncomfortably long career in graphics. For me, it’s always about getting pictures onto the screen, no matter where I’m working in the stack.
But to your point, I have spent the past few years trying to apply the lower-level techniques to benefit the higher levels of application development. For example, the timing framework and animated transitions libraries were about providing capabilities to developers that made it easier for them to add animation to their applications. I think that most business application developers have their hands full just implementing the logic of the core application without also worrying about how to use graphical effects. The more painless we can make it for them to do so, the more possible it is for them to apply the techniques.
I suppose this point also relates to designer; if we require people to dive into low-level code to implement nice animated effects, then it’s just not going to happen. Whether those people are designers, who probably don’t want to fool around with code at all, or business developers, who may not have the time or expertise in this area, if that’s what it takes to get those nice effects then the applications will ship without them. On the other hand, if the people that understand the low-level details can create higher-level utilities and tools that accomplish these tasks, then we can broaden the reach of these techniques and make it possible for all kinds of developers to use them.
Are you a proponent of visual designer tools? Do you think that automating the UI development results in better looking and more maintainable applications?
Yes. I’m not a designer myself, so I can’t speak for the tools and experiences that designers have in application development. But having used some decent GUI builder tools, I’m completely in favor of letting the tool do the visual stuff and saving the code for the logic. If it’s all in code, then it’s too easy to decide that that little UI tweak isn’t a big deal because recoding it would be a hassle. But if you have a design tool that makes visual aspects easier to develop and modify, then it is much easier to make small adjustments that may have big user and usability impact.
Do you think that a UI toolkit should enforce a specific and rigidly “guided” approach to UI development and restrict the API expressiveness, even at the cost of alienating advanced developers that will not have access to low level APIs.
Software, for me, is an art form in which nearly anything is possible. If you restrict the capabilities too much without letting the developer get in and hack things differently, that’s not software. I do believe that guidelines should exist, but in a way that helps the developer get the basics right. You want to help them get decent-looking applications without trying too hard, or having to learn and custom-code things too much. But that doesn’t mean you should prevent the developer from changing things as they see fit. It’s their application, after all, and they probably know their target audience and domain far better than the folks designing the toolkit do.
Do you see added value in cross-platform UI toolkits? Is it worth sacrificing access to platform-specific native functionality?
I see value in both. Some applications need to run on multiple platforms and need to have the same, or at least similar, user experiences on these platforms. Other applications need to be tightly integrated into the native desktop and need to have user experiences that are very tightly tied to the native look and feel.
It would be safe to say that the web is winning as the delivery layer of rich applications. What do you thing about a browser as the container layer? How important is it for a UI toolkit to live both in the browser and on the desktop?
Most applications that we run today, if you include the simple shopping and reading experiences we have on the web, are browser-based. So clearly that is an important medium for any application, and I personally would like to see richer experiences than HTML for much of what I do on the web. But I also use many applications that have nothing to do with the web, or at least nothing to do with the browser. For these applications, I don’t want to bother with the browser container, but instead would like the application to run as a normal window on the desktop. The extent to which a rich UI toolkit can serve both of these needs is beneficial to the toolkit as well as to users that can share common experiences and paradigms between the browser and desktop.
The Parleys.com application that was demonstrated at JavaOne and earlier at JavaPolis is a good example of a program and toolkit that spanned both spaces effectively. It has a browser-based model where the user runs the application on the parleys site and a desktop-based version where the user gets a very similar experience in a native desktop window. Since the application uses the same toolkit and UI for both cases, the user can apply their knowledge of the UI on one version directly to the other version; there’s nothing new to learn and they’re up and running quickly.
Is the browser the new platform? Extending the previous question, how important is it for a UI toolkit to be cross-browser?
Cross-browser functionality for a toolkit is quite important today. In some situations, there is only one browser that matters. But even in these constrained situations, you cannot alienate your potential user base by having your application behave poorly if people happen to run it on a different OS, or with a different browser. Also, you can never tell how the future dynamics of the browser wars will work out. So it’s far better to use an approach that makes your application work well regardless of platform and browser.
Is there anything you would like to add?
I would just like to encourage everyone to go out and create rich experiences. Life is too short to run boring applications.
The biggest pain point of Substance users, as expressed in the comments on the announcement of release 4.3, is the performance. While this might have been less noticeable for smaller UIs, the users of Substance NetBeans module were especially vocal about the performance issue. As i have already said before, you made your statement very clear, and performance improvements were placed as the top priority for the next release (code named Substance 5).
Following the extensive correspondence with Java2D team (thanks, Dmitri), i have created a new java.net project named LightBeam. LightBeam has a collection of static and dynamic scenarios that are targetting different core Swing components and different interaction scenarios. The static scenarios create a number of components and then render the main frame onto an offscreen buffer. The dynamic scenarios run a number of typical interaction scenarios that involve changing the components or models.
Here is the spreadsheet with the performance improvements on static scenarios. The numbers are milliseconds it takes to render a frame with a number of matching components (buttons, sliders etc) to an offscreen buffer (the smaller, the better).
An average improvement from version 4.3 is by factor of 3.5. As you can see, slider rendering was especially slow in version 4.3. Another important number to notice is the heap memory consumption – it has not grown at all as compared with version 4.3.
And here is the spreadsheet with the performance improvements on static scenarios. The numbers are milliseconds it takes to run a specific scenario and repaint the contents (the smaller, the better).
An average performance improvement from version 4.3 is by factor of 2.5. As before, sliders were especially slow in version 4.3. Note that there is one scenario where the additional visuals provided by Substance prove to be still too costly – in dynamically selecting table rows. This would, obviously, have to be addressed before the final release.
Project LightBeam is targeted at look-and-feel developers that wish to test the performance of their libraries. The SVN repository contains a number of sample scripts that were used in creating the above spreadsheets. It will be extensively used throughout the development of Substance to make sure that there are no performance regressions from this moment forward.
The latest development drop of Substance 5 should be very stable. You’re more than welcome to try it out and report any exceptions, errors or visual inconsistencies on the project forums, mailing lists or in a direct mail. Also, if you’re using NetBeans 6.1, please consider trying out the latest drop of Substance module. It should be noticeably faster, but if you’re experiencing any slowdowns, just right-click on the title bar and select “Show cache stats”. In the new dialog, click “Copy to clipboard” and attach the results to a forum / mailing list posting.
Update – the numbers above are for software unaccelerated rendering. No work has been done yet to analyze and improve the performance on d3d-accelerated pipeline.
