Interview for the We Are Embedded video channel

March 13, 2012

The transcript of Mark's interview from https://www.youtube.com/watch?v=ro293z1bkLY

Welcome to another episode of The Embedded, an ongoing exploration of the space between the glass and the technology. In this episode we will be talking to Mark Coleran, design director at Bonfire Labs. Mark is also a veteran of the film industry. He's designed interfaces used by actors in films such as the Bourne series, Tomb Raider, and The Island. We talk about interface design, device personalization, and how his unique background has brought some interesting insight in the evolution of the natural user interface.

Thanks for joining us. I really, really appreciate that you took the time for it. It's very cool to see you again. Tell us a little bit about how you got into interface design and everything from film design to where you're doing right now.

My background is actually graphic design. I looked into everything from print, commercial to advertising agency. That actually involved into motion graphics design in itself. I actually found myself specializing in a very odd little area of it and it was creating screen interface graphics for movies, and the group of guys that did this work actually created interfaces for film, and so I was kind of hooked and stuck with them for many years and learned the craft from there.

When you're actually doing that kind of work, you want a certain level of authenticity. Initially it can just start as things like copying what I didn't even know, understand was just the correct term at the time - things like affordances, why things look the way they look, the certain sizes of things and the way things are laid out. Although it's not the same, they'll look at it and think, oh yeah, that's a computer. It's kind of like mine, but it's not real. That real focus on real interface work bled more and more into the kind of work that I was doing in film, and so eventually started to become a much more bigger element of it.

I guess that must have been a bit of a transition though between creating your fantasy interfaces versus real interfaces.

The transition was a very, very long time actually. The very first real interface I did was only like two or three years into doing the FUI stuff in films. We just sat creating false interfaces and you're using a piece of software to do that. You start to look at that piece of software itself and the deeper you get into using something, you start to find those edges and those weird frustrations that you develop using any pro piece of software and that expresses itself in two different ways.

One was that you joined the beta programs for them, so you can get a feedback and you start to develop a deeper understanding of what you need and how it works. But on top of that I also started taking just those skins of the programs and looking at them and thinking, oh how could it be better? Essentially making makeovers of software I'm having to use at the time - things like Commotion and After Effects and things like that. Those were the very first steps I took in thinking, okay what can I do here? How can I do this? And a very immediate realization of actually how difficult the job that is.

You're not designing a thing that's going to be seen for three seconds in a film. You're designing something that might be on somebody's desk for three years before you get a chance to change it.

But even still the film design is so influential to what we're doing right now. Clients constantly say I want something like X from this movie, that movie, the other movie. It's hard to tell people that that's not really realistic.

Yeah I have the same problem. In fact I particularly have that problem having created a lot of that stuff as well [laughs]. There's several reasons for it I think. One of them is a fundamental misunderstanding of what is actually being done in a film with an interface. They're there for very specific reasons. They are there as a set dressing to imply a certain level of sophistication, a certain type of task, and in particularly tell a story. All of those things, that's all they do. They serve the story, and usability has nothing to do with it.

You don't want people actually using them in a film. Actors are more concentrated on the performance, and not using a computer. People are aware there's a trend to make things simpler and easier to use. The old desktop model of software is fine. I have my mouse pointer. I need a function. I go and find it. Whereas a lot of the newest generation of software is about discovery. I have certain things I can do, and any one of those things leads me down a path. That does not work on film. We don't have the time. We don't have the facility to actually be able to tell a story that way. We have to imply that something is going on and do it very very quickly.

There's a feeling that those are predictive interfaces.

There are very few exceptions where some stuff is completely out of bounds and completely made up, but the vast majority of things that you see in a movie do appear to be predictive, because you'll see them used in film. You'll see them exposed there, and then anyway between six months to two-three years later this stuff hits the market in some ways. Or it's just the mechanics, whether it's touch or like a table or things like that.

We're actually taking a lot of these ideas from labs, from garages, from universities. People publish this stuff, they want to share it.

It's almost like you can look at things that are being experimented with right now, and just ignore that they're clunky, because that will be worked out, and predict what it will look like when it's worked out. Now we're looking at a time where we have all these gesture-based interfaces, connection to the cloud, and we have this personalization layer. How important do you think that is in personalizing user interfaces or experiences.

The hardware is just hardware. Everything about it is, and what you actually you do on it - that layer is everything. The device is almost irrelevant now.

We see all these things coming together, the connection with the cloud, personalization, how incredibly important it is. Any predictions on what we can see in the future?

One of them is that the interfaces we design in films - they have an identity. Each one almost becomes like a character in itself. I think people are breaking away from how something is supposed to look in a certain setting in a certain system. And the other thing we did a lot was the use of animation. A lot of interfaces have been very static and the way you use animation is going to become critical. It's not just about a bouncing icon. It's about using animation to guide transitions, to show and apply what can be done, especially in environments where the traditional affordances no longer exist.

My own personal prediction on all of it is a radical change in the way we actually even think about these devices right now. You look at the first command line interfaces to the first GUIs, and everything is still built around what you might call the industrial model. We have tools and we have tasks to be done. I go to that tool and I go do that task. Maybe I have some result of that task, and I have to grab that and go and take it to somewhere else and do something else with it.

It's becoming a really clunky method, especially now with the way mobile devices work. Some people are trying to get solutions to this with some of the cloud services, but in some way they're obfuscating it and making it even worse. The biggest change is that the whole front end of the interface and the entire hub of how we interact with these devices has to become contextual. It has to be about who you are, what you're interested in doing, where you are, and what you want to do. It's going to result in some radical new interfaces - right from the complete system level up.

Forget about files and folders as objects. They're just events the things that happened.

As we get to more gesture-based and touchless interfaces, the more natural interface, the more subtle it has to be.

Right now it's all touch. You have people developing the real versions of the Minority Report. He stood up on stage at TED and talked about their system is going to be everywhere and everything in five years time. I disagree. It's one thing. It's going to have some great uses in certain places at certain times. People have a tendency to stick with a touch. There is audio, there is visual, there gyro - and people aren't using. How can you combine those things to create unique and fluid interfaces?

Can you tell us a little bit about what you're maybe working on right now or in near future?

I'm working with a small company called Bonfire Labs. They were traditionally an editorial and motion graphics company. They service a lot of tech companies in the Bay Area. A lot of the work they do involves simulation of interfaces that people have designed.

Outside of that I'm trying to get people to work together to actually just play at interface design. Get people who are interested in it, get people who use stuff - there's a big little dream, or maybe even pipe dream that I have [laughs] to actually get almost a little salon type thing going, to create the equivalent of an open source interface design that could even go with a open source software.

If there was one thing, anything that you could that you could work on and turn into reality, what would it be?

The common thread amongst all of the screen design that I did and the FUIs for films would be to give them their own identity and call them an OS. The one thing that's always sat in the back of my mind is to actually create a real one. From ground up, just try and rethink a whole OS. How could it look? How could it work? That would be the one thing that I'd love to do.

I'd love to see it.

I'm already working on it [laughs].

Thanks again for joining us.

Thank you.