Session at Frontiers Of Interaction 2011 conference

June 21, 2011

The transcript of Mark's session from https://www.youtube.com/watch?v=im8baPJW_IQ

Bonjour now. I really have a hard time when I go to parties. The second question you always get after somebody introduces you to somebody new is, what do you do? It's really difficult to answer that question. I can say I'm a graphic designer, I do motion graphics, I do some interface design, but the main part of my job over the many years has been creating FUIs. So these are fantasy user interfaces. And then I get a look from people and it's just blank. It's like, what? Never heard of that.

So I have to try and explain it. It's like somebody in a film is tapping away on a keyboard and the clock's ticking and they're trying to get into something and they're on the computer and then all of a sudden you get Access Denied. Sometimes they go, oh yeah, okay. And then there's three responses come with that. And it's either, really, somebody does that for a job? Or the other response can be like, oh cool, I love that stuff. And then the other response can also be, but that stuff is rubbish. It's like, it's nothing like the real world at all. It's like, it's junk.

But of course the easier way I could do it, but I can't do this at parties, is to actually show the work that I do. So what I'll do is I'll play the show reel I have. There's a compilation of the work that I've done over the years. It'll be a much easier way to show it.

Thank you. It is as a friend of mine once described it, beautiful junk. And there is no group in the world that will actually look at this stuff with a more critical eye than yourselves - people who create interaction. And the foundation of all work that you do is based on requirements. What is the requirement for this thing? People will criticize this kind of work but never even think about what the requirements are for it. So what I'd like to talk briefly about is what are my requirements for doing this kind of work?

It comes down to three elements really. The first one of these is style. Why do these things look the way they do? There's three parts to this. The first context that I do this stuff is what does the film look like? What is the film? Some stuff that you have to do has to be very realistic. It has to have a foundation in reality and its feet in the way things really work. So there has to be familiarity about it.

Although I don't use things like Windows or OSX because it gets so difficult in licensing and people giving permission. We create our own look based on the feels from that. We give it the same affordances. But still, and speed it up a little bit so visually it looks good and it's what a guy called Chris Noessel called cinemagenic. It's faster and it's visual in the way that it works rather than just real.

And the second category in that is what we call kind of realistic. You get to push it a little bit, but it still has a foundation in reality. Even though this was for a futuristic film called The Children of Men, I'm still using cues and design cues from real software in there like Microsoft Excel. And that again ties it back and people think, oh yeah, I recognize Microsoft Excel there. It gives them a hook and comes back to it.

And the third context and the style of the actual film itself is fantasy. For this kind of stuff you can just do anything you want. And in some ways it's very difficult because you still have to reel it back. You still have to make it believable. You can't just put anything on screen. You have to give it all those same design cues that say this is a button that can be pressed or this is an information. You still have to keep all those things in. No matter how complicated it looks, you're still putting it into this environment and making it look like a real system.

Now the next major context we have to think about in film and the major requirement is action. It has to do something in that film. And the first part of this is set dressing. You actually build this set and you want to put screens on there, and what is on those screens is part of that set. It's part of a whole thing. It really adds and gives a layer to the feel of this thing and makes it look in some ways a very realistic environment. All this stuff is done live. It's not a visual effect. On the set is a camera shoot. We are there and it actually works.

The other context within action that we do is what we call location linking. This is a sample I drew from one of my favorite films which is Doctor Strangelove.

Digital displays can serve as a window in the film. You have two things going on in two very different places, and yet it brings that action together. You can use these interfaces and these screen displays to do that thing is to tie them together and make them part of the whole. It can be very difficult to do that without. Even though all this goes on in the war room in the film, all the time you can see the bomber in the background on the map. You have that connection.

Another aspect that we use in action is what we call insight. There's a complicated part of a plot, something that has to be explained. Instead of having the actors talk about it which they might do for like say a film of a Dan Brown book. But in most films you just want to show it. It's much faster and easier and simpler to do it visually. So you can use the screen as a display to be part of the story.

And the last part of action that we use is really just pure action. The screen itself is part of the actual process, whether it's a heads up display or somebody hacking into something or somebody finding something. That actually serves as a key part of the action and process in the story.

The other major requirement we have, and this is actually a similar requirement that all you have, that is dealing with people. People make every job difficult. But unfortunately without people we wouldn't have those jobs. I am doing this for somebody. I'm doing this for a director. I'm doing this for the production designer. And those people bring all of their own ideas. They bring all of their preconceptions about what they think it should be, and they're not necessarily the best people at judging that kind of thing.

hey have their own ideas that they saw in another film or something they have in the way that works which might not work on screen. Their primary purpose, which is what all of it comes down to, is that all of these things in the film are not there to be used. They're not supposed to be realistic. They're all there to actually tell a story. That is the primary requirement over everything that we do. Instead of it being realistic in how it works, it is really just to say this is part of the story. That is our major requirement.

Now one thing that people talk a lot about to me and ask me about, what do you get these ideas from? Sometimes this stuff seems to be futuristic, and we almost predict what is going to happen. And it's wrong. It's absolutely wrong. At best what we show is we expose things. Everything that I put in a film, I've seen somewhere else. I've seen it in a university lab, or I've seen a company show some tech demo, or some guy or a girl in a garage is building some cool idea.

The touchscreen table was around for 15 years before we put it in the film. It was around 15 years before Microsoft released it. But these ideas develop and we just take those ideas and we make it look like you can buy it. We make it look like it works. In that sense we don't invent the future. People also get the mistake of thinking what we actually show in the film is futuristic because of its complexity. We imply sophistication through complexity. We make it look complicated. The irony in the film is people can just sit down and use it really, really easily. But in real life sophistication is coming through simplicity.

It's almost the opposite. The clarity is not there. What we actually have is there are areas where we actually have predicted the future. But they're not areas that you might imagine at all. So what are those areas? What we can do with a lot of this stuff is, there's three areas overall that we have really predicted the future. So what we've done is show, in some ways, innovation. We show how innovation actually works in design. What we're doing is taking something. We'll take a standard interface that does something and shows something, and then we go and take something else and we mix the two things together. It's a classic way of innovating on an object. Take two things, combine them. People like Apple, none of the stuff that they ever produce is really that radically new. They are fantastic at combining different things and doing it really, really well.

The other area that we do is, and predicted quite well, is using animation in interfaces. We're using animation to show states of change, to show how something works or explain something to people. Animation in interfaces has become a big, big part. It's no longer just a mouse over state and a button press or things like that. You're talking about the whole screen changing, sweeping, folding. There's so many ways of doing it and that's become a fundamental part of interface design.

And the other area is we actually give the screens and interfaces in a film, we actually give them an identity. Each particular part, as I've shown here, these are both from the same film. And each part of that for different people has its own look and feel, it has its own identity. They almost become characters in the film. In some ways that's one of the biggest predictions we actually made, is that software now needs to have an identity. Nobody is no longer using the standard interface guidelines and they're using all the buttons that Apple supplies or all the buttons that Windows supply. They're actually creating their own unique look and feel. For even things like, you know exactly what this comes from, what film this comes from, what TV this comes from.

It has an identity that tells you something about its location. And the real software, I'd like to say for instance Twitter, second you see this, you know where it came from. It starts to develop its own look and feel, it's unique to that piece of software.

It's only really those three things that we actually have predicted, but they're not always what people think. But I hope that's given you a little bit of very quick into what I do and what I have done. And thank you very much. Thank you.