Interview for the Unplugged series of creative and developer interviews

October 10, 2009

The transcript of Mark's interview from http://www.motionworks.com.au/2009/10/unplugged-9-mark-coleran/

Hey everyone, welcome to Unplugged. Today it's my great pleasure to be talking with my good friend Mark Coleran. Hi Mark. Good to see you again. Now where are you at the moment?

At the moment I'm sat in my office over at Gridiron Software. We're based in Canada in the capital in Ottawa.

And you are originally from England right?

I am, yes.

I'm not sure where you were born Mark. Were you born in Buckinghamshire?

No, I wasn't. I'm actually a Northener by heart, a Lancashire man.

But before you came to Canada you were based in Buckinghamshire which is where I was born. I was actually born in England. There you go. So Mark, you work at Gridiron Software now. What's your role there? You're an interface designer, is that correct?

My official job title is Director of Human Interaction and Visual Design, which is a real mouthful and sounds very official.

Human Interaction, that's wonderful. I would never have picked you as the guy who'd be in charge of Human Interaction.

I've been accused of not being a people person in the past, but essentially it is really about what buttons do we press, how do we press them, all those points of contact we have with the things that we work with in the software. It's quite an involved job in the end. There's a lot of common sense and some technical things to work out with it, but overall it's just applying how you interact with stuff to software.

And this is a fantastic story because before you got started with Gridiron, had you actually had any experience creating real-world interface for actual products?

I have actually, yeah. I did some work with the guys at Silhouette. Even over the years, for a long, long time I've been beta testing and testing software for different companies. And a lot of what you feed back is suggestions on how stuff can look better, work better, feel better than what you work on. And some of that included making mock-ups occasionally. So that was an area for designers I was already familiar with because of the film work.

That's your heritage, isn't it? You are probably better known for your on-screen, graphics for cinema - is that what you would call it?

We've been struggling with trying to actually define what to actually call out for years. Some people call it playback graphics, on-screen graphics, screen design, fantasy interfaces. I've seen just about everything you can imagine to describe it. But essentially, it is creating fantasy computer systems for films, so it looks like people are using and interacting with real devices - and sometimes not so real - in all different types of films, whether from drama, action, thrillers to sci-fi's to all kinds of stuff.

We had a discussion about why people like Gridiron Software would be interested in your work. And it was totally obvious to me because you're the guy who imagines what the futuristic interfaces would look like. It makes sense that these companies would want you to work on their interfaces now. Is that fair enough to say?

You do get a lot of latitude in the film to push the boundaries a little bit.

You mean Gridiron Software doesn't have any like retinal eye scanning? [laughs]

No, we've passed on that kind of stuff, but maybe it's a good feature idea for the future.

We met about nine years ago at the After Effects user group in Sydney, and you actually contacted me and asked whether you'd be able to join in. And I was like, my God, you know. You came to the Computer Graphics College and you showed us some of your work. I think you were working on Tomb Raider at the time. How did you get started in creating on-screen graphics? Had you only just started around that time or had you been working in a long time before then?

I think Tomb Raider was the third film that I actually worked on, so I was still pretty new to it. It was actually by accident I ended up doing a lot of that work. And the other irony is that actually my interest in graphics has always been in information design and the UI type of stuff anyway. I've always loved that kind of work. I was working at Pinewood Studios and I was doing some work with a small special effects company. We were doing this very strange task called pre-visualization, which at the time nobody had even really heard of. It was this idea that you could use 3D applications to simulate film setups and stunt setups.

We were playing around with that, and I got exposed through that to a couple of stunts that the guys were working on for a film called Entrapments. On one of the setups that we were doing for that film, there was a device that had a screen on it. The company I was working with was building that device. I was like "I want to do the screen". They said "You can't do that, that's somebody else's job to do that." So I found out who the guys were doing it and begged and begged and begged and begged to try to get a job with them. Eventually they gave in, I don't know whether they just shut me up.

Why did you beg and beg? Because you thought it'd be a cool thing to do, or because you thought that you would be really good at doing that?

It was just a very interesting little area to work in. I'd seen some of their work before on things like The Avengers and the original Mission Impossible film. It just looked like a really cool thing to do, so I really wanted to have a go at it.

I think a lot of people feel the same way when they look at screen graphics. We'll talk about the actual industry in a moment. So you got that gig, you were able to convince them to give you that work?

Well, as you can see, I did. I think they just did it to shut me up in the end. Keep him quiet, stick him in the corner. But the thing is, I was not good at that job. I had a little bit of ability, but the thing is, I had some really good people around me. There's one particular guy who is the art director there, a guy called Simon Staines, who is probably one of the best people in this business. I had a really, really good mentor in him who taught me a hell of a lot of stuff.

Was he teaching you about just onscreen graphics design, or was he teaching you After Effects techniques?

It was really more about that attention to detail. It's about what makes these things work, what makes them look good. Simon will admit himself he's not a very good After Effects artist at all [laughs], and the fact is, neither am I. What you see belies a lot about how it's actually created and made. I still find it laughable to this day that people regard me as an After Effects artist. 90% of my work is Photoshop. It is quite that simple.

But being good at Photoshop is a key skill for an After Effects user.

It's a key skill for any designer, really. It is the pad on which you build a lot of your work. But even though the stuff in the film looks very sophisticated, a lot of that's implied from the actual design itself. The vast majority of what I do and what I used to do, the closest comparison I would have for it is cell animation. It's not really sophisticated motion graphics. There's no complex techniques.

A lot of it's repetitive, isn't it? Like dials and little widgets moving up and down. And I guess you build up little elements piece by piece, and then you just put everything together, which makes it look complicated. Would that be right?

Yeah. And over the years, what I actually started doing was standardizing a lot of those elements. If I have a small LED type bar that moves up and down, I'll only do and animate one and I'll use it in 15 different films.

There you go. So you have the Mark Coleran and on-screen graphics toolbox.

I built that toolkit over the years. All those elements are standard. I'm doing layouts and everything. I already have pre-rendered movies with their alphas that I can just drop straight into place, use a little bit of After Effects magic and control them as well, so you do retain a piece of control over those movies. You can very quickly prototype and make these things work.

I've been telling people for ages when I talk about workflow in After Effects, whether you're creating onscreen graphics or stuff for the web or whatever, having a great toolbox of things you've created in the past, and that you can just draw on and drop into jobs is a really, really important skill.

Absolutely, and it got highlighted again and again. The longer I continued to do the movie work, the more it became an issue. When I first worked on a film, we made 20 screens maximum on a film. And later on it became 200 to 300, but you got the same time and the same budget. So essentially, you have to find lots and lots of ways of improving that stuff. Unfortunately, it's like a lot of design, it's actually become very commoditized in some ways. The screen is a commodity, they'll pay so much per screen. You have to find ways of being really efficient. You have to simplify a lot of the basic work you do.

Then, you have the occasional screen that comes up - which is what you might call hero screen - these are the screens that have the really cool stuff going on. They're not in the background. They'll look good. They'll tell you something that's going on in the film. They'll tell you part of the story. By saving this time and almost relegating a lot of the other stuff, you actually have the opportunity then to spend that extra time on the really good ones. It's worth it. You do get a pay off for doing it.

You probably would have gotten a lot faster because of your toolkit. In the early days, you had to build things from scratch. I appreciate that you had to build more screens in the later films, but because you had your toolkit, maybe things were a little faster for you?

Absolutely. It's something that doesn't really matter what kind of thing you're doing. You continually adapt to a job as you're doing it. You don't even quite realize it at the time. Even with maybe the last five or six films that I did, they'll look radically different. But if you actually look to them at their core, they're exactly the same screens. There's no point in continually reinventing the wheel. I'm using exactly the same layouts and exactly the same screens in different films. And really, I'm just keeping a workflow there that allowed me to change the layer styles for a particular image or chase how the buttons look. I immediately have a different kind of OS for each film.

Did you use Photoshop a lot? Do you use Illustrator a lot doing those graphics?

It depends on the nature of the film. For a lot of layouts, I actually work pixel perfect. That requires very precise control over the actual line work and the text, because you want the sharpest possible image on screen. It's like in motion graphics, you'll have an output format, and that output format was in the past beta SP, or maybe 720, or maybe 1080, and those output formats dictate a certain size, whether it's 1080x1024, or all those kind of considerations. But for us, the output is whatever particular screen is going to go on. So that can be anywhere from 1280x1024 to 2560x1080.

Would they tell you the size of the screen in advance? Would that change? Would you be three quarters of the way through a job and they'd tell you that they decided to put it on a smaller monitor or a bigger monitor?

Oh, it's never on a smaller monitor [laughs]. Yeah, that's an absolute nightmare. Originally, when I first started working, we would supply everything. I was working originally with a company called Useful Companies, and they would supply all the screens, all the video equipment, everything for that production. Of course, over the years, we have this beautiful thing in films called product placement. So now, every computer company on the planet wants to get their product into the film [laughs]. You generally don't even know what you're going to get until very, very shortly before you start filming. And things can change very, very suddenly as well.

A part of what I actually build into those screens now is it's not particularly visible, but I can actually create and highlight some examples for you. All of the screens I do are based on a very, very specific grid, a very regular grid. So even if I say a screen that was built in 1280x1024, it can actually be cut at a perfect point and just down to 1280x720. So if you want to play it on a widescreen image, all you have to do is cut a certain point. It doesn't affect the design. You just have to trim the animation. So a lot of that stuff is actually built into them to start with.

Is that technique something that you learned as you went? Did you develop it, or is that something someone taught you?

It's a combination. You're on set and you're continually having to retask and do stuff. You learn the hard way that this is something you've got to do. And people like Simon, of course he was always playing around with ideas, ways to make it more efficient and easier. He came up with a lot of stuff like this as well.

It sounds high pressure. It sounds very stressful. I've known you for a while, and I've seen some of the results of some of the pressure on some of these jobs. When I first saw you and I saw the graphics and even yourself, as you just mentioned, when you saw the potential for doing on-screen graphics, it seemed very glamorous. It seems like it's a dream job that the After Effects guy, especially when I saw that the stuff that you'd done had been built in After Effects, we all want to do. Was it glamorous? Is it glamorous?

Quite categorically, no [laughs]. Depending on the work you're doing and the way you approach it, it can be very rewarding. But in the end, it becomes like any other kind of job. You have to find and work out your own rewards from that job. Is it creating that stuff, designing that stuff? Or is it sitting back in a movie theater and watching it on screen? You have to find your own space to be able to deal with those kinds of jobs.

The harsh reality of it is, as far as I'm concerned, it's quite simple. I am not the best guy out there that did this stuff, even though I seem to have this reputation for doing screens.

Why is that? Why do you think you have the reputation?

It's time. That's all it is. It's time and quantity. I've seen some amazing artists come in and do this stuff, and they'll do it for a year or two, and then they're out of it [laughs], they don't want to do it ever again. But whether that's particularly because a lot of the stuff we actually did, you actually end up on set doing this stuff as well. You're actually dealing with crew, you're dealing with animation that has to be made to work live on a set - with the attendant foot stamping and screaming. That happens when things don't work according to plan.

I thought that you'd be sitting in your office, building the graphics, and then you'd send them a hard drive and say, here they are. But you're actually on set, and you were changing the graphics live, basically.

There's two reasons for us being on set. And the first one is that you know those graphics. If they have to make changes, then you can do it very, very quickly. As you know, with After Effects, there's attendant turnaround time. So you're having to work very fast to make small changes to color, to the way it animates, and the way the story is being told on those screens. You got to get it there and back up on set, and you probably generally have 20 minutes max as far as tolerance goes for changes. So you have to work very, very fast.

Ironically, that's how I got introduced to Gridiron. Working on there, you had to get stuff done fast, so you look for every possible way of squeezing out extra speed. That's when I actually got introduced to Gridiron's Nucleo, and that's how I came to know these guys here. You got to render fast, and it's just a nightmare situation [laughs], especially when renders start failing.

Why did you choose After Effects? Is that because things were created in Photoshop? After Effects isn't the fastest renderer in the world. Was it good enough to do what you needed?

It's just the closest link with Photoshop. It's easy to look at a lot of these products now and think, well, why would you use that and not this? But the fact is, 10 years ago, half of these other products didn't even exist. Even at the time, it was a very, very flexible and very easy to use product that tied pretty seamlessly into Photoshop, just things like the ability to import Photoshop files as compositions - which even to this day, still makes me sick when I think about it - because I didn't find out about that for about a year. I could actually attribute a couple of overnights to me not knowing about that.

It has to be said, though, that After Effects is even more tightly integrated now with Photoshop. The features now compared to 10 years ago make it an even more obvious choice for this.

Now it's just beautiful. It just makes it far easier to actually do stuff with it. I remember at one point where if you had a Photoshop file with X amount of layers in, you actually had to create 20 to 30 dummy layers on it. If you wanted to add anything new to it, you had to add them in order. One layer would disappear and they have something else on it completely [laughs]. There were lots of little errors with it. But bit by bit, they get squashed and the whole thing sits and works a lot better. There's massive improvements to it.

Even now, I continually look and try different products. Even though I'm now working with Gridiron, I still keep my finger in with this work.

So you aren't doing some of this work? What have you done recently? Is there anything you can talk about?

Not really.

So these are the films that haven't been released yet?

Yeah. I'm getting more involved back into it. I've played with a couple of different projects over the last couple of years. I even turned a couple of projects down, much to my annoyance as well. There was this one really pokey film called Iron Man. And I thought that doesn't look very good [laughs]. So yeah, you live and learn some of these things.

Oh, that's amazing. Well, Iron Man 2 is coming up. So you missed your chance. The boat sailed for that one.

But then again, the guys who did that work did an absolutely beautiful job on it. That's not exactly an issue.

"The Island" featured some on-screen graphics where the character was pushing graphics around on a table. Can you tell us a bit about that? You designed all of those graphics?

What the production did, they had a guy come in from MIT, a guy called John Underkoffler. He was actually the guy responsible for the concepts for "Minority Report", using the hand control things and systems. His concepts are essentially about the interaction. It's how do you interact with these devices.

What they had was this table idea where the table is a screen, and how do we interact with it? His work was to have these devices on there, almost like token objects that you could use to move stuff around. Because then we get the job to actually make it look like it works and like it's real. Essentially, it's not particularly difficult because it's just one large screen. All you're doing is you're doing after the fact. You see their actions and see what they're doing, and you're placing objects in certain places and then interacting with them and making it look like they're interacting with that stuff.

Was that done some motion tracking?

No, it's all done visually. You can track a finger or an object or whatever for the first little bit, and then you let it go, and then just use little After Effects expressions to create pseudo dynamics and stuff, so everything looks natural.

That's the thing. You say you're not very good at After Effects, but you make After Effects very simple. I think that is one of the great skills.

I'm not. The expressions I know are extraordinarily simple. The vast majority of them are no more than one line. Half of them are involved in picking objects up to something else. Of course, you do crib a few from the real masters like Dan Edwards and people like that occasionally. So I will take no credit whatsoever for it.

With a table, I animated the vast majority of it. But that's really just for the interaction. If you're actually going to put that screen itself into an object that was moving around, then you would obviously have to motion track the whole thing in. Using a Wacom tablet, I can actually do draw the animation using the motion sketch tool. Using things like that, you can do interaction yourself to make it look like it matches. But then of course, the whole thing has to get put onto that table, and a lot of that was done as a post composite in Flame by a different set of guys. You supply them that work and they put it together.

They do all the rotoscoping and that sort of thing?

The table itself was actually just a light source, so the rotoscoping was minor. They could actually just use a luma key for the whole thing.

It's important to set up your scenes like that. You get people who go "Let's just shoot a green screen or blue screen or something like that" and get the total wrong method. And then it just is a killer for the person who has to composite the shot.

The actual worst thing about that was that they wanted to do it live originally. They actually wanted the table as a screen. To do it at that resolution, you would need a 2K projector. It would have to be underneath the floor and go and shine up onto that surface. They got a projector, they tried it out, it worked, but the table itself is seven feet long. It just about worked. It looked pretty good. But then the whole set got pushed back by two weeks, so instead of keeping hold of the projector, they sent it back to the rental company. And then they got a new one as it came on set. It was something around a 1024 projector that was supposed to project like seven feet across, and it just didn't work. In the end, it had to be a composite here.

It was so realistic, shifting the digital pages on the table. Microsoft had a product around that time, and someone got a hold of what you'd done and made a connection between Microsoft product and what you'd done.

Yeah, there was an odd thing happen with that. The Microsoft Surface came out a couple of months after the film did, also announced and shown. It was a beautiful looking thing, and of course, it looks and feels very, very similar to what we did. The odd thing about it was that somebody had seen Microsoft, like a lot of companies that take support and sponsorship roles and finance for product placements, and they put two and two together and got 15. They got a little carried away [laughs].

There's a very famous Microsoft blog called I Started Something by Long Zheng. He covers a lot of breaking Microsoft technology and does a really good job of it as well. He picked up on that story and really gave it some legs. One thing led to another, and it ended up as the story that "The Island" used Microsoft Surface. Of course, Microsoft has kept their mouth shut [laughs], which is good publicity for them anyway. Somebody I know there actually tipped me off to it. I had a chat with Long Zheng and they actually put a really good correction on there. We talked about how it was done in reality and the real story behind it.

The weird thing is, in some ways, for a lot of that stuff, there is actually some truth to it in a very abstract way. Even though that was not Microsoft Surface, a lot of the research that I'd done, and a lot of all of us do when we're doing these kinds of jobs, is we look around at universities, company labs, and on their beta sites and pages and what they're up to. You see all these cool tech demos they're doing and all these ideas they're playing around with. You take elements off of that, throw them into the films and make it look like it's real and it works.

So it's not surprising that someone would make that link then.

Yeah. Of course, you're looking at the lab stuff. By the time the film comes out 18 months later, some of that stuff is actually hitting the street and being released. Sometimes that goes full circle. The irony is I hear that people in some of these places are inspired by the stuff we do in the films. So this all gets quite incestuous really. You don't know where one starts at the beginning [laughs].

We're going to have to wrap it up in a moment, Mark. You also do some presenting and talking about the on-screen graphics work. Have you got anything coming up? Are you doing anything like that?

Yes, I do. I'm doing a long session at the Motion Conference in Albuquerque in mid-October. That should be a really good little session because instead of a quick overview of what I actually used to do in the films and still do, this is actually going to sit down for three hours and go through a lot of the techniques that I used - from different types of screens, whether it's a hero's screen on a monitor or some like a heads-up display, all those different types of applications you get. What do you do? How do you do it? How do you get this stuff to look realistic in a way and convincing? At the end of the day, tell a story because that's all it's about.

My mouth is watering and I'm sure a lot of you guys are thinking, maybe I'll buy tickets at Albuquerque.

There's another one coming up in Barcelona in November, which should be quite an interesting one, and it's a bit more abstract. It's more about the business of design and the mentality that you need to approach it. The talk is essentially titled "You Are Not Your Job Description".

I do have a project in the pipe. What you actually do is living here in Canada is you have a six-month winter. It is a nasty six-month winter as well [laughs]. It's not an easy winter. So I've been developing winter projects at the moment. One of them is to put together a good training resource on this kind of thing. Not specifically, although the way I apply a lot of stuff is for screens. There's a lot of good little techniques in there that hopefully people can get something out of.

There's a big argument I've had with a few people in the past. When I've done training, it's been built as advanced After Effects training. And then 90% of it turns out to be Photoshop and the After Effects stuff is really, really simple. They complain about it. But the fact is you don't get out of bed in the morning expecting to do advanced After Effects. You've got to work to create something. Hopefully I can actually get some of that stuff into this training. There's good shortcuts and ways of utilizing your time and resources a lot better. It is the way you apply something, not which buttons you press or which complicated scripts you write. You find the easiest way to get that done and get it out the door and get paid for it. At the end of the day, that's all that counts.

As usual with you and with everyone I speak to, I could talk all day, but thanks very much for your time.

You're welcome.