Continuing the series of interviews with designers and artists that bring user interfaces and graphics to the big screens, it’s an honor to welcome David Sheldon-Hicks of Territory Studio. Prior to founding the studio in 2010 David has worked on “Casino Royale” and “Dark Knight”. Since then, Territory’s work can be seen in movies as diverse as “Zero Dark Thirty”, “Jack Ryan: Shadow Recruit”, “Prometheus”, “Guardians of the Galaxy” and the upcoming “Jupiter Ascending”. In this interview he talks about collaborating with the director, the art department and the VFX department throughout the different stages of a production, staying true to the look of the specific production such as aerial tracking in Zero Dark Thirty or high-tech science of Prometheus, the approach he’s taking to design futuristic interfaces, the physicality of technology around us and how it projects into his latest work on Guardians of the Galaxy, and the diverse gamut of projects Territory is working on and how they feed into each other.

From left to right: Peter Eszenyi, Nick Hill, Marti Romances and Ryan Rafferty-Phelan of Territory Studio at work on Guardians of the Galaxy.

Kirill: Please tell us about yourself and what you do.

David: I set up Territory in 2010 together with Lee Fasciani and Nick Glover. Having met while on a project, we wanted to work together and Territory seemed like a good name to describe our ambitions to carve our own path in the competitive creative sector here in London.

I’m Creative Director of Motion, with a background in graphic design, moving image and motion graphics – cutting my teeth in music videos before moving onto films, computer games and TV commercials.

The studio combines a number of disciplines under one roof – motion graphics, animation and live action production, branding and UI across print and digital (web, mobile, tablet) and with the recent addition of Luke Miles as Director of Brand Experience, we are able to help brands engage audiences more affectively through physical and non-physical (service) design.

While that sounds like a strange mix, what ties it all together is our shared creative focus on human interaction, and future and near future challenges.

Film and gaming is really my heartland. Our first computer game project was for EA on “Medal of Honor”, which was one of the newer titles in 2010. They were going for a more “Modern Warfare” type of a game, and wanted an opening cinematic to tell the background story of why you [the player] were going to war.

Our first feature film project was “Prometheus” – a historic moment for Territory. Out of the blue we got a phone call from someone who wouldn’t tell us what the project was but it sounded interesting enough so we signed a couple of NDA’s. Then we were told that we’d be working on computer screen graphics for a new Ridley Scott film that was going to be a prequel to the Alien universe – at which point we freaked out. I’m a personal fan of Ridley Scott, and Ron Cobb’s universal language and iconography for the original Alien movie is held in very high regard in the design community. So to have the opportunity to explore that at an early stage was really exciting. The project itself has defined Territory’s approach to on-set screen graphics.

Kirill: And before Prometheus you worked on a couple more movie productions.

David: Yes, as a freelancer I worked on “Casino Royale” and “Dark Knight”. Both those projects were amazing and I loved the challenge and excitement of communicating complex concepts in a brief moment of pure user interface design that is also an intrinsic part of a sound stage environment that the actors engage with!

As Territory has grown, our user interface work for films has grown and developed as well, and we approach it as a discipline in its own right. We get different unique briefs and each one is a creative challenge. The beauty is in the idea of considering user experiences and interactions without the constraints of how you might build it. Sometimes it’s high tech, sci-fi future-thinking user interface, narrative thinking in computer games, or exploring future applications for tech giants such as Sony, Samsung or Microsoft.

We now find that the conceptual approach that we bring to film work appeals to other clients.

Tracking overlays for Jack Ryan: Shadow Recruit. Courtesy of Territory Studio.

Tracking overlays for Jack Ryan: Shadow Recruit. Courtesy of Territory Studio.

Kirill: What about Zero Dark Thirty and Jack Ryan? Was it just you, or was it done by the Studio?

David: That was completed by Territory, rather than by me as an individual. We did “Zero Dark Thirty” after Prometheus, and it was at the opposite extreme in terms of sci-fi versus real-world narrative. It had to be very real and very authentic. There were a couple of visual effect shots but most of the computer screen graphics were on-set playbacks. The large screens where they’re looking at the Predator drone footage and reviewing aerial photography were all CGI plates that we had built and rendered in Cinema 4D, referencing data from Google Maps and then projecting on-set live for the actors to perform against.

We worked on two aspects of “Jack Ryan”; we created a lot of on-set graphics, and we also created what is called an ‘original content’ trailer. We took footage from the theatrical release trailer and did something unique with it – using hard overlays and surveillance technologies to suggest a narrative and suggesting background for some of the characters. In effect, we were using our UI graphical language both to set-dress the film and to advertise it.

Kirill: And for this type of movie your main constraint is that it’s very much rooted in the current technological world.

David: Definitely. Case in point, on Zero Dark Thirty I believe we had CIA signing-off on how realistic our work was, and what we could say to be true. For instance, we looked on Google Maps to see what the compound where Osama Bin Laden was hiding in looked like, and they asked us to change it to a degree so that it wasn’t exactly the same. There was online photography of the helicopter that crashed, and we approximated it as a 3D model and, again, changed it a bit for fictional purposes. It was an interesting, if delicate, balance to strike. And of course we had a typical sign-off from the art department – normal sources other than the CIA!

Control center from Zero Dark Thirty. Courtesy of Territory Studio.

Aerial footage from Zero Dark Thirty. Courtesy of Territory Studio.

Kirill: Is that who you usually work with, at least in the initial exploration stages?

David: It varies from project to project. We tend to work with the art director and production designers in the early stages and we then work with set decorators, producers and directors during filming to make sure that the graphics look right for the actors. After that we work with the director of photography and finally with the VFX supervisor to make sure that the graphics are supplied correctly to the different post-production houses, to maintain consistency. In that case we’re normally delivering TIFF sequences or animation files of some description so that they can be composited in.

We can work closely with the director, art director or the production designer to define the overall style of graphics for the film. But those can vary in themselves. On Prometheus there was just one contained super sophisticated scientific research spaceship but we had to style graphics for each room. We had a particular UI style for the bridge, one for medical, and one for the social area – while distinct, they all still had to feel part of the same operating system.

On our most recently released film project, “Guardians of the Galaxy” we had 7 or 8 different sets, and each had to have a distinct UI style and a different experience for the actor involved. We’d get our style guides per planet or per spaceship – whatever it might be – then worked closely with Director, James Gunn, Production Designer, Charlie Wood, Art Director, Alan Payne and VFX team, Susan Pickett and Stephane Ceretti.

What’s interesting for us is that the graphic style for Guardians was, for the lack of a better word, more ‘human’, more ‘personal’ than for Prometheus. It all needed to feel used, more practical, as though it had a lot of history having come from different cultures and different interaction points. Sometimes it had to feel that there was a layer of technology, and then another layer of user interference, as though it had been hacked and chopped for the owner’s own purposes. It’s a lot of thinking about how characters might change or evolve their technology or software to make it fit their own needs.

Kirill: Do you have to be physically present there or can you do most of it as online collaboration?

David: It depends on the production. While we like a hands-on approach, Zero Dark Thirty was shot in Jordan, and it wasn’t the easiest place for me to get to. We sent all the files to our friends Compuhire, a company that program and deal with on-set playback, and have a great history in delivering the graphics for real on location or in a studio. We’ve worked with them since Prometheus, and are the wizards behind getting our graphics on-set in front of actors and directors.

With “Jupiter Ascending” and Guardians – and Prometheus to a degree – we had people on-set generating graphics in real time, so that the director could change something on the shoot. There are often opportunities to pick up on the decisions being made and render something so that it doesn’t get into the costly process of going into post-production.

This on-set approach helps us understand how an actor performs against graphics, how the graphics can feed into the script, and how it can contribute to frame and atmosphere. For example, the director might not always pull in close on our graphics but he might decide that it’s a background texture to be used by the cinematographer as pure kinetic lighting. But if we give him great graphics that look rich and credible and relevant, then the director can choose to use it to support the narrative.

Watch tower schematics for the Kyln prison sequence. Courtesy of Territory Studio.

Orloni Table graphics. Courtesy of Territory Studio.

Orloni Table graphics. Courtesy of Territory Studio.

Equally, there are plenty of things that you just can’t do on-set. I’m sure Ridley would have loved to have had that holographic table working for real but it just wasn’t going to happen. You go into post, and you create anything you imagine, and there’s a beauty in that – considering interfaces and interactions without any limit.

I like both methods. I like the on-set dynamics and direct interaction with the director, the cinematographer, the production designer, the art department – to make them feel that the set was designed as a whole, to know how it’s going to be seen when it’s shot. And equally, VFX is powerful and it’s lovely to be able to do crazy stuff that you can’t get real-world technology to do right now. It allows for a greater form of escapism.

Kirill: When we design and develop real-world software, our users are, well, our users. But in your case you have the director who is telling a certain story with the help of your visuals, the actor who interacts with those pieces on the set, and then me as a viewer who doesn’t want to be taken out of the story. Do you see this as conflicting forces, or different facets of the same puzzle from your perspective?

David: Ultimately all of those forces, or elements, are about effective storytelling. Motion graphics, on-screen or otherwise, is there to support the director’s vision and visual narrative, hopefully with some emotion and expression – that’s not something that is often brought to UI and UX.

My priority is to make the graphic performance and interactions with it believable, authentic, interesting, and with richness and depth. Sometimes it can be quite expressive; the audience or actor sees graphic buttons on a screen that look good and feel right for the context. In film, that’s more important – it’s all about supporting the narrative and the actor’s performance.

And sometimes it’s about enhancing the story with information. For instance, in Prometheus we had to create the UI for a head scanner, where the engineer’s head is analyzed. We researched different 3D medical scanners but for Ridley it was more about ‘feel’ than about clarity or realism of information, as we think of it today.

Concept artwork for head scanner sequence in Prometheus. Courtesy of Territory Studio.

Coral reef based visuals for head scanner interface in Prometheus. Courtesy of Territory Studio.

Ridley would talk about user interfaces from the perspective of emotion and fine art. He wanted it to light the scene and be an extension of the life and soul of the spaceship itself. The color schemes, the forms and the UI aesthetic within the Prometheus interfaces were based on underwater sea life like corals and anemones. There is a primal connection with those forms – a tentacle-like aura to them, making them organic, almost an extension of Giger’s work rather than ‘real’ as we know it today.

For Ridley, the original Alien ship was a freighter, a very basic raw piece of technology. His idea for Prometheus was an incredibly high-tech and advanced science and research facility that needed to be able to process a lot of data. That level of complexity and user interface ‘confusion’ is a tool that we sometimes use to our advantage in envisaging futuristic UI.

The projects we’re working on right now are about simplicity and purity of information but they still need a level of interaction and expressiveness that sets them apart from the ‘real world’.

Stock photography used as reference for Prometheus interface elements. Courtesy of Territory Studio.

Kirill: Were you also less constrained because it happens in some kind of a nebulous near future?

David: That’s a tricky one. As a user interface designer, I’m trained to think about information in contemporary functional terms but no one really knows how interfaces will look in the future. Combine that with a director who wants something that feels new and fresh and we have the opportunity to think and explore without constraints.

The beauty of this position is that we can forget about some of the rules that govern real-world applications and just enjoy exploring a speculative scenario. Often we find that it’s the really crazy stuff that serves the story better that brings most pleasure – as though the freedom from realism taps into some escapist need we all share.

But there’s a fine line between crazy and flamboyant that we have to tread carefully in user interface designs for films. There’s the temptation to be a little bit showy so that the director feels that your graphics add to the story enough to give it screen time. If it’s just some standard interface that you could see anywhere he’s probably not going to choose to show that over a very well known actor or actress. All the time we need to be mindful of supporting rather than hijacking the story – there’s a time and a place for exciting and fun and sometimes it’s just not appropriate to support the story and performance.

Kirill: Even if it’s not necessarily a very useable experience if you start analyzing it from the real-world perspective?

David: That depends on what type of movie we’re talking about and about how open one is to new ideas. On Prometheus we explored organic node forms in how data is managed and interpreted. While that seemed fantastical at the time, we’re now at a stage when neural networks are a reality. So maybe we were actually tapping into a scientific zeitgeist through our research.

Node forms for bridge interface elements in Prometheus. Courtesy of Territory Studio.

Node forms for medical interface elements in Prometheus. Courtesy of Territory Studio.

In-shot of node forms for bridge interface elements in Prometheus. Courtesy of Territory Studio.

And more recently, I thought the way that “Her” purely used voice to interact with the computer was a really beautiful way of balancing that need for theatrical performance and some sort of an emotional connection to the machine, while still being believable. Of course, we’ve seen voice interaction before – “Star Trek” and “Blade Runner”, for instance – and there is sophisticated software on the market today that lets a person control every aspect of their computer with voice, but because it hasn’t been picked up on by ordinary consumers, the concept as presented in Her felt fresh, contemporary and very ‘human’.

Kirill: Although you can argue that voice interaction or something like Google Glass doesn’t film well because it implies a lot of interaction and doesn’t have a lot of aspects that can be shown on the screen to support the story.

David: That is always the trick, especially when you talk about Google Glass and VR headsets. It’s a very personal space to ‘invade’ with cameras and it will be interesting to see how people start filming those interfaces and technologies. I’m sure they’ll come through – with the advent of Oculus Rift and PlayStation Morpheus it feels like we’re going towards virtual reality in a more mainstream way, and while I have ideas on how to show that from our work on “Killzone” and pioneering films like “Total Recall” and “Ghost in a Shell”, it’s still a whole new world and I’ll be interested to see what people come up with.

HUD interface overlays for Prometheus. Courtesy of Territory Studio.

Cryogenic pod interface overlays for Prometheus. Courtesy of Territory Studio.

Kirill: And you don’t want to miss the physicality of technology?

David: Even if the technical products we use today, such as TV’s, phone’s, stereo’s, etc, are increasingly ubiquitous casings with screens that run digital apps, and are incredibly convenient, there is a beauty and pleasure in the tangibility of well-designed physical products.

There’s a product design company called Teenage Engineering that has come up with some really beautiful sound engineering devices, like a MIDI keyboard that you can sample and write music on. It’s a beautiful physical device made out of metal, with lovely iconography design and screen graphics. It’s a pleasure to hold and use, and there’s something about that physical joy that is really appealing.

I remember when everybody went with mp3’s and digital formats for music and now there’s a real resurgence over here in the UK in the richness and ritual of LP’s and analogue music. There’s a real pleasure in enjoying the physical analogue world. I’d like to think that the relationship we humans have with technology and UI will become more human, more emotional over time – be that physical or intellectual, the dynamics are really exciting.

Kirill: As you get contacted to pitch for a movie, how does it go on your end? Do you have brainstorming session at your Studio, throwing ideas back and forth to shape the approach your pitch is going to take?

David: We’ve been very lucky to have been approached directly by most of our film clients, many times as the preferred partner.

The real test begins once the contract has been signed. And that’s a team effort between our art directors and motion designers that involves going through the script, researching ideas and technology, and using our imagination.

The script gives us leads to key moments where a hero piece of user interface might feature. Research is extensive for all our projects and involves exploring the latest thinking in real-world interface applications and design, and some fantastical interfaces; as Corning is doing with glass. For Prometheus we looked at how NASA present data to the public – we went to a high-tech medical facility, looking at what sort of technology is coming in the next 5-10 years. And we looked at MIT, Cambridge and other research institutions to see what sort of projects they were pushing through.

Explorations in color and movement for the bridge interface elements for Prometheus. Courtesy of Territory Studio.

We also look at the creative influences from the art and costume departments, so that our work is in tune with the set designer, the costume designer, prosthetics and others.

We make sure that the production designer agrees with our points of reference and our frameworks, and then it’s iteration after iteration of style frames. We start tackling key interfaces as benchmarks, and once we have those benchmarks in place for each set we might produce several hundred screens per set if it’s a really large one – smaller controls, auxiliary screens, life support, whatever it might be.

And if it’s not sci-fi based as with Zero Dark Thirty, it’s purely research driven. What is the military using, who are their suppliers, who do they go to, do they talk to a general radar supplier and what do their screens look like, can we go and take a look at one of those screens, can we film it, how bright does it burn? It’s a lot of research even for interfaces that are completely fantastical.

From this point in the research, we think about what sort of content will hypothetically be pumped through the screens, and then we think about our styling as the filter to pull it together visually. We also consider the type of command interactions, such as touch screens, voice or gesture control. What’s most interesting to me right now are the raw physical interactions that happen when an individual deals with technology that doesn’t quite work – it’s more interesting and natural for an actor to move something, push something, shake something around until it starts working. That’s what we do with technology all the time. If my laptop is not working, I’ll start mashing keys. Those frustrations, the reality of interacting with a machine when it doesn’t do what you hope it might do, there’s beauty and emotion in that.

And finally, when our work is done, we hope that the screens have come from a place that feels authentic and true to purpose.

Kirill: Things rarely break apart in modern sci-fi productions.

David: You think about “Star Wars” and how the Millennium Falcon was barely held together, and they made a joy out of that technology not quite working without having to bootstrap stuff together. There was a pleasure and warmth to that and it’s nice to see some of it coming back in some of the films we’re working on.

Kirill: There’s some of that physicality in Guardians of the Galaxy, right?

David: Definitely. We love the dynamics of human-technology interactions, and on every film we take stock of the main characters and think them through, to be able to reference them in the UI. On Prometheus, the ship was a character in its own right, but Guardians needed a more human treatment.

Working through the script with the director, production designer, art director and the VFX team, we looked at the main character’s history, personality, temperament, style, job, preferred weapons, relationship to his spaceship, always asking questions like ‘What’s the ship going to look like in the hands of this character, is it new or old, fast or slow, will it be messed up from previous owners, have battle damage, modifications, etc?’ Once we get the feel of the character, we began to express visually how his personal style could be referenced in the UI.

Our reference points were old airplanes, engineered metal and the lovely warmth of worn leather seats. So, how are screen graphics going to look in that environment? We gave the graphics a hard overlay, like the projections on glass in jets, but keeping true to the idea of a varied history – the glass was worn and scratched, with a bit of misty fuzziness to it.

And, we drew on Marvel’s own visual language to maintain consistency with the franchise but added alien elements to create a sense of ‘not Earth’, something uniquely Guardians.

We were always conscious about being true to character – be they human or alien – and crafting graphics that feel authentic to their context.

Left – screen graphic elements for the Kyln prison sequence (courtesy of Territory Studio). Right – the same elements incorporated into the final frame.

Kirill: Some of your productions are set in existing universes, from Bond to Batman to Aliens to Marvel. Do you need to be really well acquainted with what was done in those universes before you do not diverge too far away, but yet still have your own take on it?

David: Yes absolutely and we do it naturally. All of us in the team here love films and we have to, because we work long hours immersed in that world.

We put a lot of emphasis on knowing our research, and strive to present concepts that are true to a franchise. But, again, it’s tricky. While we’re mindful of maintaining continuity, we also try to find a sweet spot where there’s an opportunity to evolve, refine or re-interpret the visual language through a new context, character or technology.

We have been very lucky to have won the trust to be able to do that – maybe not always, but then there are times when it’s more appropriate than others.

Kirill: Do you find yourself wanting to reuse some of the graphical elements between different projects?

David: We are able to re-use some of the unique scripts or coding tools that we’ve had to write to create a special graphic effect but these are strictly on the implementation side.

We never re-use graphic styles, unless we’re working within the same franchise and then we keep them to maintain continuity, such as when we worked on Killzone 3 and Killzone: Mercenary – even then it’s more about evolving references for new contexts.

We’ve been very fortunate to split our work between sci-fi and thrillers. Each new film is different and each one is it’s own brand. Each one wants to differentiate itself. Directors pride themselves on trying to do something new. They are very aware of how the interfaces are perceived – almost as a brand element of the film. It’s important for them to have something original and ownable.

We like to build human experiences in the interfaces that we create at Territory. The nature of our on-set work means that there are certain technical restraints that don’t allow us to create hugely detailed pieces of work. The necessity to be responsive to change and think on our feet means that a lot of our work is quite loose and often quite abstract. We’ve learned to celebrate and enjoy that. It means that the work can be varied in look and approach.

Explorations in color and movement for the bridge interface elements for Prometheus. Courtesy of Territory Studio.

Kirill: When do you get to see how your pieces are integrated into the movie?

David: Normally a couple of days before the general release. If we’ve done a good job, hopefully somebody from the art department or VFX will get in touch to invite us to cast and crew screening. So we have a little bit of warning to tell our families whether they should go and see it!

If we’re working on VFX shots, we have an idea of how they might look in final, but even then they might get cut. You’re never quite sure what made it and what hasn’t, and it’s always a lovely surprise to see a lot of your graphics in there.

Kirill: Do you worry about how dated your interfaces are going to look like a few years after the movie is out?

David: We have to plan on our work dating. We get an opportunity to do super-stylized work, and there is no way it’s not going to look dated in two year’s time, let alone ten year’s time. We try to create a cultural moment that is true to the zeitgeist of the day, and then it dates. And, if we’re super lucky, the graphics become design history, like Ron Cobb’s work on Alien.

I don’t resist looking dated but I do resist looking boring. If there’s an interesting idea or an interesting story point, it will last. And hopefully it has enough interest and humanity to it that when it does date, we remember it with a bit of joy.

Semiotic standard iconography for Aliens by Ron Cobb. Courtesy of Territory Studio.

Extending the original iconography into the visual language of Prometheus. Courtesy of Territory Studio.

A detailed look into specific icons of Prometheus. Courtesy of Territory Studio.

Kirill: And going back to the variety of work that Territory is doing, everything is feeding back into everything.

David: At the moment we have product designers making products, graphics designers doing print work, developers writing app code, editors cutting live-action footage, traditional animators doing hand-drawn animation, and UI designers doing film and game graphics. And all of these people talk to one another, and that’s really interesting to me. A product designer informing a traditional piece of animation, and an animator talking to a UI designer on how animation might inform design.

In terms of UI we’re always encouraging designers to think in sketchbooks first, to think about what they want to achieve. Just the act of drawing without a technical aid seems to liberate the imagination. When a film director comes up with the script, he is not thinking about what VFX can do, or the limitations of costume. He’s thinking about what image does he want on screen and how does he want to tell that story.

We’re trying to work in the same way here – thinking about the interaction and how is that going to tell the story. The cross-pollination of ideas across disciplines is really rich and definitely influence user interfaces for films that we do. We don’t just come at a creative challenge from a UI perspective. We have the freedom to look at the challenge from the point of view of other disciplines and that’s quite special.

Interacting with bridge screen interfaces for Prometheus. Courtesy of Territory Studio.

And here I’d like to thank David Sheldon-Hicks for the wonderful work Territory Studio is doing on screen graphics and movie user interfaces, and for sharing the background materials for the interview. Guardians of the Galaxy is playing in theaters now, and Jupiter Ascending will be out in early 2015.