Behind the scenes of an immersive sci-fi graphic novel for iPad, the creators of Upgrade Soul talk music, motion and motivation

Upgrade Soul is the new digital comics project from artist Ezra Claytan Daniels, developer Erik Loyer and composer Alexis Gideon. It’s a fantastic sci-fi graphic novel in app form with it’s own unique audio soundtrack it tells the story of aging couple, Hank and Molly Nonnar who are given the chance to extend their lives by upgrading their souls. But all is not as simple as it might seem and the events play out with exquisite, highly detailed artwork from writer/artist Daniels and moody atmospheric music from Gideon. It’s so much more than just a page turner though with a fantastically readable user interface and motion graphic elements within each page that help it really come to life. Upgrade Soul is an absolute gem of a title, so we got in touch with Ezra, Erik and Alexis to find out just how this fantastic project came into being.

Tell us about what inspired you to create Upgrade Soul? Was it a desire to create a fantastic comic, an awesome app and or something completely new?
Ezra Claytan Daniels: Upgrade Soul came together from both ends of this spectrum. I’d been working on the story and preliminary designs for years before Erik approached me with the idea of developing it through Opertoonity. I think this is part of what keeps the presentation so subtle–it was born as a very straight-forward narrative. If we started with the technology and built a story around that, it would’ve become a much different thing. I’m very interested in exploring something like that in the future, but for me, Upgrade Soul is all about telling the best story we can, and using the technology to serve that.

Erik Loyer: Since the late 90’s I’ve been very interested in what happens to the visual language of comics when it goes digital, and in collaborating with Ezra I’ve been fortunate enough to explore different facets of that over the years. A few years ago it started to become clear to me what a good digital comics engine would need to do and how certain problems could be solved, and so once Ezra showed me the first chapter of Upgrade Soul, the strength of the story and the illustration immediately jumped out at me and I started thinking “how can we make this work?”

Who was the driving force behind the collaboration and did you look at other titles for inspiration, if so which?
ECD: I admit I wasn’t exactly the first person to want to jump into the digital publishing experiment. I probably wouldn’t have seriously considered releasing Upgrade Soul as an app if it weren’t for Erik. I’ve been an enormous fan of his work for a long, long time, and I have the utmost trust in him as a collaborator. I kind of just closed my eyes and leapt when it came to the decision to do this. When we started, almost three years ago, there really wasn’t a lot of precedent. Over time, other teams that had been thinking along the same lines started releasing apps, which was really cool to see. The two main apps we found impressive during development were Operation Ajax and Nawlz, both amazing examples of what can be done with the form when taken seriously.

Your approach feels very similar to the Madefire app what do you think of them and how do you think they compares to Upgrade Soul? What do you think this approach offers in comparison to digital titles released via app stores like ComiXology?
ECD: I actually hadn’t seen the Madefire stuff before you mentioned it, but yeah, it is similar to what we’re doing. Great presentation, too. There are so many comics coming out for mobile platforms, it’s hard to keep up with everything! Bane of Yoto is another nice one I just stumbled across–they do a very similar parallax effect. Deep Comix also does a terrific parallax effect, but they’ve yet to release any content beyond the tech demo as far as I know. We’re absolutely thrilled every time we see a new tablet-native comic that really takes the technology seriously. The more apps there are like this, the bigger the audience will get for them, and the better the content will get.

I think what Comixology is doing is somewhat different. Their model is great, but they’re starting with existing print-based comics and essentially wrapping them in a souped-up PDF reader. I don’t think Comixology is the future of immersive comics–it’s more of a transitional technology designed to make publishers’ existing catalogs available on mobile devices. We didn’t really look too much at that as an example of pushing the technology further. I think Double Feature is actually doing some cool stuff, though, with their DVD-style extra features like commentary and the ability to view the art in pencil sketch and black and white modes on the fly. This is great example of something that is still very print-based in design, but adds something that would be very difficult to do in print.

EL: It’s interesting because I see the field of experimental digital comics stretching towards this realm that isn’t just Comics Plus X, it’s a synthesis of a whole range of audiovisual storytelling techniques, anchored by the basic principles of comics. You see works like Stevan Živadinović’s Hobo Lobo of Hamelin and Daniel Burwen’s Operation Ajax approaching it from the content side, and other works like Daniel Benmergui’s Storyteller and Jason Roberts’ Gorogoa approaching it from the systems and programming side, not to mention the history of electronic literature going back decades that is largely unknown to the general public. It’s the point at which those streams meet that some really interesting things are going to happen, and some of its lineage will be traced back to comics, but I think its implications for media in general will actually be even more broad.

The artwork, look and feel really reminds me of the work of Geof Darrow and the story reminds me a bit of Akira, what artists and creators do you admire and who inspired you with your work?
ECD: That’s a huge compliment, thanks! I’m a big Geof Darrow fan. As far as comics, I’m in love with 80’s European stuff like Vittorio Giardino, Enki Bilal, Francois Boucq, and of course, Moebius. I also drew a lot from Junji Ito and Katsuhiro Otomo. I think I actually draw more inspiration from movies when it comes to panel composition and story, though. David Cronenberg and Andrei Tarkovsky are my two biggest inspirations for Upgrade Soul, particularly The Fly and Solaris. I also have a background in medical illustration, and one of my goals with the art on Upgrade Soul was to make everything very clean and clear. I didn’t want to fall back on effects or textures or shadows to cover my tracks. I wanted it to be totally apparent and honest. This is as good as I can draw, and if something isn’t drawn well, I wanted it to be on clear display. To me, this was important in telling the story, which is all about identity and the face you put on versus who you really are.

How do you plan out each page and transitions? Is it collaborative or do you send finished pages to each other to work on each stage individually?
ECD: I have to give credit to Erik for designing the panel transitions. We don’t really plan much out in advance. I’m pretty much focused exclusively on the story and art, and for the most part try not to think about panel transitions or effects unless I come up with something I think is really cool. I then send Erik layered Photoshop files of each page and he just does his thing, which is really like magic to me. In fact, we all work fairly independently, I would say, including Alexis. I try to make everything as easy as possible for Erik to do his thing and have some freedom to toy with his own ideas. We all converge once an initial pass has been done, and hone each scene from there.

EL: Ezra has been very generous in creating space for me to play, and he has good instincts for digital production, which helps a lot. A big part of getting the first couple of chapters of Upgrade Soul out the door was just the learning process of “how do we do this,” but now that the production is hitting its stride, I’m looking forward to seeing how we can continue to push the format creatively while remaining true to the overall vision—there’s a lot of exciting territory to explore.

The app has a rich, lush soundscape, how did you go about creating it? Is it gone once the pages are finished or do you create pages to match a kind of mood of music?
Alexis Gideon: We went at this from a pretty philosophical place, and had many long conversations about the line between comics, animation, and interactivity, and where we wanted to fall. With Upgrade Soul, the first thing we decided was that we didn’t want to do a motion comic, and by motion comic, we meant any comic that attempts to approach animation with movement and sound effects.

EL: It’s wonderful to have material as rich as Alexis’ to work with. I’m very keen on musical interaction at the level of the individual note, because through the intricacies of pitch and rhythm you can add additional layers of storytelling. The Opertoonity engine allows for musical events that occur in sync with the story, with the user’s actions, or with the underlying tempo, and even (as you hear in a sequence at the end of Chapter Two) in counterpoint across panels. Alexis’ music has a sonic richness and personality to it that lends itself to a variety of arrangements, and both Ezra and I come up with ideas for how to use the score in specific scenes. Once the original source tracks are released, I think it’ll be interesting for people to play musical archaeologist and try to figure out where this or that element came from!

When it comes to sound on an interactive comics/books there seems to be no right answer about what is best, but you have chosen to go for mood music rather than sound effects and dialogue. Why was that and how far do you think sound can go before it becomes obtrusive and less like a comic?
ECD: One of the fundamentally great things about comics is that the pace of a story is really a collaboration between the author and the reader, and motion comics miss that with scripted animations, sound effects and voice-over work. Suddenly, the reader doesn’t have any room to react to the author’s intent–they’re just being muscled through it by the effects. And while there are a lot of motion comics that do some really amazing things with animation, like some of the Marvel stuff especially, it’s still not animation, and the moment something moves onscreen it sets up an expectation in the reader that it is.

I think one of the things that people find disappointing with motion comics is that this expectation is, by nature of what it is, repeatedly dashed. Similarly, the quality of sound effects and voice-over work in motion comics typically isn’t on par with animated productions and so, again, there is a dashed expectation. Then, if it approaches certain level of polish, like the Marvel stuff, you have to ask, why didn’t you just make this an animated film, and market it that way? What we’re saying is that comics are not animation, and we’re not trying to make our comic more like animation. With Upgrade Soul, almost every element of art that moves, and every element of sound that changes is tied dynamically to user input, bringing them back into the fold and making it an intimate experience again.

EL: There is a very specific temporal logic at work in Upgrade Soul, and that extends to the sound design. I find a lot of times in motion comics or comics with added sound effects that by the time the effect plays, I feel like I’ve already “heard” it—the literal nature of the sound design ends up feeling to me like it’s doubling the visual storytelling unnecessarily. Musical sound is inherently abstract, and as such it seems to combine more fluidly with the visual storytelling of comics, which is already heavily stylized and abstracted from reality. I’m also very big on the idea of story as instrument—that’s the core idea behind Opertoon as a publisher—that these are performative experiences in which users are constantly active, even if only in subtle ways. I’m less interested in choose-your-own-adventure-style interactions in which the player chooses A, B, or C direction for the plot; what’s fascinating to me is that almost unconscious constant tuning and tweaking that goes on between a musician and the instrument they’re playing. Upgrade Soul has some connections to that in the way that your touches can play the notes of a melody and your tilting lets you explore the parallax in a scene; it’s about a constant low level of expressive interaction rather than these big dramatic moments.

The same can be said for animations and transitions. You’ve kept it all quite simple so far, but with neat little touches like flickering lights and blinking characters, how far do you think interactive comics can push the boundaries before they go too far away from what a comic actually it?
ECD: It’s funny you mentioned that because the flickering lights and eye blinks were one of the things we debated the most. We even cut out some other motion effects at the last minute. In concept, they fundamentally go against our core comics philosophy, but honestly it just came down to the fact that, whenever we showed the prototype to someone, they would always comment on how cool it was that the characters blinked and the screens flickered. The decision to keep these in was purely superficial.

EL: For me the temporal logic extends to these elements as well—you may have noticed, for example, that there are some panels where characters don’t blink. This is usually because the amount of ambient animation a panel can handle without feeling weird is directly proportional to the amount of time represented in the panel. If the panel is depicting a split-second moment of time, then blinks feel out of place. So there’s a real push and pull to how much motion you can introduce. In Upgrade Soul a lot of that is also driven by the kind of storytelling that Ezra is doing, which has a very deliberate pace and makes frequent use of repeated compositions within a small number of panel configurations—all pretty counter-intuitive choices in a world of “enhanced” comics that seem to be working so hard to keep our attention. The positive responses we’ve been getting to Upgrade Soul have been especially gratifying because they confirm what we’ve suspected: that more, faster, louder isn’t the only way to create a compelling experience.

Finally, does Upgrade Soul have a planned out story arc and what can we expect from future issues – more fancy effects for example?
ECD: Upgrade Soul will be 18 chapters total, around 300 pages. It’s all written and just over halfway scored. Completing it is mostly just a matter of drawing and coding each chapter. Each chapter takes about a month to build, and I’ll have to stay a month ahead of Erik on the art so he can code the next chapter while I draw the one after that. We will be adding more interactive and immersive effects as we get more comfortable with the process, and get feedback from readers, but we won’t diverge from our core, comics-reverent philosophies, we promise.

EL: Clutch Cargo. It’s all about Clutch Cargo, baby.

Upgrade Soul is available to download from the iTunes Store. It’s currently free, but will soon be increasing to $1.99 or $6.99 for a three issue subscription to the monthly updates. You can find out more about Upgrade Soul at their website or you can visit Ezra’s site Dream Chocolate, Erik’s site Generous Machine or Alexis’ site.