Abstract
Bob Sabiston has been working in animation since the 1980s, when he studied at the Media Lab at Massachusetts Institute of Technology. In this conversation with Paul Ward, he talks about his development of the software with which he is most associated, the digital rotoscoping program Rotoshop, as well as the artists and animators who have influenced him. Central to Sabiston’s work is an interest in the everyday and how animation can capture and creatively treat it. Any discussion of animation and realism, or animation and documentary, arguably has to engage with his work. Rotoshop’s often misunderstood status as a form of image filtering rather than a sophisticated form of digital mark-making means it also goes right to the heart of debates about how we define animation, what constitutes ‘proper’ animation (as opposed to some form of ‘short cut’) and how we view different kinds of animation labour. Although Sabiston is most associated with Rotoshop films, he is also active in the development of software for other platforms.
Bob Sabiston and his company Flat Black Films are mainly associated with the films produced using the Rotoshop software, a form of computer rotoscoping. Developed during the 1990s, Rotoshop was first used for a series of television interstitials which went on to be compiled as the short film Project Incognito (1997). Other shorts – Roadhead (1998) and Snack and Drink (1999) – followed, which explored a similar fascination with human interaction and the weird randomness of the everyday. The oddly shimmering, floating aesthetic imparted by Rotoshop, where live-action footage is drawn over using a computer and Wacom pad and pen, appeared to lend itself to this kind of baroque, yet seemingly improvised form of documentary. Rotoshop is based on simple, gestural mark-making (with the attendant ability to capture the immediacy of movement, character and subtle nuance) whilst at the same time being a complex form of computer animation. 1
Although Rotoshop has since been used for feature-length work such as Waking Life (2001) and A Scanner Darkly (2006) – it was notable in one of the sequences in Lars von Trier’s The Five Obstructions (2003) – Sabiston has continued to have an interest in using his software to interpret real people and situations. This puts his work at the forefront of one of the most vibrant areas of inquiry – animated documentary.
Most people will associate you with Rotoshop and the films made using that process. Can you say a little about your background in animation, as well as the background to the development of the Rotoshop software?
Rotoshop was developed primarily as a reaction to the type of animation I had been doing for several years out of college and through 1994. That was laboriously constructed CGI, hand-drawn 2D animation on top of 3D-rendered scenes. At the time, no one had done much of that. I was using a hand-drawn animation program – which did not yet have the interpolation feature seen in Rotoshop – I’d written for my second film Grinning Evil Death (1990), as well as my own 3D modeller in conjunction with the software-renderer developed in MIT’s Media Lab animation department. The computer was an HP UNIX workstation I got from John Whitney Jr’s Hollywood company, USAnimation, in exchange for my software. The resulting film, God’s Little Monkey (1995) won the Prix Ars Electronica Golden Nica 2 for that year, and I knew the look and process were special, but it was just too time-consuming.
After the success of God’s Little Monkey I despaired of ever completing any long-form animation projects. The 2D/3D thing was just too much work for the results. At the time, I was working completely by myself in a rented studio space. I was a perfectionist about the look and didn’t trust anyone else to do it. Besides, I had no money to pay people. In 1996, I was really growing concerned for what I would do with my life, when another MTV contest came along; I’d participated in two previous ones. That jump-started me back into animation.
What kind of material did you come up with for the MTV competition?
I was interested in documentary filmmaking and decided to do some animation exploring personality and expression. I figured I would just shoot video of some interviews and try to elicit some funny or illuminating moments from conversation with random people. Then I would ‘animate’ a few select scenes by tracing the video. Easy and simple compared to the work I had abandoned. Well, it turns out that in late 1996 there was no clear, easy way to do rotoscoping on the computer. I had trouble finding software to do it. Photoshop had the ability but required each frame to be opened and saved as a separate file, whereas I had in mind a fast, very life-drawing approach. I just wanted to quickly and gesturally sketch out the facial expression on each frame. With about two weeks before the contest deadline, I dusted off my programming skills and wrote a quick program for drawing black, horribly jaggy lines on top of the video frames.
My first test of a scene was encouraging but took an extremely long time. In thinking of how to speed things up, I realized that really I was just drawing the same lines over and over in subtly different positions. That gave rise to this interpolation feature, where instead of drawing a whole face on each frame, I would just draw a single line and then move forward to the next frame. I could skip over frames that changed only slightly, and only draw a new line whenever I judged that the video had changed enough to warrant one. I used spline-based interpolation to ‘make up’ the intervening frames. That was really the genesis of the Rotoshop technique. The step of prioritizing time over space in the drawing technique resulted in a smooth-looking but realistic animation. The contest entry won second place, which led to 9 months at MTV developing a string of 25 interstitials that used the technique. I called the compiled collection Project Incognito (1997). Upon leaving MTV, I made Roadhead (1998) with basically the same look. It did well on the film festival circuit and we were off to the races.
It’s interesting that Rotoshop’s development appears to be clearly driven by an attempt to synthesize two very powerful things – the processing ability of the computer and the interpretive ability of the artist-animator. I can certainly see your references to ‘a fast, very life-drawing approach’ and working ‘gesturally’ striking a chord with animation students everywhere. In this context, it would be interesting to hear which artists and animators have most influenced you.
My formative years creatively were all about drawing and painting – a traditional visual arts training. It was natural to think of life-drawing, especially quick gestural drawing, when considering ‘animation as art’. I think this, rather than any previously-produced rotoscope films, like the Fleischer Brothers or Bakshi, is what interested me about it. It’s just the idea that with a few quick strokes you capture something of the subject, and seeing many of those drawings in motion should be revealing as well. However, I must say that I loved Bakshi’s Wizards (1977) as a teenager – but I think that had more to do with the spirit of Bakshi than the technique of rotoscoping, which he seemed to use in a way that seemed like obvious corner-cutting.
I had a job in high school driving around between county libraries and showing film projections of old Disney movies. Watching Cinderella (1950) and especially Alice in Wonderland (1951) every day for a week probably did something to me. But mostly it was the stuff shown on MTV, like the early logo interstitials of Henry Selick, and then the travelling short compilations of work from Bill Plympton, Marv Newland and the like which really struck home with me. It was something one person could do.
What about computer animation more specifically?
When I was in college, I saw Pixar’s Luxo Jr (1986) on my first trip to SIGGRAPH. It was a hugely influential film for me. It compelled me to write a whole software animation system during my junior year and make a short film (Beat Dedication, 1988) to submit to the conference. John Lasseter was the jury chair that year and he told me that he personally had favoured my film, so that was a big encouragement. By the time I got into rotoscoping, however, my heroes were [live action] indie film directors like Richard Linklater, Hal Hartley, Whit Stillman and Errol Morris. Slacker [directed by Linklater in 1991] changed my life – it portrayed the antithesis of the MIT geek’s sequestered laboratory existence. During college I also went to a screening of Miyazaki’s Laputa: Castle in the Sky (1986) which had a great impact on me. The plot was so free, so open and strange. I really had never even paid attention to Japanese animation before, though people said my animation style resembled anime. Personally I was trying to draw with more of a Tim Burton aesthetic.
How do you see Rotoshop in relation to the original rotoscoping used by the Fleischer Brothers, Disney and others in the studio system?
Well, the idea of tracing live action to achieve animation is just something that naturally occurs whenever that quantity of drawing is involved. It is a time-saving device and one for capturing the type of realism that is very hard to do with ‘true’ animation. I got into it for probably similar reasons to the Fleischer Brothers, Bakshi, and anyone that has ever tried it. However, the use of interpolation changes it into a more powerful thing. ‘More powerful’ in the sense that using interpolation is immensely freeing. Your work no longer has that jittery shimmer that comes from small line variations from frame to frame, so it looks smoother and a little otherworldly. But what’s especially interesting is that since you know the computer will smoothly morph between what you faithfully trace and what you make up, you are free to move off into pure animation, change the design, engage your imagination and it will not be clear where that departure has taken place. It’s difficult to describe but, simply stated, it allows you to move fluidly between plain tracing and what may more properly be called animation.
It’s interesting that you identify the idea of a ‘time-saving device’ as this, for me, is central to how rotoscoping is viewed in a negative way by certain animators and critics: it’s seen as a form of cheating, like animation always has to be a laborious slog to be ‘proper’ animation. Ironically, of course, Rotoshopped animation does, still, take a long time to produce, simply because it is not just tracing; it is a complex aesthetic interpretation of the footage. This is something that animator George Griffin (2002: 11) pinpointed beautifully in a review of Waking Life, where he said ‘it looks like what may have started off as an LSD (labour-saving device) ends up being a perfect trope for a universe unhinged.’ I like the way he synthesizes process and aesthetics in that remark.
That’s a great quote – I’d not heard that before. To me, Waking Life was primarily Rick Linklater seeing in our style of animation a means to make this crazy movie he had in mind. The blueprint for how radically this type of animation could differ from the Bakshi style, or that seen in the video for A-ha’s Take on Me [animation by Mike Patterson, 1985] I think we established with the two projects we did previously in 1999, Snack and Drink and Figures of Speech.
Honestly, from the very beginning, as soon as I saw the very first test with the interpolation, I got totally excited because it was something that looked a lot better than the purely frame by frame approach, but it was easier for the most part. For the first few years, from Project Incognito to Waking Life, we still used old-school frame-by-frame rotoscoping in addition to the new ‘interpolated’ technique. Many scenes were still done on the computer but without any interpolation – just tracing, in other words. It is easier to just draw naturally and not think about the sequential, methodical drawing requirements imposed by the interpolation. It’s also just another ‘look’ in the arsenal, so to speak. We had to taper off on it from about 2002, because we started doing everything at the full frame rate of the media, for instance 30 fps or 24, instead of 15 and 12. Perhaps surprisingly, Waking Life is animated at 12 fps. Doing 30 fps frame-by-frame is pretty brutal, workwise, but sometimes we’ll still throw it in there.
So, despite interpolation being ‘labour saving’ in some ways, it can still be a lot of work?
The interpolation feature as implemented is a double-edged sword. By skipping frames (tying a single brush mark together with its kindred on other frames, and just calculating in-betweens for the frames between these ‘fenceposts’) there’s a smoothness due to the mathematical blending performed by spline interpolation. You can draw every third or fourth frame, approximately, and get lifelike, realistic results. But, in painful violation of the traditional artistic process, you have to draw one single line of your scene – say, one eyebrow – and carry that on throughout the entire scene before moving on to eyebrow two. Needless to say, that can be a problem as it is contrary to artistic nature. Artists are not used to keeping track of their lines mentally – their order and direction. It takes getting used to. That said, I think it’s still a far more natural and spontaneous process than that required by 3D CGI, which is filled with enough of these left-brain-trumps-right-brain software issues to make it downright artist-unfriendly. However, it has been a long time since I worked with 3D animation, maybe it has gotten easier.
Despite the fact that to do it properly, rotoscoping (and Rotoshop) take a great deal of skill and artistic ability, one thing I’ve found consistently is that there is a discourse around rotoscoping that sees it as ‘not proper animation’. I certainly found instances of this discourse being used in relation to Rotoshop when I was researching another article on this topic (see Ward, 2004). Some people even reject the idea of Rotoshop being animation at all. How do you feel when you are labelled as ‘an animator’? And how do you feel when your work is dismissed as ‘not animation’?
Well, I think of myself as an animator, but I understand how rotoscoping is not truly ‘animation’. I guess because I started out doing ‘true’ animation, and because the end result when I am working in Rotoshop looks essentially like traditional animation, that I still think of it in those terms. People dismiss work for lots of reasons. It’s perfectly reasonable to value making something up from scratch as a skill and an art, and this is obviously something that we are not often practising with rotoscoping. However, what we are doing is sort of a new thing, it is not just tracing. It is taking features from the live video and extending them, amplifying, commenting upon, even veering away into ‘true’ animation.
The one thing that is kind of funny to me is that people now think of the ‘Rotoshop process’ as being nothing more than the Scanner/Schwab 3 look: the realistic thing. They don’t really know about the 10 years of crazy shit that led up to it. It makes me kind of excited to go back to looser, non-realistic material like in Waking Life and the early shorts, which we will probably do for our next project. Our last film, the 23-minute The Even More Fun Trip (2008), was supposed to be like that, but we found that it was really difficult to shake the ‘realistic’ bug when you’ve got it. It’s just so satisfying to get all that detail in there. It didn’t help that the subject was a detail-filled theme park with people, colours and rides everywhere. By the end of that project, I think people were loosening up again but it was interesting how difficult that was. It felt almost like cheating to just go quickly and be more abstract again.
As far as the classically trained animators’ view, I think they see Rotoshop as cheating. A lot of them might even think it is just done with filters. All I can say is, what we do just isn’t the same thing as traditional animation. I think of it as animation because it looks like animation. But it is something different. It is more about a painting aesthetic, and capturing very real performances in a way that traditional animation cannot. It is about one person being able to do a lot more animation in a certain timeframe than would ever be possible the old-fashioned way.
When my program is used the ‘right’ way, I think it carries with it an acknowledgement that it isn’t really animation. That’s kind of what’s remarkable about it. You can tell it started as video and then some person added their stamp, or twist, or embellishment, to it. Maybe it is more like video art. However, several of the scenes in my film Grasshopper (2004) were done using a technique where I would place the live action on one side of the screen, and then on the other side I would draw the animation. So they weren’t traced at all. Those scenes are probably my favourite in the film because they are very rubbery and loose, but they still capture some of the expression and emotion.
All of us working on Rotoshop over the years have become a bit dissatisfied with the trend towards photorealism in our use of the technique. It’s not something we explicitly chose, it is just something that curiosity seemed to demand. Everyone wanted to see how real we could get. You see a similar thing in 3D CGI, and in painting too, and ultimately people move beyond it and into some more exciting territory.
Yes, a lot of the Rotoshop films appear to be exploring notions of realism in animation, but looking to push those boundaries in some way.
The whole development of Rotoshop has definitely had an arc that tends towards the photorealistic. Scanner and our Schwab adverts are about as far as I can see taking that. When I look back at my MTV work, Snack and Drink, or Roadhead, it seems like we were doing a very different thing than what we typically do now. Then it was more about the animators, how can we inject creativity and invention into this? Lately, it has become more about the polish of the image, and faithfulness to the actor and their performance. I’m not saying one is better than the other, but when you are in the photoreal territory, that is when you really start getting these assumptions from people that it is all done with filters. You can find people online who swear that there is actual live action in parts of Scanner, but to my knowledge there isn’t, and I don’t see it when I watch it. And at some point it is definitely fair to ask ‘why bother animating this? It looks almost real.’ But, for me, it’s like photoreal painting, like Ralph Goings or Richard Estes. It’s cool that people can do that. Sure, why not just take a photograph? But it isn’t a photograph, it’s something someone did by hand, and if you pay attention you can see their aesthetic at work. My own paintings before I became interested in abstract art were directed toward that goal – for example, in high school I painted a big 8 x 8 ft reproduction of a Charles Bell gumball machine.
The way that animation mediates performance is currently quite a hot topic – and a controversial one. 4 Although the motion-capture and performance-capture types of animation are different from Rotoshop, there is a similar debate around how the animation interprets, embellishes (or distorts?) the ‘original’ performance. What are your thoughts on this, especially in the light of what you say about ‘faithfulness’?
The comparison is valid of course and it makes sense in a way, but in another way there is no comparison at all! With motion capture and the like, you are having the machine record something that becomes the spine, even the heart, of the animation derived from it. With Rotoshop, you are hand-drawing the expressions and forms that you see. It is usually traced, yes, but even then you are starting from something hand-drawn. There cannot help but be the smallest stamp of the artist in every line. From the very beginning, before it even enters the computer, the artwork is coming from someone’s hand. The computer assistance happens afterward, and it springs from your artwork. That’s very different in my eyes.
One could argue that motion capture data serves the same purpose as the video that we trace, but I think there’s something important about the fact that the ‘recording’ with our method is done by a person and not a machine. There is also just the primacy, the human element of drawing. A hand-drawn picture of Jimmy Stewart is likely to be much more charming than a 3D model of him. And 10 different artists will have wildly different takes on that, each one with a true style. Not so easy with 3D or motion capture.
In many of your films, you interact with real people, getting them to ‘perform’ – whether this is the interviewees of Roadhead or engaging with Ryan in Snack and Drink and The Even More Fun Trip. You mentioned earlier that you have an interest in documentary filmmaking. Can you elaborate on your interest in representing reality via animation?
The power of this rotoscoping technique to capture and ‘filter’ reality – but not in the machine-sense – is still captivating to me. You could say documentary is filmed reality, and so therein lies the bond between the two. For me, documentary is easy as well, as compared to writing a fictional script; ultimately, I am really mostly interested in the visual field, the motion and the colour. I feel that what I am looking for with this type of animation can be found in almost any reference/source material. So it is not so much an interest in documentary as in the ease of obtaining the source material. As a bonus, you get the aforementioned ties between tracing as filtered reality and documentary as the same.
I can certainly see what you mean about the ease of obtaining the source material – though I wonder if this is more to do with things like immediacy and getting hold of footage that has an ‘off-the-cuff’ feel to it? And then mediating this in a way that makes the viewer ‘see it anew’, as it were?
Yes, that’s definitely a huge part of it. The idea that you could easily run out and quickly capture anything, then use this animated lens to reinterpret it, amplify it and comment upon it. It owes its roots perhaps to Aardman Animation’s landmark Creature Comforts (1989), where they recast audio interviews with various people on the street as animals in a zoo. It also borrows from Errol Morris’s work where he seems to just be taken with weirdos and wants to show you just how weird they are. I was never interested in ‘documentary’ per se, but when I saw Gates of Heaven (1978) and then Vernon Florida (1981), it really sparked an interest in me. Using animation to do this kind of thing is especially effective. You can caricature people, but you also protect them with this anonymizing ‘screen’. People have remarked to me several times that because they cannot see the actual person, they end up listening more closely to what the subject is saying.
As well as continuing to develop Rotoshop, I understand that you are working on animation software for other platforms. Can you say a bit more about those projects?
Whether or not Rotoshop leads to further development and growth as an animation tool remains to be seen. I hope so, but my interests have branched out over the past few years. I’ve become much more conversant with my programming side, having spent 5 years developing Inchworm Animation, an animation software for the Nintendo DS. Inchworm Animation’s journey to publication is an epic tale in itself. More recently I’ve become completely enamoured with Apple’s iOS and the app phenomenon. I’ve got two apps, ‘Headspace’ and ‘Voxel’, and I’m working now on a videogame. The whole app-store idea really democratizes software publishing and is tremendously exciting to the solo programmer.
Inchworm Animation sounds interesting.
It started out as a weekend hobby – an experiment to write a tiny but professional animation system for a handheld device. For years it was an ‘old car in the driveway’ kind of thing. During that time I approached many software publishers with it and almost signed a contract with Disney Interactive in 2008. Then for a couple of years, 2009–2010, it became a full-time job as I became serious about publishing it myself. Over the long and winding course of its development, I learned a lot about commercial software development. I learned how bad a programmer I can be. I learned how little interest exists in the videogame industry for animation software. During that time, the iPhone fell from the sky and has now eclipsed the Nintendo DS almost completely.
I initially thought the project might take 6 months. Five years later, it is finally a real thing, being bought by a satisfactory number of people who are enthusiastic. I am readying the European market version. Although I never got a PhD, to me this project felt like the equivalent of the graduate student whose thesis has overtaken their life. They cannot move on until they finish it. It was like that. Like college, it wasn’t the most pleasant experience, but it was thrilling at times and I’m so glad I finished it. And actually I would love to release a version for the new 3DS with its glasses-free 3D. The project deep-down has always had great power for me. As a child I think I would have loved this thing. And though the iPhone has gained ascendancy, I will always prefer drawing with an instrument, a pen or stylus, to drawing with my fingers.
And what about the iPhone apps?
I first started to write an outlining program on the Mac years ago. Then when the iPhone came along, the whole app phenomenon was exciting, especially in the face of the harsh software publishing world I’d been experiencing with Nintendo. Anyone could write a piece of software for this cool gadget, sell it for whatever they wanted (even a dollar!), and get 70 per cent of the proceeds. It was so simple and easy. ‘Headspace’ is the first iOS app I wrote, in 2008. For a long time I have been interested in outlining, where you have a topic, sub-headings, ‘children’ of those headings and so forth. Hierarchical lists, in other words. An outliner seemed like a very useful thing to make, especially if I could incorporate 3D graphics so that you could pinch, zoom, rotate the thing in space like a big data model. So I spent a few months diving into the iOS, objective-C and openGL world. Thanks to appearing relatively early and receiving some good reviews, ‘Headspace’ has done well in the store. I would love to make a sequel because so much more could be done with the basic concept. But I don’t know if I will find the time. I hope so. My other app, ‘Voxel’, is an interactive pixel editor, like ‘digital Lego’ – a low-resolution construction/sculpting program. 5 It has been exciting to see what people make with it; there is an online gallery built into the app so that people can share their work. Earlier this year, I even added animation tools to it though not many people use those – it is a laborious process.
Whilst you have been developing Inchworm Animation and the apps, what has been happening with the Rotoshop productions? Are there any films in the pipeline?
The money my production company, Flat Black Films, received for the Charles Schwab commercials has enabled us to independently produce more shorts. While I’ve been off on my software follies, I am engaging each of my co-workers, Katy O’Connor, Jennifer Deutrom (née Drummond), Patrick Thornton and Randy Cole, to direct their own films for the company. 6 I personally have also been doing some animated transition segments for my friend Bob Byington’s movie Somebody Up There Likes Me. Jennifer recently directed a short called Get With The Program (2010) with Rotoshop. It is ‘true’ animation in that there was no underlying video. Patrick is working now on another film which thus far uses no video. So the Rotoshop software itself doesn’t force you to trace anything. The new work that Jennifer, Katy, Patrick and Randy are turning out is truly jaw-dropping. It is very satisfying to see what these guys can do now with all the experience under their belts and the freedom to do what they want. I’m hoping to use the new films to get some people interested in another feature film. That’s the grand plan anyway – who knows what will happen. Overall, I just feel very fortunate to have been able to pursue work that really interests me.