You like comics, right? Sure. That’s why you’re here. Chances are you like music too. And you’ve probably used the web as a means to enjoy both. The Internet has proved a stunningly efficient instrument for distribution of both comics and music separately, but what happens when technology brings the two together? Shouldn’t two good things combine to make something great?
Well… maybe, maybe not…
A MATTER OF DEBATE
Whenever the subject of combining music and comics comes up, there are two arguments almost invariably arise against it:
- The addition of sound to comics is an unnecessary gimmick and only results in distraction and inconvenience for the reader, and
- the addition of music takes something away from the essence of what comics is in the first place.
The former argument boils down to personal preference. Some people would rather to choose their own soundtrack for their reading experience than have one prescribed, and some prefer no soundtrack at all. There’s not much to argue against that. The latter argument, however is open to considerable debate.
Try as they might no one has come up with a clear-cut, irrefutable definition of what "comics" is. The advent of web technology and the possibility of integrating elements such as animation and sound has blurred the picture even more. What is it that makes a comic a comic, and how does the addition of these other elements affect that? Is there something essential to comics that is lost in the attempt to integrate sound?
It is beyond the scope of this article to propose any new definitions, but when discussing comics in combination with music it’s important to note some distinctions of form.
PACE YOURSELF
On the surface comics and music are not dissimilar forms. Both rely extensively on rhythm and timing to create their desired effects. The disparity of image versus sound is not an insurmountable distinction either. After all we both see and hear thing simultaneously all the time. The main difference between comics and music, and the stumbling block toward integration, is the way we process the information given by each. Music is assimilated passively – tones and rhythms are processed in real time with the performer in complete control. Comics are processed more dynamically. The audience is in control as the interplay of elements is deciphered, or read. And everyone reads at their own pace.
So what happens when we mix the two distinct forms? Can the differences be reconciled, and if so what does it mean for comics?
Let’s take a look at some techniques.
LAYING OUT THE BEAT
As stated comics and music have a common reliance on "beat." Comics’ rhythm comes from a division of space while music’s is temporal. The obvious idea for depicting the rhythm of music in comics is a visual abstraction.
If time can be represented by space in comics then the division of space can mimic divisions in time. All we need are visual cues. We have a head start in finding them in fact, as a visual abstraction of music has already existed for hundreds of years.
The staff and bars are used to represent tone. The type of note gives us the cue to the rhythm. It’s a visual depiction of sound. Musical notation is not comics, though it too is read and deciphered. But it gives us the necessary divisions of time. If we imagine the note’s duration as panels of varying size and imagine a measure as one tier of panels we get this layout:
Still not quite a comic, but if we introduce elements that reinforce the notion of the panels duration…
Voila… music as comics
It’s similar to the theory (apparently expressed by Matt Madden and recently put into practice by Derik Badman) of translating poetic rhyme scheme to comics form. Sound is interpreted visually by putting into effect a series of design constraints on the images. Here, music suggests the rhythm but the result is pure comics.
How effective is it as integration? An example such as our dancing friend falls well inside established boundaries of comics form. Music played a part in it’s creation, but it’s not quite a combination. Ultimately the strip stands alone. Comics? Yes. Music? No.
The next step, then, is to bring the sound into the equation – simultaneous presentation. Not actually new to comics (have you ever listened to music while reading?) it’s a method made more convenient by the web.
MOOD MUSIC
The technique of embedding an MP3 on the page with a comic is the most common of combinations currently employed in webcomics. Most of the stories in Kean Soo’s Exit Music use this technique, as do Jonathan Altschuler and Colleen Macisaac’s Music Comics and many of the installments in Cat Garza’s Those Were the Salad Days. The music is meant to be played as you read and acts as a kind of emotional setting or background for the narrative. The theory is that the emotional timber of the song reflects and intensifies the sentiment of the comic. The technique can be quite effective.
Take Colleen MacIsaac and Jonathan Altschuler’s music-comic, #1. The narrative is simple and disquieting. Altschuler’s looping, discordant of piano progression is an excellent set piece for the eerie, dream-like sequence depicted by MacIsaac’s brushwork. Similarly, Kean Soo’s In a Lonely Place uses the background of Joy Division’s song of the same name to reflect a half-conscious emotional resonance. In each comic image and sound reflect on each other and the result can be quite striking.
In reading a combination of this type one might unconsciously fall in step to the rhythm of the music, but essentially the strip can be read at it’s own pace. The music serves as background only. Overall, it retains its identity as "comics" but there are other limitations to this type of combination.
The songs tend to go on long after you’ve finished reading the comic. And one is left wondering if the addition of music is really vital to the reading. All of Kean Soo’s Exit Music comics have appeared in print (sans music, of course). Neal Von Flue’s music/comic experiments (scroll down to Synesthesia Drive In) highlight the uncertainty by offering up the comics and songs as separate elements on separate web pages. Th Music is essentially unnecessary. Still, it can make for interesting effect and the combination retains the reader controlled pacing of comics.
Sometimes it’s necessary to give up a little of that control, however, for a song to better reflect a narrative. Rarely does a story (or song) only express one mood throughout. But to express any shifts in emotion comic and song must work in tandem. To synch these shifts a greater reliance on creator controlled pacing is needed.
N’SYNC – whoops, I mean, IN SYNCH
Kean Soo uses visual cues to pace the reader in Bottle Up and Explode. Printing the lyrics at key spots in the flow of the strip forces the reader to follow along at the pace of the music. The controlled pacing attempts to better match certain parts of the song to key moments in the narrative. It’s a clever solution, and uniquely suited to the visual nature of comics. But it only works if the reader consciously uses the cues. Even then it can make for a clumsy reading experience as you stop and wait at each chunk of lyrics for the song to catch up with your reading.
MUSIC VIDEOS?
The next step in synching music to narrative is limiting the duration the reader is focused on any given panel. Using programs such as Flash (as in Broken Saints) or QuickTime (as the first few installments of Cat Garza’s Salad Days) places a set time for each panel to appear and animated transitions take you to the next panel. This is where the distinction of comics/not comics really starts to make itself evident. The images themselves are static, and in the case of Broken Saints all dialogue and exposition are presented as text within the panels. But the experience of "reading" is lost with the animation, and it becomes a more passive experience of "viewing." Image and sound are perfectly synched but at the total loss of reader controlled pacing that is a hallmark of comics.
Animation can have it’s place in comics, though. Garza sees the QuickTime files in Salad Days as animated "meta-panels." Taken in context with the rest of the strip, the quicktime file acts as an attention getting splash panel, or a music video embedded into the course of the comics narrative. The "meta-panel" is a concept that works equally well in a gag-strip format. Take this example from Kidd Radd. The second panel is an interactive animation (through a click-through exchange of animated gif images) set to an 8-bit soundtrack (note – apparently the sound does not work on some browsers). It’s a fun little gag to play around with and it works as comics due to its simple panel-to panel-context.
SUPER SCIENCE
Up to this point any attempts at a true synching narrative to soundtrack in a whole-comic scale have largely been failures. Kean Soo’s lyric-printing method makes for clumsy reading and the creator driven pacing of Broken Saints results in something that’s not quite comics anymore. It’s been said that it would take some kind of super-science to find a method that could match a soundtrack to any readers preferred pacing. So what would that "super-science" be?
One could imagine sound loops linked to certain areas of a comics page triggered by some sort of eye-tracking technology. Or perhaps with a greater understanding of brain activity, certain electro-chemical reactions used in the process of reading could be tracked and exploited to this measure. There are a number of science-fiction-like possibilities that could be put forward. The uncertainty of process, inconvenience of any needed hardware, and considerable expense of even studying the credibility of these methods renders them no more than idle speculation. Besides a simpler answer may already be right in front of us.
THE READER-DRIVEN SOUNDTRACK
Adobe/Macromedia’s Flash, a web-native program, has proved fertile ground for animators. Flash animations proliferate on the web and offer streamlined integration of motion and sound. But the program also offers something traditional animation cannot – interactivity. It’s a feature that is not lost on some of the more technologically ambitious creators of webcomics. From the experimental animated panel transitions of John Barber and the special effects work of Patrick Farley, to the clever embedded panels B. Shur uses at I Am A Rocket Builder the interactive possibilities of web animation are making their mark on comics storytelling. With click-scroll interfaces such as Daniel Merlin Goodbrey’s Tarquin Engine the reader still chooses the pace but with panel-to -panel navigation, the reader’s focus becomes a lot more predictable. Certain elements can be matched to specific panels. Flash has the capability to link up mouse-clicks to audio files as well, and that’s where the idea of an integrated webcomic soundtrack becomes a real possibility.
Flash’s sound integration has been used in webcomics before, often to somewhat irritating effect. Adding sound effects to a comic’s click-through navigation can be unnecessary and distracting. Dark Horse Comics’ initial online presentation of Mike Mignola’s Hellboy stories offers a somewhat clumsy first look at the technique. It’s a somewhat jarring combination here, the background "music" is clumsy and repetitive drone (later additions to the series dropped the music altogether) but it begs an interesting point. Perhaps with a little imagination, using layering and repeating tones and sound loops, an ambitious cartoonist could design a real-time interactive soundtrack for their comic. Dynamic, adaptive musical scores have been a staple of the video game industry for years. Given the technology available on the web it’s not hard to imagine them for comics as well.
Of course even click-through, animated transitions, such as those found in Tarquin and Infinite Canvas, have their opponents in the comics/not comics debate. It seems the closer we get to true music/narrative integration the further we stray from comics. Perhaps a fully integrated soundtrack just isn’t possible. But when you get down to it, all these questions of form are really beside the point.
WHO CARES?
Comics or not comics there are still those who prefer no sound. The deciding factor in the accessibility of these experiments in form is how compelling the content is, not how well it meets the definition of comics. Ultimately it’s the entire reading experience that matters.
Does the combination create an interesting effect? Are the elements integrated in a clever way? Most importantly, is the addition of music truly necessary to the overall message? These are the only questions we need to answer. Whether or not the result retains the "purity" of comics is of interest only to nerds like me.
In order to be effective a music/comics combination only needs to be interesting. Whatever way you mix it, it just has to make sense. The music must be absolutely integral to the overall piece. That is the only measure of whether a combination works. If a cartoonist can then pull that off it just might convince the other half to turn up the volume in the first place.
Tym Godek does occasionally make webcomics, but he’s never tried one with music so what does he know anyway?
Comments are closed.