As a research unit [RHP] CDRs conducts a huge quantity of work which for whatever reason doesn’t result in any kind of resolved, public facing project; this includes being in constant dialogue with practitioners, academics, collectives and institutions from around the world. Throughout 2014 fragments from these discussions will be appearing online and in print as a part of [RHP] Transcripts.
The third in this series of text’s is by Warwickshire based musician turned researcher Harry Grainger. We originally published this work as an A5 zine as a part of our Group Residency at Coventry Artspace, the following is only an excerpt. The fully text can be received for free by paying more than the RRP on anything on our http://www.rhpcdrs.bandcamp.com page (whilst stocks last)…
The idea of Effectiveness
A recurring theme that often appears in scientific studies investigating combined auditory and visual material is the idea of ‘effectiveness’. Many studies use this term in an attempt to objectively define how well a specific combination of auditory and visual material ‘fits together’, often using that exact phrase as a simple question for test subjects. The vague phrasing of this question highlights the problematic nature of effectiveness, as every individual has their own reasons and aesthetic preferences that determine their opinion of an audiovisual combination, and what might make an effective combination for one person may make it completely ineffective for another. What exactly does it mean to say that a piece of audio ‘fits’ with a visual sequence? Can the idea of an effective combination of auditory and visual material be further defined in a way that is not purely a subjective analysis? Studies that employ this line of questioning often substantiate responses by using a numbered preferential rating system, which does provide a more quantitative response than a description or simple positive or negative answers, but, ultimately, only amounts to a negative or ‘this much more’ positive result. Attempts to produce an objective response to a subjective decision brings us closer to recognizing an effective combination of auditory and visual material, but understanding what actually constitutes an effective combination involves thinking beyond simply asking whether the two ‘fit together’.
One study expanded the idea of effectiveness by focusing more on how an effective combination of auditory and visual stimuli changes how we view the combined material rather than solely on how it is aesthetically perceived. It determined that a combination was effective if it held the subjects attention to the whole, combined audiovisual material instead of each sensory element in isolation. This process of focused attention on a single event across multiple senses was further defined by splitting it into two perceptual judgments, both of which are made when a subject is presented with combined auditory and visual material. The first judgment is one of association, and is a subjective decision that determines the stylistic and aesthetic suitability of the combined material. The second judgment examines the structural and temporal relationship between each sensory perception, and involves making correlations (if any) between emphasized or accentuated points occurring within auditory and visual material. The resolution of these two perceptual judgments establishes whether or not an audiovisual event is experienced as a single event perceived across multiple senses or experienced as two individual sensory events that divide the focus and attention of a perceiving subject. A positive or affirmative resolution to both perceptual judgments will result in an audiovisual event achieving the basis for ‘effectiveness’, as the combined auditory and visual material would have been successful at creating a unified, harmonious synthesis between what is seen and heard. In contrast, if both judgments are resolved in a negative manner, then this can be thought of as an ‘ineffective’ combination due to the divergence of perceived meaning and temporal structure between what is seen and heard, and the divisive affect this has on our attention and focus on the combined sensory material. The notion of what makes an effective combination of auditory and visual material seems particularly elusive to define. Despite this, I believe that most people can make the distinction between ‘effective’ and ‘ineffective’ fairly easily, almost instinctively, as personal stylistic and aesthetic preferences are such an integral part of our subjective decision making that it is not difficult to decide what one likes or dislikes. Similarly, I think there is a general standard of acceptance as to what is recognized as technically ‘effective’, with, again, many people able to notice discrepancies that may exist between the temporal structures of auditory and visual material, and so determine it ‘ineffective’. This consciously known but vague idea of effectiveness that most people seem to hold can be described as the basic impression that what is heard naturally emanates from what is seen, that the two senses naturally seem like one event. An effective combination will make you forget that you’re listening to a piece of music and watching a visual event, but that you are hearing and seeing the physical or digital repercussions of one single action or event. Likewise, an ineffective combination will make the perceiver very much aware of one sensory perception over the other, as the unharmonious and possibly conflicting meaning generated by simultaneously perceived audio and visual content causes their attention to alternate between focusing more on either one sense or the other.