frey

Wind

Media for this project, including videos and source code.

Wind feeds an unmediated image of a natural phenomenon through a perceptual lens that alters the audience’s experience of that natural phenomenon. This perceptual lens is constructed using technology, but the piece itself is not about the interaction between the natural environment and technology; rather, it simply uses technology to focus the viewer’s attention upon the beauty that is present. By using technology as a filter on perceptual experience, the audience is brought into a heightened state of awareness that is tuned to one particular aspect of their environment.

The piece is built around a live-captured visual representation of vegetation—tall grass, or leaves on a tree or several trees—moving in the wind. A consumer level DV camera is mounted on a simple tripod and placed in front of a piece of such vegetation, and the resulting visual representation is fed into a laptop computer, operating a custom software tool, that is installed on a plinth, a high stool, or a small table near to the camera. Treating the video input as a pure data stream, the software tool extracts movement as differences between successive frames of data, rendering its processing to the computer screen as a series of six small video images, representing different parts of the processing algorithm. The resulting data, compiled into a compact structure that is freshly built for every incoming frame, is continuously transferred as a matrix of floating-point numbers to an audio synthesis engine running under Pure Data, where it is applied to an array of oscillators tuned to a pentatonic scale. The resulting audio is sent out to an amplifier and, finally, to a stereo pair of speakers, embedded within the visual source itself: if the source is tall grass, the speakers are placed in the grass, their forms not visible above the tops of the stems; if the the source is leaves on trees, the speakers are placed within the trees themselves.

The core of the process is the software tool. Starting from the premise that the human visual and aural perceptual systems are more acute pattern-recognition engines than a computer ever will be, the main processing algorithm has been crafted to preserve as much of the richness of the original data stream as possible. It performs the absolute minimum amount of interpretation necessary to shift the data stream from the visual realm to the audio realm, thereby creating sound that represents a visual phenomenon in an aural way with as little interference as possible.

The overall result is the following. You, the viewer, see some tall grass moving in the wind; seemingly connected to it, you hear what sounds like a digital wind chime, but it is more than this, somehow; it seems to be related to what the wind looks like when it blows across the grass, rather than just to the wind itself. Intrigued, you step over toward the blatantly technical installation of video camera and laptop computer in front, to try and figure out what is going on, to see if you can ‘understand the work.’ Examining the computer and the video camera you find the process laid out bare in front of you: here’s the input, here’s the processing, here’s the output. Armed with this understanding, you begin to pay more attention to the connection between what your eyes are seeing from the real world—the input—and what your ears are hearing in the real world—the output. And at this final stage, you find yourself experiencing nothing but the phenomena themselves.

An open-source (GPL v3) project, made with openFrameworks and Pure Data, with production completed during a New Interfaces for Performance residency.