A team from Art-A-Hack™ Summer 2016

What can the limits of technology tell us about the limits of communication? The Sonic Equations team explored this question by creating a two-player game which electronically distorts participants’ communication in real-time.

The first player reads a script about communication into a microphone. The second player hears an algorithmically distorted feed of first player’s voice, and tries to interpret it as text.

The text is displayed on a screen so the speaker and audience members can see how the message’s meaning has been conveyed.

The idea was prompted by a peculiar moment in the team’s process, during a video-call meeting with program organizer Ellen Pearlman. Team lead Christopher Edwards, who is hearing-impaired, was relying on assistive technology to interpret.

This assistive technology, in the form of an app on a smartphone, misrepresented some of Ellen’s words to Chris, creating a new layer of interpretation in the communication process. This layer changed the perceived meaning of what was being said, and this moment became a metaphor for the project.

The real-time distortion in the game is generated by a customized Pure Data sound programming environment. The software was created by developer Laura Mun, who learned the environment from scratch.

The program they created manipulates the audio between the two players, transmitting distorted segments of sound. In essence, this creates a ‘displaced conversation’ event, which embraces and explores the limitations of technological communication.

The team faced challenges in finding just the right sensitivity to apply distortion to the audio feed. If it was too sensitive, background noise and chatter would end up being transmitted. If it was not sensitive enough, soft-spoken players would not be able to participate.

The team also overcame obstacles in setting up the microphone to avoid picking up player’s keystrokes.

For the Art-A-Hack 2016 showcase, the team set up the game for visitors to play following their presentation. As the evening progressed, they printed the resulting text from each game played, for audience members to see.

Team lead Christopher explains the rationale: “When technology is stretched, or at it’s seams, this is when you see about the limits and the promises of technology”, as systems “reveal their larger dysfunction”.