Exploring Interactive Interfaces Under Sound and Visual Content
M.F.A (Master of Fine Arts) Thesis
Supervisor
Dr. Adam Tindale,
Prof. Nick Puckett
Support: Graduate Gallery,
OCAD University
Tools
©️ TouchDesigner, Arduino, Rhino,
©️ Leap Motion, Web Camera,
Computation Coding
The Challenge
“ How can interactive interfaces transform passive audiences into active participants in audio-visual experiences?”
The Inspir
“ Inspired by VJing, where live visuals are created and manipulated in response to music, this project explores the relationship between the performer, the interface, and the audience.
VJs often use MIDI controllers, knobs, and other tools to control visual parameters during a live performance. While these interfaces give performers direct control over the visuals, the audience typically remains a passive observer.
This project explores what happens when those controls are opened up to the audience—allowing them to interact with the visuals and become part of the creative process. ”
[ Visual Jockey works performance by Sheri(Siyu) Sun]
Sound Visualization - Making sound visible
Transforms sound into visual forms.Makes audio-visual content easier to explore and understand.Creates a more engaging connection between the audience and the experience.Encourages audiences to actively participate and influence the visuals.
Noise Generated- Concept Development
©️ TouchDesigner
Installation Design- 3D modeling Technique
©️ Rhino
3D Print - Digital Fabrication
3D Printer Technique
Three ways to interact
“ Three interfaces: Arduino knob sensors, a web camera, and a leap motion, allow audiences to control different visual parameters. Each input creates a different visual response, giving audiences the freedom to explore and interact with the visuals in real time. ”
Inspiration & Research
Set up an interactive environment for audience participation.Explore three different interaction methods.Investigate how familiar interfaces can encourage audience engagement.
Interactive method：MIDI controllers ( a technique for VJs during live shows).Outcome: Keyboard trigger different parameters.Create Visual Contents by yourself!
Create Visual Contents by yourself!
Interaction Design
Tools：Arduino, TouchDesignerInteraction way: Turn two potentiometers to adjust visual parametersGoal: Provide direct and intuitive control over visual elements through physical input.1. Knob Control - Arduino Potentiometer
2. Camera Tracking - Web Camera
Tools: Webcam, TouchDesigner
Interaction way: Touch Surface/ Move under the camera
Goal: Encourage audiences to explore the space through body movement and discover how their movements affect the visuals.
3. Gesture Interaction - Leap Motion
Tools: Leap Motion, TouchDesigner
Interaction way: Use hand gestures, such as making a fist or waving, to control the visuals
Goal: Create a natural and intuitive interaction between the audience, the interface, and the visual environment.
Participation,
Exploration, Feedback
Outcomes and Contribution
1. AUDIENCE ENGAGEMENT
The key contribution of this project is its exploration of how interactive interfaces can create engaging artistic experiences and how sound can be translated into visual forms. Through a series of experimental prototypes, this project demonstrates how thoughtful interaction design can engage audiences and shape their experience in real time. By encouraging active participation, these interfaces make digital art more engaging and accessible to a wider audience.
2. COMMUNICATE IDEAS
Another important aspect of the project was exploring different ways of visualizing sound. I developed three interactive prototypes that allowed audiences to experience different forms of audio visualization directly. By using quick, visual, and interactive examples, I learned how to communicate complex ideas in a simple and engaging way. This approach is also relevant to marketing, where visual examples can help audiences understand a product, message, or experience more quickly.
3. IDENTIFY POTENTIAL AUDIENCES
This project also helped me explore how to identify and engage potential audiences. By presenting different prototypes and experiments, I was able to demonstrate the possibilities of interactive art and audio visualization to people interested in creative technology and interactive experiences. This experience taught me how to identify audience interests and communicate the value of a creative concept in an engaging way.