Sensorium Modality
The Context
Sensorium Modality is a thesis exploration on the experience of chromesthesia. Thesis projects were intended to be displayed in 3D spaces at the end of the academic year.
My Role
I’ve always been fascinated by experiences I can’t have - one of those being synesthesia. Specifically, I was drawn to chromesthesia which is described as the ability to see color when one hears sound. I combined my growing interest in UX and VR to better understand the synesthetic experience and how to replicate it in a 3D metaverse.
Exploration & Investigation
One of the first things I did after some groundwork research was look into synesthetic art and see how others have recreated the experience themselves. What I found was that color varied for everyone, but bright colors consistently were associated as “good” colors while browns and greys were “bad”.
Zeroing In
The next piece of the puzzle was exploring ways to bring this to an interactive and virtual space. Playtronica would be the device I used to provide the first interactive experience in my final exhibition. It acted as a midi board that allows you to play music with any object that conducts electricity.
For a portable version, I developed an app called Colorscape. Colorscape would work in the same manner, allowing a user to choose sounds that would culminate is a swirling of colors. Colorscapes could be saved to an archive to revisit over and over.
Try the prototype here.
Upon visual exploration, I played around with techniques that seemed rhythmic and used marbling as a way to represent the blurring and swirling of colors chromesthetes experienced.
A virtual component to my exhibition that could truly be immersive for a viewer was necessary. I used Unity to create the a VR experience that a user would be able to choose various sounds, landscapes, and locations to customize a never-ending blur of colors.
Then the World Shut Down
COVID hit my senior year during the peak of thesis, and I was unfortunately never able to see this exhibition through and bring it to life; however, here are my afterthoughts.
My intent coming into this project was to better understand a condition I didn’t personally experience, and how to bring that experience to others in an immersive world.
In a perfect world, I intended to create an exhibition space where users would be able to utilize a VR headset to enter an immersive world. They would also be able to test the portable app with a headphone set.
This would be followed by a questionnaire that allowed a user to reflect on how they were feeling following the experience, and if they have any newfound understanding of the chromesthetic experience.
Here’s an AI rendition of what that may have looked like, powered by Craiyon:
