Context
What if our emotions could be translated onto a physical object?
Inspired by the complex feather movements in birds as a way to express their emotions, I created a sculpture that dynamically responds to the viewer's facial expression.
A constant feedback loop between the sculpture and the viewer, emotions are translated into the movements of the feathers, creating a pattern of their expressions.
RESEARCH
I began by exploring different ways that human emotions could be measured. Wearable sensors can sense blood pressure, pulse, and body temperature which could be some indication of emotion, but don't directly correlate. Instead, I started exploring how AI facial recognition could bring emotion to life in a physical object
I used FFEM (Fast Facial Emotion Monitoring), which is an open source package that uses MediaPipe for face detection, DeepFace for emotion recognition, and OpenCV to apply the emotion to each frame of the video.
Dr. Christine Metz, PhD
Endometriosis Researcher
Feinstein Institute for Medical Research
Dr. Jennifer Unger MD, MPH
OBGYN
Brown University
Celinda Kofron PhD
Biomedical Engineering Dean
Brown University
Aparna Kola
Biomedical Engineer
Nextgen Jane
After incorporating user feedback, high fidelity prototyping consisted of iterating on form and material using PLA, TPU, and silicone.
Final design
The sculpture consists of 61 servo motors connected by 4 PCA9685 drivers, a wooden base, a laser cut acrylic top, and 61 ostrich feathers. When activated, the video senses the viewers emotion through FFEM, it is processed through a python script, and then sent to the Arduino which tells the servos to move in different patterns depending on the emotion input.
Once the main product was developed, it became crucial to envision how it would be delivered to users and efficiently returned to the lab. The packaging was designed to incorporate a clear, intuitive four-step process, ensuring ease of use and seamless guidance for users.
Reflection
When testing with users, I received feedback that it "evoked a sense of curiosity and excitement" and felt like "the work of a craftsman". Interestingly, most people compared the experience to watching a field of wheat sway in the wind.
Winner of the Most Innovative Vision on Sustainability and People's Choice Awards at the ACM TEI '25 Conference
View our published paper here
