We have plotted brainwave activity into a knitted pattern. Using a wearable, non-invasive EEG headset, we recorded users’ affective states while listening to Bach’s “Goldberg Variations”, concretely the aria and its first seven variations. The audio was about 10 minutes long and we downsampled each second of the signal coming from the 14 channels of the EEG device. Three main features were measured: relaxation, excitement, and cognitive load. After recording, those features were converted into a knitting pattern. Hence, every stitch of a pattern corresponds to a unique brain state stimulated by the act of listening. It means the user’s affective response to music is captured every second and memorised in the knitted garment pattern.
Why have we used music? Because music is one of the most powerful mood inducers, provoking immediate affective reactions that can be deduced by looking at human physiology, as in the case of brain cortical activity. These affective states, that are implicit to every human being, can be measured through EEG technology. By applying this technique, we were able to create unique patterns coming from unique humans traits. It other words, personalized, implicit knitting with context and message.
Concerning the selected music, the first case study uses Bach’s Goldberg Variations as a stimuli for the users. After the statement of an aria at the beginning of the piece, there are thirty variations, from which we took the first seven. The variations do not follow the melody of the aria, but rather use its bass line and chord progression. The EEG correlate of relaxation, engagement and cognitive load was recorded while the users were listening to the musical pieces. This information was later sent to the Knitic framework to create a bicolor pattern for knitting.
The knitted garments picture the listener’s affective and cognitive states during the experiment. It is a way of making tangible the implicit the states of users and visualizing them in an original way as a large and personal data footprint.
Neuro Knitting represents a novel way of personal, generative design and fabrication. An approach that brings together affective computing and digital crafts. And thus, it offers new applications and creative thinking to both areas.
More photos about the whole process here>>
This project is a collaboration between artist-duo Varvara Guljajeva(http://varvarag.info) and Mar Canet(http://mcanet.info) and MTG researcher Sebastian Mealla (http://mealla.net/). Sebastian Mealla is a PhD candidate whose research focuses on the possible uses of brain and body signals for multimodal interaction. Sebastian Mealla has brought his expertise on physiological computing to this project. Varvara Guljajeva and Mar Canet are the developers a open hardware knitting machine Knitic allowing creative use of digital knitting technology.
Together they had brainstorm this art project to demonstrate art and science collaboration.
The project is a perfect example for art and science collaboration.