M: )SK
A mask with a mouth： ）
People communicate though four ways: spoken or verbal communication, Non-verbal communication, written communication and visualizations. Those usually involve many parts of the body: mouth, eyes, facial muscles, hands and even the whole body. However, during coronavirus pandemic, wearing masks is considered an effective way to prevent infection. Especially for healthcare workers who have to wear heavy protection suits covering from head to feet. How might healthcare workers better communicate with each other? In this project, I explored how to augment verbal communication by non-verbal communication, non-verbal communication and visualizations.
M: )SK is a respiratory mask with a LED matrix screen. Healthcare workers can reuse it by changing the filter core. The screen is connected to Raspberry Pi and Google AIY Kit. Google Cloud API and Google Assistant API was applied to achieve voice recognition and critical words trigger functions. By adding the screen, M: )SK is able to display three modes: name card, synchronized lip motion, and Emuoji.
“Sometimes, we can even not recognize each other with protection suit covering from head to feet. Inside the hospitals, air ventilators are usually on, leading to noise. Besides, talking through the dividing glass in ICU rooms, are extremly hard. We have to raise their voice in order to have their colleagues heard. ”
-Interview with a nurse who participated Covid-19 rescue.
Name Card
From interviews and preliminary research, I found a frequent issue for healthcare workers was they cannot recognize each other. That is because all their faces were cover by masks and goggles. In the default mode, the mask shows the healthcare workers’ name. Red is obvious and easy to be noticed, so even they are far away from each other, or through thick glass, they are able to recognize each other.
L.Q.
Synchronised Lip Motions
Non-verbal communications involve gestures, body and facial languages. Due to the protection suit, it is hard to behave similarly as we are dressed as usual. How do we recognize others’ words in a noisy environment? It is no surprised that we usually read their lips. Based on this, lip motions are synchronized and displayed simultaneoously on the screen, helping them better read each others’ lips easily. Red LED matrix, which has the same color as lips, is a reasonable presentation of our mouths.
Emouji - Mouth Emoji
Emoji as a universal, clear and instant emotion expression language is frequently used in written/typing communication, and enhances written/typing messages conveying. For example, we want to show we are surprised at a message. Instead of saying ‘I am surprised that…’, we simply send him/her 😯. Same here, to augment a sentence, I add Emouji to verbal messages, since the non-verbal way(facial/ mouth languages) has been blocked. For instance, if I want to say I love you on the phone, we see ‘I love you😘’. by Emouji, after the LED screen shows the synchronised lip motions, we will see a heart at the end because ‘love’ as a critical word activates Emouji. Critical words such as: surprised, speechless, shut up, happy, love, naughty, smile, and sad are set to trigger the Emouji.
by Ling Qin
Apr. 2020 - May 2020