Application I: REAL signal from speakers’ masks

Wearing masks in public places such as in airports or hospitals (Figure 3a) is a new norm since coronavirus disease COVID-19. The mask will cause a noticeable reduction in loudness and clarity of speech[26] which is more susceptible to background noises. Additionally, masks prevent facial expressions and lipreading,[27,28] which makes speech understanding more difficult without audio-visual understanding. We demonstrate REAL as an ideal tool to probe the audios in these cases. In Fig 3b, a speaker wearing a mask and face shield in an environment with loud background noise from a loudspeaker. One microphone is placed near the speaker’s mouth to collect clear audio as ground truth, and another microphone is placed on the REAL platform to represent the ear of the listener. Figure 3c shows that the microphone ear of the listener completely failed while the audio measured by REAL is similar to the ground truth both in waveform and STFT spectrogram. Audios obtained from REAL on masks could often be understood by humans directly without additional processing (Supplementary Video 1). In many cases, the audios (Supplementary Audio 1 and 2) can be accurately transcribed using speech-to-text services such as the Google Cloud platform (https://cloud.google.com/speech-to-text) (see Figure S3).