ADVERSARIAL SPEECH FOR VOICE PRIVACY PROTECTION AGAINST PERSONALIZED
SPEECH GENERATION
Authors: ShiHao Chen, Liping Chen, Jie Zhang, KongAik Lee, Zhen-Hua Ling, Li-Rong Dai
Abstract: The rapid progress in personalized speech generation technology, including personalized text-to-speech (TTS) and voice conversion (VC), poses a challenge in distinguishing between generated and real speech for human listeners, resulting in an urgent demand in protecting speakers' voices from malicious misuse. In this regard, we propose a speaker protection method based on adversarial attacks. The proposed method perturbs speech signals by minimally altering the original speech while rendering downstream speech generation models unable to accurately generate the voice of the target speaker. For validation, we employ the open-source pre-trained YourTTS model for speech generation and protect the target speaker's speech in the white-box scenario. Automatic speaker verification (ASV) evaluations were carried out on the generated speech as the assessment of the voice protection capability. Our experimental results show that we successfully perturbed the speaker encoder of the YourTTS model using the gradient-based I-FGSM adversarial perturbation method. Furthermore, the adversarial perturbation is effective in preventing the YourTTS model from generating the speech of the target speaker.
Here, we showcase the adversarial samples generated using the I-FGSM method to attack the YourTTS model, along with the speech generated from both adversarial samples and reference speech. We randomly selected one sentence each from 2 males and 2 females and synthesized them using the Coqui TTS tool. In this context, 'Reference Speech' is defined as the original speech that is used as reference speech in YourTTS, 'IFGSM' as the speech with adversarial samples added, 'TTS' as the speech synthesized using Reference Speech for TTS, 'VC Source' as the source speech for VC, 'VC' as the speech synthesized using Reference Speech for VC, 'TTS*' as the speech synthesized using adversarial samples for TTS, and 'VC*' as the speech synthesized using adversarial samples for VC.