Asynchronous Voice Anonymization by Learning from Speaker-Adversarial Speech

Fig. 2 illustrates the Mel-spectrograms of an original speech utterance and its speaker-adversarial variants, generated with 5, 10, and 20 iterations, respectively. The comparison reveals that as the number of iterations increases, the perturbation effects on the Mel-spectrograms rises. This indicates that the perturbation is captured by the Mel-spectrogram, thereby validating the loss function defined in (3).

Extended experiments were performed to investigate the effects of the iteration number K for speaker-adversarial speech generation on the performance of asynchronous voice anonymization, with K set to 10 and 20. Evaluations were conducted in both ignorant and informed scenarios, following their respective configurations outlined in Section V-B-1) and V-B-2 in the manuscript. Besides, the human perception evaluation was performed following the configurations presented in Section V-C-2) in the manuscript. With the results, Table I and II are expanded to be Table V and VI, respectively. The human perception preservation rates are presented in VIII.

As presented in Table I, the performance of the proposed method with varying numbers of iterations (K=10 and K=20) demonstrates that the number of iterations significantly impacts the anonymization effectiveness. In the three evaluation models, with K increasing from 10 to 20, higher EERs were obtained. This indicates that increasing iteration number K achieves stronger obscuration of the machine-discernible speaker attributes in the ignorant evaluation scenario.

From Table VI, it can be observed that with K increasing from 10 to 20, higher EERs were obtained. This demonstrates that higher K obtains stronger obscuration of the machine-discernible speaker attributes in the informed evaluation scenario.

The results in Table VII indicate that as K increases from 10 to 20, the human perception rate declines. This suggests that a higher K degrades the human perception preservation of the proposed asynchronous voice anonymization method. Considering the machine perception protection capabilities shown in Tables V and VI, K was set to 15 in our proposed method to achieve a trade-off between human perception preservation and machine perception protection.

Next, subjective evaluations were conducted with original speech as reference to measure the human perception preservation capability. The human perception preservation rates obtained between Original(Ori)-Regenerated(Regen), Ori-Proposed(Pro), Regen-Pro pairs are presented in Table VIII. From the comparison, it can be observed that:
1) The regenerated speech obtained a 76.50% human perception rate when compared with the original speech, indicating the influence brought by the YourTTS model. As a result, the regenerated speech was utilized as the reference in our subjective evaluation, with results presented in Table III (b) of our manuscript.
2) The anonymized speech demonstrated a lower human perception preservation rate when compared to the original speech than when compared to the regenerated speech, i.e., 54.00% vs 93.50%.

Finally, experiments were conducted by replacing the regenerated speech with the original speech for perceptual loss calculation in (4). Experiments were conducted in both ignorant and informed scenarios, following their respective configurations outlined in Section V-B-1) and V-B-2 in the manuscript. Besides, the human perception evaluation was performed following the configurations presented in Section V-C-2) in the manuscript. With the results, Table I and II are expanded to be Table IX and X , respectively. The human perception preservation rates are presented in XI.

As shown in Tables IX and X, using the original speech in perceptual loss computation achieved higher EERs in both the ignorant and informed evaluation scenarios than the regenerated speech. Furthermore, as shown in Table XI, Pro(ori) achieved a higher human perception rate compared to Pro when the original speech was used as the reference. These results demonstrate that the original speech is applicable for perceptual loss computation in our proposed strategy of learning the voice protection capability from speaker-adversarial speech. However, as indicated with the results presented in Table VIII, to ensure a fair comparison, regenerated speech was applied as the reference for human perception preservation evaluation. When the regenerated speech was used as the reference, the anonymized speech generated from the original speech for perceptual loss calculation exhibited a lower human perception preservation rate compared to that generated from the regenerated speech, at 81.00% versus 93.50%. Thus, to explore the optimal performance of the proposed strategy of generating asynchronously anonymized speech by learning from speaker-adversarial speech, the regenerated speech was used for perceptual loss computation in the manuscript.