Audio Deepfake Detection: A Comprehensive Survey of Generation, Datasets, and Detection Strategies

Datasets are essential for training, validating, and benchmarking detection algorithms, making them fundamental to the progress of audio deepfake detection. Researchers use high-quality datasets to evaluate how well their methods perform across different types of forgery, languages, and recording environments [31]. In the following, we will review and discuss a set of datasets commonly used for audio deepfake detection that were generated using the aforementioned techniques.

(1) ASVspoof2015 [44]

This dataset marks the first major international competition dedicated to exposing vulnerabilities in ASV systems to spoofing attacks and advancing the development of effective countermeasures. Built upon a standardized database, it contains both genuine and spoofed speech samples.

The genuine recordings, collected from 106 human speakers (45 male and 61 female) without any alterations, exhibit minimal channel or background noise interference. In contrast, the spoofed speech was generated by applying various speech synthesis (SS) and VC algorithms to the original recordings. The resulting synthetic copies, generated using multiple forgery algorithms, were systematically partitioned into training, development, and evaluation subsets.

(2) ASVspoof2017 [45]

The primary objective of the ASVspoof2017 competition is to assess how effectively ASV systems can detect spoofing attacks under realistic conditions. By pursuing this goal, the challenge aims to advance the development of a robust framework to combat voice spoofing, with particular emphasis on detecting replay attacks.

The ASVspoof2017 database contains 3,565 genuine utterances from 42 speakers, collected through the Red Dots program, and 14,465 spoofed audio samples from 177 speakers, created by replaying the original recordings across a wide range of devices and acoustic environments to simulate realistic replay scenarios.

(3) ASVspoof2019 [46]

The edition of ASVspoof2019 was the first major evaluation campaign to deal with all three types of spoofing attacks: TTS, VC, and replay attacks—within a unified framework. It’s the third expansion version of the ASVspoof challenge series compared with ASVspoof2015 and ASVspoof2017. This dataset is separated into two scenarios: logical access (LA) and physical access (PA). The LA scenario emphasizes the use of advanced synthesis and VC technologies such as neural acoustic and waveform models, whereas the PA scenario addresses replay spoofing attacks in simulated physical environments. The 2019 edition also introduces a new scale for reliability, incorporating the impact of spoofing and countermeasures. Original recordings were obtained from 107 speakers, 46 male and 61 female under conditions with minimal channel distortion and background noise. Systematically, spoofed speech samples were then generated from this source material using a variety of spoofing algorithms to achieve wide coverage of potential attack strategies.

(4) ASVspoof2021 [47]

ASVspoof2021 is the fourth edition of the ASVspoof challenge series. It was designed to evaluate the detection of synthetic and manipulated speech across three evaluation partitions (LA, PA, and DF). In this edition, no new training or development datasets were provided; participants relied on the ASVspoof datasets previously released to develop their systems.

The only difference in this evaluation set compared to past evaluations is the added emphasis on differences across channels that produce varying degrees of coding and transmission artifacts (i.e., telephony networks and IP-based communications).

The evaluation dataset for both the LA and PA tasks uses recordings taken from the same 48 speakers that were used for evaluation in the ASVspoof2019 challenge. In addition, the Deepfake task dataset was collected from VCTK, along with additional unpublished datasets, to provide a wider variety and greater complexity in fake speech scenarios while also yielding a more realistic overall evaluation. The evaluation dataset presents completely new and highly variable challenge conditions against which to evaluate current state-of-the-art spoofing detection systems.

(5) ASVspoof5 [48]

ASVspoof5 is the fifth edition of ASVspoof, which is built to assess the effectiveness of systems in detecting spoofing and deepfake audio. This dataset represents a major shift toward “in-the-wild,” which means that the recording is done in a crowdsourced environment instead of studio-quality recordings. It is built upon the Multilingual Libri Speech (MLS) English partition, offering diverse sources and a broad range of speaker voices using diverse consumer devices in uncontrolled acoustic environments.

ASVspoof5 expands the challenge by incorporating a suite of 32 different attack algorithms that include legacy and advanced TTS, VC, and adversarial attacks for the first time. These additions aim to generate synthetic samples that are more sophisticated and difficult to detect. The dataset includes seven speaker-disjoint subsets to support tasks ranging from standalone detection to spoofing-robust ASV. Furthermore, it provides around 30000 additional speakers for training robust encoders.

(6) Fake or Real [49]

This open-source dataset supports synthetic speech detection research. It contains 198,000 English-language audio samples designed to train DL models without overfitting. It includes 87,000 synthetic utterance audio recordings generated by Deep Voice 3, Amazon AWS Polly, and Google Cloud TTS, and 111,000 real utterance audio recordings created from the Arctic Dataset, LibriSpeech, VOXForge, and YouTube educational videos featuring diverse speakers. The dataset is available in four versions (FoR-original, FoR-Norm, FoR-2sec, and FoR-rerec).

(7) Wavefake [50]

This dataset comprises 117,985 generated audio files (16-bit PCM WAV) totaling approximately 196 hours. Primarily based on the LJSpeech (English) and JSUT (Japanese) language sets, the samples were generated using six advanced architectures (MelGAN, MelGAN Large, FB-MelGAN, MB-MelGAN, HiFi-GAN, Parallel WaveGAN, and WaveGlow) and cover ten subsets, including two languages and a complete TTS script with new phrases.

(8) ADD2022 [51]

This audio deepfake detection challenge addresses gaps left by previous challenges, such as ASVspoof2021, by focusing on realistic and complex scenarios. It covers audio generation and detection through three main tracks: Low-Quality (LF) fake audio detection, Partial Fake (PF) detection, and Fake Game (FG). Using diverse datasets such as AISHELL-3, the challenge employed well-defined protocols and evaluation metrics. The results highlight significant difficulties in model generalization and the need for improved evaluation metrics.

(9) In-the-Wild [52]

The in-the-wild dataset is a widely used benchmark specifically generated to address the generalization gap in detecting speech deepfakes. Instead of using controlled laboratory settings to generate datasets, this dataset consists of real-world audio collected from publicly available sources, such as social media and video streaming platforms. It includes both genuine and synthetic audio recordings, totaling approximately 38 hours, with 20.8 hours of authentic speech and 17.2 hours of deepfake audio, produced by 54 English speakers. Due to generating the samples from various unknown generating algorithms and recording them under diverse acoustic conditions with varying compression levels, it serves as a rigorous test for how well a detection model can perform against realistic, unseen threats.

(10) Multilanguage Audio Anti-Spoofing Dataset [53]

MLAAD v8 addresses linguistic bias in existing deepfake detection datasets by providing a multilingual corpus. It is built as an extension of the M-AILABS Speech Dataset, whose original bona fide recordings include eight languages (English, French, German, Italian, Polish, Russian, Spanish, and Ukrainian). These original recordings consist of authentic human speech collected from audiobooks, public speeches, and interviews. By using the original or translated transcripts as input to TTS systems, MLAAD v8 extends these source recordings by generating 570.3 hours of synthetic audio across 40 different languages, using 119 advanced TTS models representing 58 distinct architectures. MLAAD v8 offers a valuable resource for evaluating cross-lingual generalization and real-world performance of audio anti-spoofing systems by incorporating an extensive collection of synthetic voices and languages.

(11) The Singing Voice Deepfake Detection challenge [54]

The singing voice deepfake detection (SVDD) is the first research challenge to develop systems that can identify the authenticity of human-sung audio recordings versus AI-made recordings of the same singer. A recent increase in advancements with generative models using AI technology to replicate the human voice for musical purposes has made this research topic significantly more relevant. The SVDD project hopes to catalyze additional research into how we can prevent the infringement of an artist’s right to their own work and create systems to automate this process. This cannot be accomplished without sophisticated forms of DL, which can extract information from both natural and artificial sounds that an investigator can use when determining the authenticity of the audio source.

To aid in research in this area, the SVDD2024 challenges created two benchmark datasets: CtrSVDD and WildSVDD. CtrSVDD consists of pure singing recordings without any accompanying music, as well as some synthetic samples produced by using a variety of advanced audio transformation and synthesis techniques. On the other hand, WildSVDD attempts to simulate real-world conditions as closely as possible by including samples taken from social media and including both types of recordings (real and fake), each with its own varying degrees of background music and different recording environments (e.g., sound quality).

Table 1. Comparative summary of prominent datasets for synthetic speech, voice conversion (VC), and replay attack detection sorted by year of creation

In Table 1, we summarize the core characteristics of each dataset, including the static information, and report the accuracy results for each dataset based on the results reported in the studies that make up the basis of this review. This table also lists the various types of attacks represented within each dataset, e.g., TTS attacks and VC attacks. In addition, in this table, we found information about more recent adversarial attack types and some technical characteristics. Furthermore, we demonstrate an increase in the volume of data available to researchers by showing an increase in the number of spoofed samples within each dataset. At the same time, we show that there has been a significant increase in both the complexity and resource requirements of the modern-day detection models across datasets over time.

DL-based audio deepfake detection leverages learning representations from speech automatically using advanced neural networks. The most popular approaches used in this field are convolutional neural networks (CNNs), long short-term memory (LSTM), and transformers. A CNN may capture both frequency patterns (spectral) as well as time-frequency patterns, while LSTM captures long-term temporal dependencies present in audio sequences [65, 66]. In recent years, transformer and self-supervised models have shown improved robustness, scalability, and generalizability against many new deepfake audio generation methods.

5.1 Convolutional neural network-based spectral and time-frequency modeling

Initial DL methods for audio spoofing detection mainly relied on CNNs integrated with handcrafted or time-frequency acoustic features. Such as MFCC, CQCC, and spectrograms. The early frameworks used standard CNNs, residual CNNs, light convolutional neural networks (LCNNs), and transfer-learning-based ResNets to capture discriminative artifacts associated with spoofed speech. Residual CNNs showed the possibility of leveraging multiple representations of the acoustic signal for spoofing detection. The proposed ResNet framework utilizing MFCC, CQCC, and spectrogram as inputs yielded a low EER on the logical-access subset of the ASVspoof2019 dataset, indicating the effectiveness of deep residual learning in discriminating between real and synthetic speech [67]. The downside of using multiple representations is a higher computational burden and no performance guarantees when switching to cross-dataset evaluation. Subsequent research investigated lightweight architectures to reduce the computational complexity associated with standard residual networks. Combining an LCNN with a genuinization Transformer resulted in an EER of 4.070% on ASVspoof2019 [68]. This result suggests that computational efficiency can be improved while preserving the ability to learn discriminative spoofing patterns. However, the evaluation was limited to logical-access attacks within a single benchmark; therefore, robustness to physical-access attacks, noisy audio, and real-world conditions remains unknown.

Transfer learning has been applied for improving representation learning while mitigating optimization challenges for deep architectures. ResNet-34 models have been able to benefit from residual connections that alleviated vanishing gradients and the capability to learn multi-scale acoustic features [69]. However, despite the abovementioned benefits, such models are computationally expensive and require diverse training data to support reliable generalization.

A more localized direction employs frequency-domain CNNs to analyze spectral abnormalities without modeling the complete temporal sequence. This approach achieved 85.99% accuracy on ASVspoof2019, indicating that frequency-domain features can be used to distinguish between genuine and spoofed speech [70]. However, its lower performance relative to more advanced models shows that frequency-domain approaches are incapable of modeling long-term temporal and context-related inconsistencies.

In summary, it can be said that CNN, LCNN, and ResNet approaches have demonstrated that deep feature learning is effective for detecting local spectral and time-frequency feature artifacts [67-70]. The major drawback of these methods is that their performance is highly dependent on the selected acoustic front-end and similarity of training and evaluation environments.

5.2 Temporal, contextual, and sequence-modeling architectures

Although convolutional layers have been successful in local pattern detection, they may be limited in capturing long-range temporal relationships and contextual dependencies. This limitation has encouraged the development of transformer-based, Conformer-based, and state-space models that are able to capture interactions within long audio sequences. Hierarchical Conformer models integrate convolutional modeling of local speech patterns with the use of transformers for global sequence modeling. The use of hierarchical pooling and multi-class tokens helps in the integration of the spoofing information on multiple scales [71]. This design produces more sophisticated sequence representations than the traditional CNN models; although its EER suggests that architectural sophistication alone is insufficient to guarantee strong detection performance. The complexity of the model and the training and dataset characteristics remain important factors.

The integration of Conformer back-ends and Wav2Vec 2.0 representations resulted in significantly lower error rates for speech samples of varying lengths. Such models were able to achieve an EER of less than 1% in logical-access experiments by combining contextual SSL and sequence modeling for accommodating the variability in the audio duration [72]. These results highlight the advantages of using pretrained representations together with architectures that can model simultaneously local and global features.

State-space models provide an alternative to standard transformer architecture due to their ability to work with long sequences without some of the computational limitations associated with attention mechanisms. Using a bidirectional Mamba-based architecture for extracting low-to-high-level speech features via parameterized sinc functions and convolutional blocks, followed by bidirectional sequence modeling, state-space processing is able to capture long dependencies while maintaining competitive detection performance [73]. However, such architectures are more complex to design, and they still need large and diverse training datasets. In general, Conformer and Mamba models provide more advanced temporal modeling than a convolution-only approach [71-73]. Nevertheless, these benefits are associated with higher complexity, higher computational cost, and continued sensitivity to input duration and model configuration.

5.3 Self-supervised speech representation learning

One of the most promising approaches to audio deepfake detection is self-supervised learning (SSL). Unlike other methods that rely solely on hand-crafted front-ends, SSL methods employ pre-trained speech models such as Wav2Vec 2.0, XLS-R, HuBERT, and WavLM for extracting high-level representations from raw audio samples.

Pre-training on authentic speech was found to be beneficial to spoofing detection even when no artificial examples were needed in the initial learning phase. It was also demonstrated that combining Wav2Vec 2.0 representations with an AASIST back-end resulted in lower error rates compared to sinc-based front-ends, which shows that the knowledge learned from authentic speech can support the detection of different spoofing attacks [74]. This means that SSL representations encode transferable speech characteristics rather than only attack-specific acoustic cues.

Numerous evaluations of SSL front-ends and neural back-end classifiers resulted in near-zero EERs in several ASVspoof challenges [75]. These findings indicate that selecting an appropriate representation may be as important as selecting the classifier. But these systems require careful fine-tuning, and their strong performance on benchmark datasets does not necessarily transfer to audio that differs in language, codec, recording device, or generation method.

Architecture search techniques have also been employed together with SSL to reduce dependence on handcrafted detection architectures. Using Wav2Vec 2.0 representations along with Differentiable Architecture Search (DARTS) produced an EER of 1.08% in ASVspoof2019 LA [76]. In this framework, the pre-trained front-end extracts speech embeddings, and the architecture search process identifies a suitable classification architecture with reduced manual design. Though this alleviates the burden of designing the architecture, it can be very computationally intensive, and performance remains dependent on the quality of the pretrained representation.

Late fusion of ensembles of fine-tuned WavLM models shows the increased robustness resulting from combining model predictions [77]. Their usage in the ASVspoof5 challenge demonstrates the effectiveness of SSL ensembles in dealing with diverse attack conditions. But ensembling of models is an additional requirement, which increases memory consumption and latency.

Another technique is provided by Siamese learning to boost discrimination without increasing the size of the model too much. The combination of Wav2Vec features and a Siamese SENet34 model achieved an EER of 1.21%, which was higher than an FF-LCNN baseline that obtained an EER of 2.06% [78]. This result suggests that learning similarities can improve the separation of real and fake representations. However, Siamese systems should be carefully designed and optimized since their performance depends on the selected margins and other training hyperparameters.

Overall, the techniques based on SSL exhibit higher robustness compared to handcrafted features, especially in cases of unknown attacks, compression, codec variation, and cross-dataset differences [74-78]. The main problems of these techniques are expensive pretraining, high resource needs, sensitivity to fine-tuning, and the presence of biases from original pretraining data in the models.

5.4 Attention, multi-branch, and feature-fusion architectures

Another research direction has explored attention mechanisms, multi-branch processing, and feature fusion for complementary acoustic representations. These models try to boost discrimination ability by utilizing multiple feature spaces, including spectral, temporal, semantic, emotional, and pretrained representations rather than just one.

The combination of three-dimensional CNNs, bidirectional LSTMs, attention mechanisms, and classic classification modules enables the modeling of both local time-frequency structure and high-level emotional or semantic features [79]. The reported AUC of 0.98 shows that complementary emotional signals may help to distinguish synthetic speech. However, the usage of various DL components increases computational costs and may restrict practical scalability.

Multi-feature attention architectures were also trained on the MFCC, LFCC, and Chroma-STFT representations before applying dense classification modules [80]. As a result of the utilization of multiple complementary acoustic views, such neural network architectures show better performance compared to simple CNN and rule-based models. The reported results on the in-the-wild and FoR datasets confirm the effectiveness of feature fusion for various recording conditions. However, the success of feature fusion heavily depends on the choice of features, since the latter must be complementary.

Attention blocks have been added to existing CNN networks to highlight the most informative spectral and temporal regions. VGGish network enhanced by a Convolutional Block Attention Module reached extremely low EER in the ASVspoof2019 contest, which suggests that channel and spatial attention might improve the process of extracting spoofing features [81]. On the other hand, preprocessing intensity and noise sensitivity may significantly decrease the reliability of such systems in practice.

Efficiency-oriented attention systems have tried to combine high feature specificity with relatively low resource usage. A combination of the MobileNetV2 network and multi-frequency attention achieved good accuracy in the FoR-norm dataset by highlighting informative frequency features [82]. However, even though this architecture is less resource-intensive than many transformer-based networks, its evaluation is still limited, and its multilayered structure increases implementation complexity.

Feature-specific ensemble methods proved that the choice of acoustic representation also plays a significant role in the success of the system. Evaluating MFCC, CQCC, Mel-spectrogram, and spectral-centroid representations using pretrained CIFAR10 and ResNet50 architectures revealed that MFCC input data performed best among other variants of the FoR dataset, reaching 99.12% accuracy, 97.54% precision, 98.44% recall, 98.12% F1-score, and 0.88% EER [19]. These findings show the utility of hybridizing pretrained CNNs with discriminative acoustic front-ends. Nevertheless, such an overreliance on one specific dataset poses problems concerning overfitting and cross-domain validity.

Attention-based and fusion models can thus help to enhance discrimination through the combination of various acoustic views as well as focusing on salient parts [19, 79-82]. Limitations of their application are greater complexity, the need for proper feature combinations, costly preprocessing, and a lack of confirmation of reliable behavior in unseen acoustic conditions and under unknown data generation.

5.5 Cross-dataset generalization and robustness to emerging attacks

Despite considerable advances in terms of benchmark performance, generalization to new attacks, new languages, new datasets, codecs, and recording settings is the critical issue in deepfake detection. High performance in a well-known benchmark does not always imply the learning of generic properties of the spoofing process by the model.

It has been demonstrated that direct comparison of handcrafted and learned representations leads to the conclusion that manually designed features can perform very poorly under out-of-domain circumstances. Training a ResNet-18 classifier on ASVspoof2019 and evaluating its performance on ASVspoof2021 DF and in-the-wild subsets yielded EER greater than 50% for handcrafted representations [83]. In contrast, SSL representations, such as those obtained from XLS-R, HuBERT, and WavLM, significantly decreased EER, especially when several learned representations were combined. This comparison proves that learned speech representations are much more transferable than handcrafted descriptors. Nevertheless, the remaining EER clearly shows that SSL does not fully solve the generalization problem.

Knowledge distillation has been studied to reduce the computational complexity of models. Self-distillation approaches leverage fine details provided by the deeper teacher neural network and propagate them to the shallow student model, making the inference faster without sacrificing significant performance gains [84]. Testing on the logical- and physical-access subsets of ASVspoof2019 revealed some gains compared to baseline systems. On the other hand, the findings have been confirmed only on one dataset family, and the performance of the distilled models under practical conditions of mismatch has yet to be explored.

Codec-based audio-language models that emerged rapidly have presented a different threat due to dissimilar artifacts compared to the standard vocoder models and voice converters. To train detection models against codec-based attacks, special codec-aware data collections and training procedures were developed [85]. The codec-aware model reported the EER of 0.616%, outperforming the baseline models and showing that attack-specific training is beneficial for next-generation synthetic speech attacks. At the same time, the speech-oriented architecture and lack of noisy-uncontrolled testing limit the conclusions. Gated convolutional architectures based on XLS-R representations have also been considered for cross-domain and multilingual robustness. Centered kernel alignment-based diversity regularization promotes complementary learning of different branches and avoids redundancy in feature learning [37]. Low EER values on ASVspoof2019 and ASVspoof2021 indicate high multilingual and cross-domain capability. However, the model needs diverse training data and is computationally heavy.

The above results demonstrate that the robust audio deepfake detector requires more than just a large-capacity classifier. Generalization is highly dependent on training diversity, transferable representations, adaptation to new synthesis techniques, multilingual testing, and strategies to prevent domain-specific learning [37, 83-85].

Overall, DL-based audio deepfake detection has evolved from CNN-based spectral modeling toward transformer-style sequence modeling, SSL-based representation learning, attention-driven feature fusion, and generalization-focused training. CNN and ResNet models are effective for detecting local spectral artifacts, while Conformer and Mamba-based architectures improve long-range temporal modeling. SSL representations offer stronger robustness than handcrafted features, and fusion or attention mechanisms can enhance discriminative learning by combining complementary cues. Despite these advances, the shared limitations remain clear: many systems are still evaluated mainly on benchmark datasets, performance may drop under cross-dataset or real-world conditions, and model robustness depends on training diversity, feature representation, architectural complexity, and unseen attack types. Therefore, the main research gap is the need for audio deepfake detection systems that combine high accuracy, interpretability, computational efficiency, and reliable generalization across unseen datasets, languages, codecs, and synthesis methods.