o j8@sddlmZddlmZmZmZddlZddlZddl m Z ddlm Z ddl m Z ddlmZddlmZmZdd lmZdd lmZdd lmZdd lmZmZdd lmZmZddlm Z ddl!m"Z"Gddde Z#dS)) signature)DictListTupleN)Coqpit)nn) DataLoader)DistributedSampler) get_optimizer get_scheduler)AudioProcessor) load_fsspec) GANDataset)DiscriminatorLoss GeneratorLoss)setup_discriminatorsetup_generator) BaseVocoder) plot_resultscseZdZd;dedeffdd Zdejdejfdd Zdejdejfd d Z d e d e de de e e ffddZ deded e de de e e ff ddZd e de ddde de de e ejff ddZed e d ejde de e e ffddZd e de ddde de de e ejff ddZ  ddeddfd9d:Z#Z$S)?GANNconfigapcs<t|||_t||_t||_d|_d|_||_ dS)uWrap a generator and a discriminator network. It provides a compatible interface for the trainer. It also helps mixing and matching different generator and disciminator networks easily. To implement a new GAN models, you just need to define the generator and the discriminator networks, the rest is handled by the `GAN` class. Args: config (Coqpit): Model configuration. ap (AudioProcessor): 🐸TTS AudioProcessor instance. Defaults to None. Examples: Initializing the GAN model with HifiGAN generator and discriminator. >>> from TTS.vocoder.configs import HifiganConfig >>> config = HifiganConfig() >>> model = GAN(config) FN) super__init__rrmodel_grmodel_d train_discy_hat_gr)selfrr __class__I/home/kuhnn/.local/lib/python3.10/site-packages/TTS/vocoder/models/gan.pyrs    z GAN.__init__xreturncC |j|S)zRun the generator's forward pass. Args: x (torch.Tensor): Input tensor. Returns: torch.Tensor: output of the GAN generator network. )rforwardrr#r!r!r"r&/s z GAN.forwardcCr%)zRun the generator's inference pass. Args: x (torch.Tensor): Input tensor. Returns: torch.Tensor: output of the GAN generator network. )r inferencer'r!r!r"r(:s z GAN.inferencebatch criterion optimizer_idxcCsi}i}|d}|d}|dvrtd|dkr||ddddd|df}||_d|_d|_|jddkrN||_|j|}||_|j||_d \} } } |j r|j j r|d } |d } t || }Wdn1svwY|jddkr|j|}n |} |} |j}tt|jjjdkr||| }|| | }n ||}|| }t|tr|\} } |durd \}} n |\}} n|} |}||| |}d |i}|dkr\d \} } } |j rHtt|jjjdkr||j|}n||j}d}|j jr+t ||}Wdn 1s&wYt|trB|\} } |dur=d} n |\}} n|} d \} } |||j|| | | |j|j}d |ji}||fS)a0Compute model outputs and the loss values. `optimizer_idx` selects the generator or the discriminator for network on the current pass. Args: batch (Dict): Batch of samples returned by the dataloader. criterion (Dict): Criterion used to compute the losses. optimizer_idx (int): ID of the optimizer in use on the current pass. Raises: ValueError: `optimizer_idx` is an unexpected value. Returns: Tuple[Dict, Dict]: model outputs and the computed loss values. inputwaveform)rz [!] Unexpected `optimizer_idx`.rNr.)NNN input_disc waveform_disc)NN model_outputs) ValueErrorrsizer y_hat_suby_sub_gshapepqmf_synthesis pqmf_analysisrrdiff_samples_for_G_and_Dtorchno_gradclonelenrrr& parametersdetach isinstancetupleuse_feat_match_loss)rr)r*r+outputs loss_dictr#yy_hat scores_fake feats_fake feats_realx_dy_d D_out_fake D_out_real scores_real_r!r!r" train_stepDs&                   zGAN.train_stepnamerDc Csd|jr |ddn|dd}|d}t||||}|dd}|d|i} || fS)aLogging shared by the training and evaluation. Args: name (str): Name of the run. `train` or `eval`, ap (AudioProcessor): Audio processor used in training. batch (Dict): Batch used in the last train/eval step. outputs (Dict): Model outputs from the last train/eval step. Returns: Tuple[Dict, Dict]: log figures and audio samples. rr2r.r-z/audio)rrsqueezer@cpunumpy) rrRrr)rDrGrFfigures sample_voiceaudiosr!r!r"_logs  zGAN._logloggerLoggerassetsstepscC8|d|j||\}}|||||||jjdS)zCall `_log()` for training.evalNrYr eval_figures eval_audios sample_raterr)rDrZr\r]rVrXr!r!r" train_log z GAN.train_logcCsd|_||||S)z$Call `train_step()` with `no_grad()`T)rrQ)rr)r*r+r!r!r" eval_stepsz GAN.eval_stepcCr^)zCall `_log()` for evaluation.r_Nr`rdr!r!r"eval_logrfz GAN.eval_logFcheckpoint_pathr_cachecCsjt|td|d}d|vr|j|||dS||d|r1d|_t|jdr3|jdSdSdS)aLoad a GAN checkpoint and initialize model parameters. Args: config (Coqpit): Model config. checkpoint_path (str): Checkpoint file path. eval (bool, optional): If true, load the model for inference. If falseDefaults to False. rT) map_locationrj model_discmodelNremove_weight_norm) r r;devicerload_checkpointload_state_dictrhasattrrn)rrrir_rjstater!r!r"rps zGAN.load_checkpointcCs|j|jjk|_dS)zEnable the discriminator training based on `steps_to_start_discriminator` Args: trainer (Trainer): Trainer object. N)total_steps_donersteps_to_start_discriminatorr)rtrainerr!r!r"on_train_step_startszGAN.on_train_step_startcCs@t|jj|jj|jj|j}t|jj|jj|jj|j}||gS)zInitiate and return the GAN optimizers based on the config parameters. It returnes 2 optimizers in a list. First one is for the generator and the second one is for the discriminator. Returns: List: optimizers. )r r optimizeroptimizer_paramslr_genrlr_discr)r optimizer1 optimizer2r!r!r"r szGAN.get_optimizercCs|jj|jjgS)zSet the initial learning rates for each optimizer. Returns: List: learning rates for each optimizer. )rr{rzrr!r!r"get_lrsz GAN.get_lrcCs8t|jj|jj|d}t|jj|jj|d}||gS)zSet the schedulers for each optimizer. Args: optimizer (List[`torch.optim.Optimizer`]): List of optimizers. Returns: List: Schedulers, one for each optimizer. rr.)r rlr_scheduler_genlr_scheduler_gen_paramslr_scheduler_disclr_scheduler_disc_params)rrx scheduler1 scheduler2r!r!r"r s zGAN.get_schedulercCsFt|dtr|d\}}|d\}}||||dS|\}}||dS)zFormat the batch for training. Args: batch (List): Batch out of the dataloader. Returns: Dict: formatted model inputs. rr.)r,r-r0r1)r,r-)rAlist)r)x_Gy_Gx_Dy_Dr#rFr!r!r" format_batch+s    zGAN.format_batchis_evalTsamplesverbosenum_gpusrankc Cst|j||j|jj|j|jd|vr|jnd| | |j|j|d }| |dkr/t |ddnd} t ||r7dn|j |dkd| |rC|j n|jdd } | S) aInitiate and return the GAN dataloader. Args: config (Coqpit): Model config. ap (AudioProcessor): Audio processor. is_eval (True): Set the dataloader for evaluation if true. samples (List): Data samples. verbose (bool): Log information if true. num_gpus (int): Number of GPUs in use. rank (int): Rank of the current GPU. Defaults to None. Returns: DataLoader: Torch dataloader. r:F) ritemsseq_lenhop_len pad_shortconv_pad return_pairs is_trainingreturn_segmentsuse_noise_augment use_cacherr.T)shuffleNr) batch_sizer drop_lastsampler num_workers pin_memory)rrr hop_lengthrrr:rrshuffle_mappingr rrnum_eval_loader_workersnum_loader_workers) rrr\rrrrrdatasetrloaderr!r!r"get_data_loader<s4  zGAN.get_data_loadercCst|jt|jgS)z$Return criterions for the optimizers)rrrr~r!r!r" get_criterionoszGAN.get_criterioncCstj||d}t||dS)N)r)r)r init_from_configr)rrrr!r!r"rss zGAN.init_from_config)N)FF)r$N)T)%__name__ __module__ __qualname__rr rr;Tensorr&r(rintrrQstrrYnpndarrayrer<rModulergrhboolrprwrr rr staticmethodrrrr __classcell__r!r!rr"rs " &x  &      3r)$inspectrtypingrrrrUrr;coqpitrrtorch.utils.datartorch.utils.data.distributedr trainer.trainer_utilsr r TTS.utils.audior TTS.utils.ior TTS.vocoder.datasets.gan_datasetrTTS.vocoder.layers.lossesrrTTS.vocoder.modelsrrTTS.vocoder.models.base_vocoderrTTS.vocoder.utils.generic_utilsrrr!r!r!r"s"