o j|@sZddlZddlmZddlmZGdddejZGdddejZGdd d ejZdS) N)nn) functionalc2eZdZdZd fdd ZddZdd ZZS) LinearaLinear layer with a specific initialization. Args: in_features (int): number of channels in the input tensor. out_features (int): number of channels in the output tensor. bias (bool, optional): enable/disable bias in the layer. Defaults to True. init_gain (str, optional): method to compute the gain in the weight initializtion based on the nonlinear activation used afterwards. Defaults to 'linear'. Tlinearcs,ttjj|||d|_||dS)Nbias)super__init__torchrr linear_layer_init_wself in_features out_featuresr init_gain __class__X/home/kuhnn/.local/lib/python3.10/site-packages/TTS/tts/layers/tacotron/common_layers.pyr s zLinear.__init__cC$tjjj|jjtjj|ddSN)gainr rinitxavier_uniform_r weightcalculate_gainrrrrrr $zLinear._init_wcCs ||S)N)r )rxrrrforwards zLinear.forwardTr__name__ __module__ __qualname____doc__r r r" __classcell__rrrrrs  rcr) LinearBNaLinear layer with Batch Normalization. x -> linear -> BN -> o Args: in_features (int): number of channels in the input tensor. out_features (int ): number of channels in the output tensor. bias (bool, optional): enable/disable bias in the linear layer. Defaults to True. init_gain (str, optional): method to set the gain for weight initialization. Defaults to 'linear'. Trcs>ttjj|||d|_tj|ddd|_||dS)Nrg?gh㈵>)momentumeps) r r r rrr BatchNorm1dbatch_normalizationr rrrrr (s zLinearBN.__init__cCrrrrrrrr .r zLinearBN._init_wcCsP||}t|jdkr|ddd}||}t|jdkr&|ddd}|S)z< Shapes: x: [T, B, C] or [B, C] r)r lenshapepermuter.)rr!outrrrr"1s  zLinearBN.forwardr#r$rrrrr*s  r*cs8eZdZdZdddddgdffdd Zdd ZZS) PrenetaTacotron specific Prenet with an optional Batch Normalization. Note: Prenet with BN improves the model performance significantly especially if it is enabled after learning a diagonal attention alignment with the original prenet. However, if the target dataset is high quality then it also works from the start. It is also suggested to disable dropout if BN is in use. prenet_type == "original" x -> [linear -> ReLU -> Dropout]xN -> o prenet_type == "bn" x -> [linear -> BN -> ReLU -> Dropout]xN -> o Args: in_features (int): number of channels in the input tensor and the inner layers. prenet_type (str, optional): prenet type "original" or "bn". Defaults to "original". prenet_dropout (bool, optional): dropout rate. Defaults to True. dropout_at_inference (bool, optional): use dropout at inference. It leads to a better quality for some models. out_features (list, optional): List of output channels for each prenet block. It also defines number of the prenet blocks based on the length of argument list. Defaults to [256, 256]. bias (bool, optional): enable/disable bias in prenet linear layers. Defaults to True. originalTFcst||_||_||_|g|dd}|dkr-tfddt||D|_dS|dkrCtfddt||D|_dSdS)Nbncg|] \}}t||dqSr)r*.0in_sizeout_sizerrr jz#Prenet.__init__..r7cr;r<)rr=rrrrAnrB) r r prenet_typeprenet_dropoutdropout_at_inferencer ModuleListzip linear_layers)rrrCrDrErrrrrr Zs   zPrenet.__init__cCsH|jD]}|jrtjt||d|jp|jd}qt||}q|S)Ng?)ptraining)rHrDFdropoutrelurJrE)rr!rrrrr"qs $zPrenet.forward)r%r&r'r(r r"r)rrrrr6?sr6) r rtorch.nnrrKModulerr*r6rrrrs   #