o jaI@sddlZddlmZddlmZddlmZGdddejZGdd d ejZGd d d ejZ Gd d d ejZ GdddejZ GdddejZ GdddejZ GdddejZdS)N)nn) init_attn)Prenetcs2eZdZdZd fdd ZddZddZZS) BatchNormConv1da4A wrapper for Conv1d with BatchNorm. It sets the activation function between Conv and BatchNorm layers. BatchNorm layer is initialized with the TF default values for momentum and eps. Args: in_channels: size of each input sample out_channels: size of each output samples kernel_size: kernel size of conv filters stride: stride of conv filters padding: padding of conv filters activation: activation function set b/w Conv1d and BatchNorm Shapes: - input: (B, D) - output: (B, D) NcsRt||_t|d|_tj||||ddd|_tj|ddd|_ ||_ dS)NrF) kernel_sizestridepaddingbiasgGz?gMbP?)momentumeps) super__init__r r ConstantPad1dpadderConv1dconv1d BatchNorm1dbn activation)self in_channels out_channelsrrr r __class__S/home/kuhnn/.local/lib/python3.10/site-packages/TTS/tts/layers/tacotron/tacotron.pyrs   zBatchNormConv1d.__init__cCsht|jtjjr d}nt|jtjjrd}n |jdurd}ntdtjjj|j j tjj |ddS)NrelutanhlinearzUnknown activation functiongain) isinstancertorchrReLUTanh RuntimeErrorinitxavier_uniform_rweightcalculate_gain)rw_gainrrr init_layers)s $zBatchNormConv1d.init_layerscCs6||}||}||}|jdur||}|SN)rrrrrxrrrforward4s     zBatchNormConv1d.forwardr-__name__ __module__ __qualname____doc__rr,r0 __classcell__rrrrr s   rcs0eZdZdZfddZddZddZZS)HighwayaHighway layers as explained in https://arxiv.org/abs/1505.00387 Args: in_features (int): size of each input sample out_feature (int): size of each output sample Shapes: - input: (B, *, H_in) - output: (B, *, H_out) cs\tt|||_|jjjt|||_|jjj dt |_ t |_ dSN)r rrLinearHr datazero_Tfill_r$rSigmoidsigmoid)r in_features out_featurerrrrJs  zHighway.__init__cCsDtjjj|jjtjjddtjjj|jjtjjdddS)Nrr rA)r#rr'r(r;r)r*r>rrrrr,Ts $zHighway.init_layerscCs4|||}|||}|||d|S)Ng?)rr;rAr>)rinputsr;r>rrrr0XszHighway.forwardr1rrrrr7=s   r7cs:eZdZdZddddgdddffdd ZddZZS) CBHGaCBHG module: a recurrent neural network composed of: - 1-d convolution banks - Highway networks + residual connections - Bidirectional gated recurrent units Args: in_features (int): sample size K (int): max filter size in conv bank projections (list): conv channel sizes for conv projections num_highways (int): number of highways layers Shapes: - input: (B, C, T_in) - output: (B, T_in, C*2) c s4t___|_|_t_ t fddt d|dD_ |g|dd}j gt |d} | dg7} g} t||| D]\} } } t| | ddddg| d}| |qOt | _j|dkr~tj|ddd_t fd dt |D_tj||dd d d _dS) Nc s2g|]}t|d|dd|dgjdqS)rrrr r)rr).0k)conv_bank_featuresrBrrr s z!CBHG.__init__..rr9rKF)r csg|]}tqSr)r7rL_)highway_featuresrrrOsT) batch_first bidirectional)r rrBrNrS gru_featuresconv_projectionsrr$r ModuleListrange conv1d_bankslenziprappendconv1d_projectionsr: pre_highwayhighwaysGRUgru)rrBKrNrWrSrV num_highways out_features activations layer_setin_sizeout_sizeaclayerr)rNrSrBrrrps0      z CBHG.__init__c Cs|}g}|jD] }||}||qtj|dd}|d|jt|jks)J|jD]}||}q,||7}|dd}|j |j dkrJ| |}|j D]}||}qM|j | |\}}|S)NrdimrJr9)rZr]r#catsizerNr[r^ transposerSrWr_r`rbflatten_parameters) rrEr/outsrouthighwayoutputsrRrrrr0s$         z CBHG.forwardr2r3r4r5rr0r6rrrrrF^s2rFc(eZdZdZfddZddZZS) EncoderCBHGz+CBHG module with Encoder specific argumentsc s*ttdddddgdddd|_dS)NrHrGrIrcrNrWrSrVrdr rrFcbhgrDrrrr  zEncoderCBHG.__init__cC ||Sr-r{r.rrrr0 zEncoderCBHG.forwardrvrrrrrxs  rxcrw)EncoderzStack Prenet and CBHG module for encoder Args: inputs (FloatTensor): embedding features Shapes: - inputs: (B, T, D_in) - outputs: (B, T, 128 * 2) cs(tt|ddgd|_t|_dS)NrHre)r rrprenetrxr{rrBrrrrs  zEncoder.__init__cCs ||}||dd}|S)NrrJ)rr{rprrErurrrr0s zEncoder.forwardrvrrrrrs rcs$eZdZfddZddZZS)PostCBHGc s*tt|ddd|gdddd|_dS)NrHrrIryrz)rmel_dimrrrrr|zPostCBHG.__init__cCr}r-r~r.rrrr0rzPostCBHG.forward)r2r3r4rr0r6rrrrrs  rcsbeZdZdZfddZddZddZdd Zd d Zdd dZ ddZ ddZ ddZ Z S)DecoderaTacotron decoder. Args: in_channels (int): number of input channels. frame_channels (int): number of feature frame channels. r (int): number of outputs per time step (reduction rate). memory_size (int): size of the past window. if <= 0 memory_size = r attn_type (string): type of attention used in decoder. attn_windowing (bool): if true, define an attention window centered to maximum attention response. It provides more robust attention alignment especially at interence time. attn_norm (string): attention normalization function. 'sigmoid' or 'softmax'. prenet_type (string): 'original' or 'bn'. prenet_dropout (float): prenet dropout rate. forward_attn (bool): if true, use forward attention method. https://arxiv.org/abs/1807.06736 trans_agent (bool): if true, use transition agent. https://arxiv.org/abs/1807.06736 forward_attn_mask (bool): if true, mask attention values smaller than a threshold. location_attn (bool): if true, use location sensitive attention. attn_K (int): number of attention heads for GravesAttention. separate_stopnet (bool): if true, detach stopnet input to prevent gradient flow. d_vector_dim (int): size of speaker embedding vector, for multi-speaker training. max_decoder_steps (int): Maximum number of steps allowed for the decoder. Defaults to 500. cs t||_||_||_||_|dk|_|dkr|n||_||_||_ d|_ |jr0||jn|}t ||| ddgd|_ t |d|j |_t||j |d| dd||| | | |d |_t d|d|_t dd td D|_t d||j|_td||j|_dS) NrrrHr ) attn_type query_dim embedding_dim attention_dimlocation_attentionattention_location_n_filtersattention_location_kernel_size windowingnorm forward_attn trans_agentforward_attn_maskattn_KcSsg|]}tddqS)r)rGRUCellrQrrrrOJsz$Decoder.__init__..rJ)r rr_initrrmax_decoder_stepsuse_memory_queue memory_sizeframe_channelsseparate_stopnetrrrrr attention_rnnr attentionr:project_to_decoder_inrXrY decoder_rnns proj_to_melStopNetstopnet)rrrrrrattn_windowing attn_norm prenet_typeprenet_dropoutrrr location_attnrrr prenet_dimrrrrs@  zDecoder.__init__cCs ||_dSr-)r)rnew_rrrrset_rPrz Decoder.set_rcCs@|d|jkr||jd|d|jd}|dd}|S)z8 Reshape the spectrograms for given 'r' r9rr)rorviewshaperrp)rmemoryrrr_reshape_memorySs  zDecoder._reshape_memorycsd|jrtjdjd|j|j|_ntjdjd|j|_tjdjdd|_ fddt t |j D|_ j|j|_|j|_dS)z2 Initialization of decoder states rrdevicercs$g|]}tjdjddqS)rrr)r#zerosrrepeat)rLidxBrErrrOjsz(Decoder._init_states..N)rorr#rrrrr memory_inputattention_rnn_hiddenrYr[rdecoder_rnn_hiddensr<newrr= context_vecrpreprocess_inputsprocessed_inputs)rrErrr _init_states^s $  zDecoder._init_statescCsft|dd}t|dd}t|dd}||dd|j}|dd}|||fS)Nrrr9rJ)r#stackrp contiguousrror)rru attentions stop_tokensrrr_parse_outputsqs   zDecoder._parse_outputsNc Cs||j}|t||jfd|j|_||j||j||_| t|j|jfd}t |j D]\}}|||j ||j |<|j ||}q2|}| |}t||gd} |jrc|| } n|| } |ddd|j|jf}|| |jjfSr8)rrrr#rnrrrrr enumeraterrrrrdetachrrattention_weights) rrEmaskprocessed_memory decoder_inputr decoder_rnndecoder_outputoutput stopnet_input stop_tokenrrrdecodezs"   zDecoder.decodecCs|jr8|j|jkr'tj||jddd|j|j|jfgdd|_dS|ddd|j|jf|_dS|dd|j|jddf|_dS)Nr9rlr)rrrr#rnrrclone)r new_memoryrrr_update_memory_inputs * "&zDecoder._update_memory_inputc Cs||}g}g}g}d}|||j|t||dkrX|dkr0||d}|||||\} } } || g7}|| g7}|| dg7}|d7}t||dks!| |||S)a Args: inputs: Encoder outputs. memory: Decoder memory (autoregression. If None (at eval-time), decoder outputs are used as decoder inputs. If None, it uses the last output as the input. mask: Attention mask for sequence padding. Shapes: - inputs: (B, T, D_out_enc) - memory: (B, T_mel, D_mel) rr) rrr init_statesr[rorrsqueezer) rrErrrurrtrrrrrrrr0s$        zDecoder.forwardc Csg}g}g}d}|||j| |dkr!|d}||||d\}}} t|j}||g7}|| g7}||g7}|d7}||jddkr]|dks\| dddf dkr]n ||j krgt dnq| |||S) z Args: inputs: encoder outputs. Shapes: - inputs: batch x time x encoder_out_dim rTr9NrrIg333333?z. | > Decoder stopped with 'max_decoder_steps) rrrrrr#rAr<ritemrprintr) rrErurrrrrrrrrr inferences.       2 zDecoder.inferencer-)r2r3r4r5rrrrrrrr0rr6rrrrrs ;  !rcrw)rzsStopnet signalling decoder to stop inference. Args: in_features (int): feature dimension of input. csHttd|_t|d|_tjjj |jj tjj dddS)Ng?rrr ) r rrDropoutdropoutr:rr#r'r(r)r*rrrrrs  $zStopNet.__init__cCs||}||}|Sr-)rrrrrrr0s  zStopNet.forwardrvrrrrrs r)r#rrr common_layersrModulerr7rFrxrrrrrrrrs   2!ap