o j]@s\ddlmZddlZddlmZddlmmZddlm Z ddl m Z de dee e ffddZ Gd d d ejZGd d d ejZGd ddejZGdddejZGdddejZGdddejZGdddejZGdddejZGdddejZGdddejZGdddejjjZGdd d ejjjZGd!d"d"ejjZdS)#)TupleN) parametrize)KernelPredictor kernel_sizereturncCs|d}|||ddfS)N)rpadr r \/home/kuhnn/.local/lib/python3.10/site-packages/TTS/tts/layers/delightful_tts/conv_layers.pycalc_same_padding sr cs:eZdZdZ       d fdd Zd d d ZZS) ConvNormaA 1-dimensional convolutional layer with optional weight normalization. This layer wraps a 1D convolutional layer from PyTorch and applies optional weight normalization. The layer can be used in a similar way to the convolutional layers in PyTorch's `torch.nn` module. Args: in_channels (int): The number of channels in the input signal. out_channels (int): The number of channels in the output signal. kernel_size (int, optional): The size of the convolving kernel. Defaults to 1. stride (int, optional): The stride of the convolution. Defaults to 1. padding (int, optional): Zero-padding added to both sides of the input. If `None`, the padding will be calculated so that the output has the same length as the input. Defaults to `None`. dilation (int, optional): Spacing between kernel elements. Defaults to 1. bias (bool, optional): If `True`, add bias after convolution. Defaults to `True`. w_init_gain (str, optional): The weight initialization function to use. Can be either 'linear' or 'relu'. Defaults to 'linear'. use_weight_norm (bool, optional): If `True`, apply weight normalization to the convolutional weights. Defaults to `False`. Shapes: - Input: :math:`[N, D, T]` - Output: :math:`[N, out_dim, T]` where `out_dim` is the number of output dimensions. rNTlinearFc stt||dur|ddksJt||dd}||_||_| |_tj} | |||||||d|_ tj j |j j tj |d|jrQtjj|j |_ dSdS)Nrr)rstridepaddingdilationbias)gain)superr __init__intrruse_weight_normnnConv1dconvinitxavier_uniform_weightcalculate_gainutilsparametrizations weight_norm) self in_channels out_channelsrrrrr w_init_gainrconv_fn __class__r r r.s*  zConvNorm.__init__cCs||}|dur ||}|SNr)r"signalmask conv_signalr r r forwardOs zConvNorm.forward)rrNrTrFr))__name__ __module__ __qualname____doc__rr. __classcell__r r r'r r s!!r csBeZdZ      dfdd Zd d Zd d Zd dZZS)ConvLSTMLinearr皙?bilstmTc  s.tt|||_||_||_tj|d|_g} t |D](} t | dkr&|n|||dt |ddddd} tj j j| jdd} | | qt| |_|jsQ|}|jd krd } |} |jd krgd } t |d} tj|| dd | d |_tj j}||jd|_|jd kr||jd|_|jrt|||_dSdS)N)prrrrelu)rrrrr%r)nameFr8T) batch_first bidirectional weight_hh_l0weight_hh_l0_reverse)rr4rout_dim lstm_type use_linearrDropoutdropoutranger rrr r!rappend ModuleList convolutionsLSTMr8 spectral_normLineardense)r"in_dimrAn_layers n_channelsr p_dropoutrBrCrIi conv_layer use_bilstm lstm_channelslstm_norm_fn_pntrr'r r rYsF        zConvLSTMLinear.__init__cCsg}t|dD]0}|||dddd||f}|jD] }|t||}q"||dddq t j j j |dd}|S)NrrTr=) rFsizeclonerIrEFr:rG transposerrrnn pad_sequence)r"contextlenscontext_embeddedb_ind curr_contextrr r r run_padded_sequences& z"ConvLSTMLinear.run_padded_sequencecCstj|dd\}}dg|d}tt|D]}||||<q|}||}tjj j ||dd}||d}tjj j |ddd}||}|S)NT) descendingrrW) torchsortrXrFlenlongcpurrr\pack_padded_sequencepad_packed_sequence)r"fnr^r_ lens_sorted ids_sorted unsort_idsrRr r r run_unsorted_inputss  z"ConvLSTMLinear.run_unsorted_inputscCs|ddkr|||}|dd}n|jD] }|t||}q|jdkrO|dd}|j |durB| |j||}n||d}|dd}|}|j ra| |dddd}|S)Nrrrr<) rXrcr[rIrErZr:rBr8flatten_parametersrprCrM)r"r^r_rx_hatr r r r.s       zConvLSTMLinear.forward)rr5r6r7r8T)r/r0r1rrcrpr.r3r r r'r r4Xs4 r4csDeZdZdedededeffdd Zdejdejfd d ZZS) DepthWiseConv1dr#r$rrcs$ttj|||||d|_dS)N)rgroupsrrrrrr"r#r$rrr'r r rs zDepthWiseConv1d.__init__xrcC ||Sr)r*r"rwr r r r. zDepthWiseConv1d.forward) r/r0r1rrreTensorr.r3r r r'r rssrsc sPeZdZ   ddededededef fd d Zd ejd ejfd dZZ S)PointwiseConv1drrTr#r$rrrcs&ttj||d|||d|_dS)Nr)r#r$rrrrru)r"r#r$rrrr'r r rs  zPointwiseConv1d.__init__rwrcCrxr)r*ryr r r r.rzzPointwiseConv1d.forward)rrT r/r0r1rboolrrer{r.r3r r r'r r|s r|cHeZdZdZdedededeffdd Zdejd ejfd d ZZ S) BSConv1d$https://arxiv.org/pdf/2003.13549.pdf channels_in channels_outrrc6ttj||dd|_tj|||||d|_dSNr)rrrrt)rrrr pointwise depthwiser"rrrrr'r r r  zBSConv1d.__init__rwrcC||}||}|Sr)rrr"rwx1x2r r r r.  zBSConv1d.forward r/r0r1r2rrrer{r.r3r r r'r r rcr) BSConv2drrrrrcrr)rrrConv2drrrr'r r rrzBSConv2d.__init__rwrcCrr)rrr r r r.rzBSConv2d.forwardrr r r'r rrrcsNeZdZdZdedededeffdd Zdejd ejd ejfd d ZZ S) Conv1dGLUzFrom DeepVoice 3d_modelrr embedding_dimcs^tt|d|||d|_t|||_|dt t dg dtj |_ dS)Nrrrsqrt?r)rrrrrrLembedding_projregister_bufferrer FloatTensorsqueezeSoftsignsoftsign)r"rrrrr'r r r s  zConv1dGLU.__init__rw embeddingsrcCs|d}|}||}d}|j||d|d\}}||d}||}||}||}|t |}||}||j }|d}|S)Nrrrrrdim) permutersplitrXr unsqueezer expand_asresigmoidr)r"rwrresidualsplitdimabrr r r r.s      zConv1dGLU.forwardrr r r'r rs$rc sNeZdZdZ  ddedededeffdd Zd ejd ejfd d ZZ S)ConvTransposeda A 1D convolutional transposed layer for PyTorch. This layer applies a 1D convolutional transpose operation to its input tensor, where the number of channels of the input tensor is the same as the number of channels of the output tensor. Attributes: in_channels (int): The number of channels in the input tensor. out_channels (int): The number of channels in the output tensor. kernel_size (int): The size of the convolutional kernel. Default: 1. padding (int): The number of padding elements to add to the input tensor. Default: 0. conv (BSConv1d): The 1D convolutional transpose layer. rrr#r$rrcs tt||||d|_dS)Nr)rrrrrvr'r r r1s  zConvTransposed.__init__rwrcCs.|dd}||}|dd}|S)Nrr) contiguousr[rryr r r r.@s zConvTransposed.forward)rrrr r r'r r#src sFeZdZddedededeffdd Zd ejd ejfd d ZZ S)DepthwiseConvModule333333?rr expansion lrelu_slopecsdtt|}tj|||||d|d|_t||_t|||ddd|_t ||_ dS)Nrrr) rrr rrr LeakyReLUactout LayerNormln)r"rrrrrr'r r rHs  zDepthwiseConvModule.__init__rwrcCs@||}|d}||}||}||}|d}|S)Nr)rrrrrryr r r r.Vs      zDepthwiseConvModule.forward)rrr) r/r0r1rfloatrrer{r.r3r r r'r rGs rcs>eZdZd dedeffdd Zdejdejfdd ZZ S) AddCoordsFrankwith_rcst||_||_dSr))rrrr)r"rrr'r r ras  zAddCoords.__init__rwrcsJ|jdkrY|j\}}}tj|tjd}|ddddf}||d}|dd}||dd}||j}tj ||gdd}|j rWt t |dd}tj ||gdd}|S|jdkr|j\}}} }tj ddd|gtjd} tj ddd| gtjd} tj| tjd}tj|tjd} |dddddf}| dddddf} t|| }t| | } | dddd} || d}| |d} |dd}| dd} ||ddd}| |ddd} ||j}| |j} tj ||| gdd}|j rt t |ddt | dd}tj ||gdd}|S|jdkr#|j\}}}} }tj dddd|gtjd} tj dddd| gtjd} tj dddd|gtjd}tj| tjd}|ddddddf}tj|tjd}|ddddddf}tj|tjd}|ddddddf}t|| tj fdd t|Ddd}t|| dddd dtj fd d t|Dd d} t||ddd ddtj fd d t| Ddd}||j}| |j} ||j}tj ||| |gdd}|j r!t t |ddt | ddt |dd}tj ||gdd}|St) Nr)dtyperrrrr6cg|]}|qSr r .0rR) xy_channelr r z%AddCoords.forward..rcrr r r) yz_channelr r rrcrr r r) zx_channelr r rr)rshaperearangeint32rrepeattodevicecatrrpowonesmatmulrrFNotImplementedError)r"rwbatch_size_shapechannel_in_shapedim_xxx_range xx_channelrrrdim_yxx_onesyy_onesyy_range yy_channeldim_zzz_onesxy_rangeyz_rangezx_range zz_channelr )rrrr r.fs    K       &)          .zAddCoords.forward)Fr}r r r'r r`srcfeZdZ      ddedededed ed ed ed ed effdd ZdejdejfddZZ S) CoordConv1drrTFr#r$rrrrrtrrc Zt||||||||d|_t|j| |_t||jt| ||||||||_dS)Nr) rrrr addcoordsrrrr r"r#r$rrrrrtrrr'r r r,   zCoordConv1d.__init__rwrcC||}||}|Sr)rrryr r r r.rzCoordConv1d.forwardrrrrTFr}r r r'r r6  #rcr) CoordConv2drrTFr#r$rrrrrtrrc r)Nr) rrrrrrrrrrr'r r rrzCoordConv2d.__init__rwrcCrr)rryr r r r.rzCoordConv2d.forwardrr}r r r'r rrrcsNeZdZdZgdddddddffdd Zd d Zdd dZddZZS)LVCBlockz"the location-variable convolutions)rr6 g?r6r5@gc st||_t||_||_t||d|t|||| | d|id |_t t |tj j tj||d|||d|d|dd|_t|_|D]&} |jt t |tj j tj|||| |dd| dt |qMdS)Nrnegative_slope) cond_channelsconv_in_channelsconv_out_channels conv_layersconv_kernel_sizekpnet_hidden_channelskpnet_conv_size kpnet_dropout!kpnet_nonlinear_activation_params)rroutput_paddingr)rr)rrcond_hop_lengthrgrrrkernel_predictorr Sequentialrrr r!ConvTranspose1d convt_prerH conv_blocksrGr) r"r#rr dilations lReLU_sloperrrrrrr'r r rs\     zLVCBlock.__init__c Cs|j\}}}||}||\}}t|jD]V\}}||} |dd|ddddddddf} |dd|ddddf} |j| | | |jd} |t| ddd|ddft | dd|dddf}q|S)aLforward propagation of the location-variable convolutions. Args: x (Tensor): the input sequence (batch, in_channels, in_length) c (Tensor): the conditioning sequence (batch, cond_channels, cond_length) Returns: Tensor: the output sequence (batch, in_channels, in_length) N)hop_size) rrr enumeraterlocation_variable_convolutionrrertanh) r"rwc_r#kernelsrrRroutputkrr r r r.Ys ( $ zLVCBlock.forwardrcCs,|j\}}}|j\}}} } } || |ksJd|t| dd} t|| | fdd}|d|d| |}||krEt|d|fdd}|d||}|ddddddddd|f}|dd}|d| d}td ||} | jtj d } | d  d jtj d }| |} | || d } | S) uperform location-variable convolution operation on the input sequence (x) using the local convolution kernl. Time: 414 μs ± 309 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each), test on NVIDIA V100. Args: x (Tensor): the input sequence (batch, in_channels, in_length). kernel (Tensor): the local convolution kernel (batch, in_channel, out_channels, kernel_size, kernel_length) bias (Tensor): the bias for the local convolution (batch, out_channels, kernel_length) dilation (int): the dilation of convolution. hop_size (int): the hop_size of the conditioning sequence. Returns: (Tensor): the output sequence after performing local convolution. (batch, out_channels, in_length). z$length of (x, kernel) is not matchedrrconstantrr6Nrzbildsk,biokl->bolsd) memory_format) rrrZr unfoldr[reeinsumrchannels_last_3drrview)r"rwkernelrrrbatchr  in_lengthr$r kernel_lengthror r r rvs( & z&LVCBlock.location_variable_convolutioncCs<|jt|jdd|jD] }t|ddqdS)Nrr)rremove_weight_normrremove_parametrizationsrr)r"blockr r r rs  zLVCBlock.remove_weight_norm)rr5) r/r0r1r2rr.rrr3r r r'r rs? %r) typingrretorch.nnrtorch.nn.functional functionalrZtorch.nn.utilsr.TTS.tts.layers.delightful_tts.kernel_predictorrrr Moduler r4rsr|rrrrrrmodulesrrrrrrr r r r s(    Hh $c**