o jkY@sddlZddlZddlmZddlZddlZddlZddlmZddl m Z ddl m Z ddl mZddlmZddlmZmZmZdd lmZdd lmZddlmZddlmZdd lmZGd d d ej Z!dS)N)List)nn) load_config) VitsConfig) setup_model)Vits) synthesistransfer_voice trim_silence)AudioProcessor)save_wav)interpolate_vocoder_inputcseZdZ             d7dededededed ed ed ed ed ededededdffdd ZedefddZd edededdfddZdededdfddZ dededdfddZ dedededdfddZ d d!Z d"ed#ededdfd$d%Z deefd&d'Zd8d(eed)eddfd*d+Zd,ed-edeefd.d/Z         0d9d1ed2ed3ed4edeef d5d6ZZS): SynthesizerNFtts_checkpointtts_config_pathtts_speakers_filetts_languages_filevocoder_checkpointvocoder_configencoder_checkpointencoder_config vc_checkpoint vc_config model_dir voice_diruse_cudareturncs^t||_||_||_||_||_||_||_||_ | |_ | |_ | |_ d|_ d|_d|_d|_i|_d|_d|_i|_d|_|d|_| |_ | |_|j rYtjsYJd|ri|||| |jjd|_|ry| ||| |jjd|_| r|!| | | |j jd|_| rd| vr|"| | |jjd|_dS|#| | |jjd|_dSdS)uGeneral 🐸 TTS interface for inference. It takes a tts and a vocoder model and synthesize speech from the provided text. The text is divided into a list of sentences using `pysbd` and synthesize speech on each sentence separately. If you have certain special characters in your text, you need to handle them before providing the text to Synthesizer. TODO: set the segmenter based on the source language Args: tts_checkpoint (str, optional): path to the tts model file. tts_config_path (str, optional): path to the tts config file. vocoder_checkpoint (str, optional): path to the vocoder model file. Defaults to None. vocoder_config (str, optional): path to the vocoder config file. Defaults to None. encoder_checkpoint (str, optional): path to the speaker encoder model file. Defaults to `""`, encoder_config (str, optional): path to the speaker encoder config file. Defaults to `""`, vc_checkpoint (str, optional): path to the voice conversion model file. Defaults to `""`, vc_config (str, optional): path to the voice conversion config file. Defaults to `""`, use_cuda (bool, optional): enable/disable cuda. Defaults to False. Nrenz%CUDA is not availabe on this machine. sample_rateoutput_sample_ratefairseq)$super__init__rrrrrrrrrrr tts_model vocoder_modelvc_modelspeaker_manager tts_speakerslanguage_manager num_languages tts_languages d_vector_dim_get_segmentersegrtorchcuda is_available _load_tts tts_configaudior _load_vocoder_load_vc_load_fairseq_from_dir_load_tts_from_dir)selfrrrrrrrrrrrrr __class__H/home/kuhnn/.local/lib/python3.10/site-packages/TTS/utils/synthesizer.pyr#sT &   zSynthesizer.__init__langcCstj|ddS)zget the sentence segmenter for the given language. Args: lang (str): target language code. Returns: [type]: [description] T)languageclean)pysbd Segmenter)r>r<r<r=r-ps zSynthesizer._get_segmentervc_config_pathcCs>t||_t|jd|_|j|j||r|jdSdS)aLoad the voice conversion model. 1. Load the model config. 2. Init the model from the config. 3. Load the model weights. 4. Move the model to the GPU if CUDA is enabled. Args: vc_checkpoint (str): path to the model checkpoint. tts_config_path (str): path to the model config file. use_cuda (bool): enable/disable CUDA use. configN)rrsetup_vc_modelr&load_checkpointr0)r9rrCrr<r<r=r6|s zSynthesizer._load_vccCsJt|_t|j|_|jj|j|dd|jj|_|r#|jdSdS)zLoad the fairseq model from a directory. We assume it is VITS and the model knows how to load itself from the directory and there is a config.json file in the directory. Tcheckpoint_direvalN)rr3rinit_from_configr$load_fairseq_checkpointrEr0)r9rrr<r<r=r7s z"Synthesizer._load_fairseq_from_dircCsJttj|d}||_t||_|jj||dd|r#|jdSdS)zLoad the TTS model from a directory. We assume the model knows how to load itself from the directory and there is a config.json file in the directory. z config.jsonTrHN) rospathjoinr3setup_tts_modelr$rGr0)r9rrrEr<r<r=r8s zSynthesizer._load_tts_from_dircCst||_|jdr|jddurtdt|jd|_|js#||jj|j|dd|r4|j|jrJt |jdrL|jj |j|j |dSdSdS) aLoad the TTS model. 1. Load the model config. 2. Init the model from the config. 3. Load the model weights. 4. Move the model to the GPU if CUDA is enabled. 5. Init the speaker manager in the model. Args: tts_checkpoint (str): path to the model checkpoint. tts_config_path (str): path to the model config file. use_cuda (bool): enable/disable CUDA use. use_phonemes phonemizerNz,Phonemizer is not defined in the TTS config.rDTrJr') rr3 ValueErrorrPr$r*_set_speaker_encoder_paths_from_tts_configrGr0hasattrr' init_encoderr)r9rrrr<r<r=r2s  zSynthesizer._load_ttscCs>t|jdrt|jjdr|jjj|_|jjj|_dSdSdS)zQSet the encoder paths from the tts model config for models with speaker encoders. model_argsspeaker_encoder_config_pathN)rVr3rXspeaker_encoder_model_pathrrYr)r9r<r<r=rUs  z6Synthesizer._set_speaker_encoder_paths_from_tts_config model_file model_configcCsXt||_tdddi|jj|_t|j|_|jj|j|dd|r*|jdSdS)aLoad the vocoder model. 1. Load the vocoder config. 2. Init the AudioProcessor for the vocoder. 3. Init the vocoder model from the config. 4. Move the model to the GPU if CUDA is enabled. Args: model_file (str): path to the model checkpoint. model_config (str): path to the model config file. use_cuda (bool): enable/disable CUDA use. verboseFTrSNr<) rrr r4 vocoder_apsetup_vocoder_modelr%rGr0)r9r[r\rr<r<r=r5s  zSynthesizer._load_vocodercCs |j|S)zSplit give text into sentences. Args: text (str): input text in string format. Returns: List[str]: list of sentences. )r.segment)r9textr<r<r=split_into_sentencess z Synthesizer.split_into_sentenceswavrNcCs@t|r |}t|trt|}t|||j |ddS)a Save the waveform as a file. Args: wav (List[int]): waveform as a list of values. path (str): output path to save the waveform. pipe_out (BytesIO, optional): Flag to stdout the generated TTS wav file for shell pipe. )rcrNrpipe_outN) r/ is_tensorcpunumpy isinstancelistnparrayr r )r9rcrNrdr<r<r=r s   zSynthesizer.save_wav source_wav target_wavcCs|j||}|SN)r&voice_conversion)r9rlrm output_wavr<r<r=roszSynthesizer.voice_conversionTra speaker_name language_namesplit_sentencesc  Kst} g} |s|std|r"|g} | rtd||} t| d| vr0| d|_| dd}d}|js>t|jj dr|rmt |t rm|j j dksm|j jre|jj j|ddd}t|dddf}n7|jj j|}n/t|jj jd krt|jj jd }n|s|std d}n|r|jdurtd |d d}|jst|jdr|jjdur|j j dkst|jjjd krt|jjjd }n<|rt |t rz |jjj|}Wn+ty}ztd|d|jjjd|d}~ww|stdtd|d|dur*|jj dur*t|jj dr*|jj jdur*|jj |}d}|jdu}|s Text splitted to sentences.rN name_to_idxttsF) num_samples randomizerz [!] Looks like you are using a multi-speaker model. You need to define either a `speaker_idx` or a `speaker_wav` to use a multi-speaker model.z; [!] Missing speakers.json file path for selecting speaker z\.Define path for speaker.json if it is a multi-speaker model or remove defined speaker idx. r)z8 [!] Looks like you use a multi-lingual model. Language z$ is not in the available languages: .z [!] Look like you use a multi-lingual model. You need to define either a `language_name` or a `style_wav` to use a multi-lingual model.z@ [!] Missing language_ids.json file path for selecting language zb.Define path for language_ids.json if it is a multi-lingual model or remove defined language idx. encoder_aprfr0 synthesize)rarE speaker_id voice_dirsd_vector speaker_wavr?) modelraCONFIGrr| style_wav style_textuse_griffin_limr~ language_idrcoutputs model_outputsr4rz" > interpolating tts model output.do_trim_silencei') rrr reference_wavr|r~rreference_speaker_idreference_d_vectorz > Processing time: z > Real-time factor: r<)9timerTprintrbrpoprrVr$r'rhstrr3ruse_d_vector_fileget_mean_embeddingrjrkrtlenrivaluesrr)KeyErrorkeysrzcompute_embedding_from_clipr%next parametersdevicerr{rdetachrfrgap denormalizeTr^ normalizerrr r/tensor unsqueeze inferencetoresqueezer4r get_embeddings_by_namer )r9rarqrrrrrrreference_speaker_namerskwargs start_timewavssensspeaker_embeddingr|revocoder_deviceuse_glsenrwaveformmel_postnet_spec vocoder_input scale_factorreference_speaker_embeddingr process_time audio_timer<r<r=ttssd             $ <     zSynthesizer.tts) rrrrrrrrrrrNFrn) rrrNNNNNT)__name__ __module__ __qualname__rboolr# staticmethodr-r6r7r8r2rUr5rrbintr ror __classcell__r<r<r:r=rs     V      r)"rMrtypingrrgrjrAr/r TTS.configrTTS.tts.configs.vits_configrTTS.tts.modelsrrPTTS.tts.models.vitsrTTS.tts.utils.synthesisrr r TTS.utils.audior TTS.utils.audio.numpy_transformsr TTS.vc.modelsrFTTS.vocoder.modelsr_TTS.vocoder.utils.generic_utilsr Modulerr<r<r<r=s$