o j5@sUddlmZddlmZmZddlmZddlmZddl m Z m Z ddl m Z ddlmZmZmZeeZidid id id id id idididididididididididiiidZeeeeeffed<edeeefdeeeffddZedeedeefddZdeeefeeBdeeefeeBfd dZGd!d"d"ZGd#d$d$eZGd%d&d&eZGd'd(d(eZed)d*d+eded,fd-d.Zd/ed0edefd1d2Z d)S)3) lru_cache)Anyoverload) constants)InferenceProviderMapping) MimeBytesRequestParameters)ChatCompletionInputMessage)build_hf_headers get_tokenloggingcerebrascohereclarifai deepinfrazfal-aiz fireworks-aigroqz hf-inference hyperbolicnebiusnscalenvidiaovhcloud replicate sambanovascalewaytogether) wavespeedzzai-org!HARDCODED_MODEL_INFERENCE_MAPPINGobjreturncCdSNrr!r!_/home/kuhnn/.local/lib/python3.10/site-packages/huggingface_hub/inference/_providers/_common.py filter_none/r$cCrr r!r"r!r!r#r$1r%cCsvt|tr&i}|D]\}}|durq t|ttfrt|}|||<q |St|tr2dd|DStdt|)NcSs&g|]}t|ttfrt|n|qSr!) isinstancedictlistr$).0vr!r!r# As&zfilter_none..zExpected dict or list, got )r&r'itemsr(r$ ValueErrortype)rcleanedkr*r!r!r#r$5s   c@seZdZdZdedededdfddZdd d ed eeefd ed edBdedBdeeefdBdefddZ d,de eBdedBdefddZ dedBdefddZ d edBde fddZd eeefdeeefdBdedBdeeeffddZd ededeeeffddZded edefd!d"Zdedefd#d$Zd ededefd%d&Zd ed ed'e dedBfd(d)Zd ed ed'e dedBdedBf d*d+ZdS)-TaskProviderHelperz.Base class for task-specific provider helpers.providerbase_urltaskrNcCs||_||_||_dSr )r2r4r3)selfr2r3r4r!r!r#__init__Is zTaskProviderHelper.__init__) extra_payloadinputs parametersheadersmodelapi_keyr7c Cs||}||}|||}|||j}|j|||d} | dur,t| t|p)i} |||||} | dur@| dur@t d| durL| durLt d| || | } t ||j |j| | | dS)z Prepare the request to be sent to the provider. Each step (api_key, model, headers, url, payload) can be customized in subclasses. )provider_mapping_infoNz8Both payload and data cannot be set in the same request.z2Either payload or data must be set in the request.)urlr4r;jsondatar:) _prepare_api_key_prepare_mapping_info_prepare_headers _prepare_url provider_id_prepare_payload_as_dictrecursive_merger$_prepare_payload_as_bytesr-_normalize_headersrr4) r5r8r9r:r;r<r7r=r>payloadr@normalized_headersr!r!r#prepare_requestNs*   z"TaskProviderHelper.prepare_requestresponserequest_paramscCs|S)z Return the response in the expected format. Override this method in subclasses for customized response handling.r!)r5rMrNr!r!r# get_responses zTaskProviderHelper.get_responsecCs,|durt}|durtd|jd|S)zZReturn the API key to use for the request. Usually not overwritten in subclasses.Nz)You must provide an api_key to work with z$ API or log in with `hf auth login`.)r r-r2r5r<r!r!r#rAs z#TaskProviderHelper._prepare_api_keyc Cs|dur td|jdt|ji|rt|j|Sd}t|D] }|j|jkr0|}nq$|durAtd|d|jd|j|jkr[td|d|jd|jd|jd |jd krmtd|d |jd |jd krtd |d|jd|S)zbReturn the mapped model ID to use for the request. Usually not overwritten in subclasses.Nz+Please provide an HF model ID supported by .zModel z is not supported by provider z is not supported for task z and provider z. Supported task: stagingz! is in staging mode for provider z. Meant for test purposes only.errorz,Our latest automated health check on model 'z' for provider 'z<' did not complete successfully. Inference call might fail.) r-r2rget!_fetch_inference_provider_mappingr4statusloggerwarning)r5r;provider_mappingmappingr!r!r#rBs6     z(TaskProviderHelper._prepare_mapping_inforJr@cCsTdd|D}|ddur(|dur |jdur |j|d<|S|dur(d|d<|S)zzNormalize the headers to use for the request. Override this method in subclasses for customized headers. cSs"i|] \}}|dur||qSr )lowerr)keyvaluer!r!r# s"z9TaskProviderHelper._normalize_headers..z content-typeNzapplication/json)r,rT mime_type)r5r:rJr@rKr!r!r#rIs z%TaskProviderHelper._normalize_headerscCsit|d|S)zwReturn the headers to use for the request. Override this method in subclasses for customized headers. )token)r )r5r:r<r!r!r#rCsz#TaskProviderHelper._prepare_headers mapped_modelcCs0||}|||}|dd|dS)zVReturn the URL to use for the request. Usually not overwritten in subclasses./)_prepare_base_url_prepare_routerstriplstrip)r5r<rbr3router!r!r#rDs  zTaskProviderHelper._prepare_urlcCsH|drtd|jdtjj|jdStd|jd|jS)[Return the base URL to use for the request. Usually not overwritten in subclasses.hf_z Calling 'z'' provider through Hugging Face router.)r2z' provider directly.) startswithrWinfor2rINFERENCE_PROXY_TEMPLATEformatr3rPr!r!r#rds z$TaskProviderHelper._prepare_base_urlcCdS)ztReturn the route to use for the request. Override this method in subclasses for customized routes. r!r5rbr<r!r!r#resz!TaskProviderHelper._prepare_router=cCro)zReturn the payload to use for the request, as a dict. Override this method in subclasses for customized payloads. Only one of `_prepare_payload_as_dict` and `_prepare_payload_as_bytes` should return a value. Nr!r5r8r9r=r!r!r#rFsz+TaskProviderHelper._prepare_payload_as_dictcCro)zReturn the body to use for the request, as bytes. Override this method in subclasses for customized body data. Only one of `_prepare_payload_as_dict` and `_prepare_payload_as_bytes` should return a value. Nr!)r5r8r9r=r7r!r!r#rHs z,TaskProviderHelper._prepare_payload_as_bytesr )__name__ __module__ __qualname____doc__strr6rr'rrLbytesrOrArrBrrIrCrDrdrerFrHr!r!r!r#r1Fs|   8   %      r1c sfeZdZdZdedeffdd Zdededefd d Zd eee Bd ed e dedBfddZ Z S)BaseConversationalTaskz Base class for conversational (chat completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/chat r2r3ctj||dddS)Nconversationalr2r3r4superr6r5r2r3 __class__r!r#r6zBaseConversationalTask.__init__rbr<rcCro)Nz/v1/chat/completionsr!rqr!r!r#rer%z%BaseConversationalTask._prepare_router8r9r=NcCtd|i|d|jiS)Nmessagesr;r$rErrr!r!r#rFsz/BaseConversationalTask._prepare_payload_as_dict) rsrtrurvrwr6rer(r'r rrF __classcell__r!r!rr#ry s rycsHeZdZdZfddZdedefddZded Bdefd d ZZ S) AutoRouterConversationalTaska Auto-router for conversational tasks. We let the Hugging Face router select the best provider for the model, based on availability and user preferences. This is a special case since the selection is done server-side (avoid 1 API call to fetch provider mapping). cstjddddS)Nautozhttps://router.huggingface.co)r2r3r})r5rr!r#r6&sz%AutoRouterConversationalTask.__init__r<rcCs|ds td|jS)rirjz>Cannot select auto-router when using non-Hugging Face API key.)rkr-r3rPr!r!r#rd)s z.AutoRouterConversationalTask._prepare_base_urlr;NcCs"|durtdtd||dddS)z In auto-router, we don't need to fetch provider mapping info. We just return a dummy mapping info with provider_id set to the HF model ID. NzPlease provide an HF model ID.rliver{)r2 hf_model_id providerIdrVr4)r-r)r5r;r!r!r#rB3sz2AutoRouterConversationalTask._prepare_mapping_info) rsrtrurvr6rwrdrrBrr!r!rr#rs   rc s^eZdZdZdedeffdd Zdededefd d Zd ed ed e dedBfddZ Z S)BaseTextGenerationTaskz Base class for text-generation (completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/completions r2r3crz)Nztext-generationr|r}rrr!r#r6JrzBaseTextGenerationTask.__init__rbr<rcCro)Nz/v1/completionsr!rqr!r!r#reMr%z%BaseTextGenerationTask._prepare_router8r9r=NcCr)Npromptr;rrrr!r!r#rFPsz/BaseTextGenerationTask._prepare_payload_as_dict) rsrtrurvrwr6rerr'rrFrr!r!rr#rDsrN)maxsizer;rcCs>ddlm}|j|dgd}|j}|durtd||S)z; Fetch provider mappings for a model from the Hub. r)HfApiinferenceProviderMapping)expandNz$No provider mapping found for model )huggingface_hub.hf_apir model_infoinference_provider_mappingr-)r;rrlrYr!r!r#rUVs rUdict1dict2csifdd|DS)NcsDi|]\}}||vrt|trt|trt||n|qSr!)r&r'rGr\rr!r#r_gs z#recursive_merge..)r,)rrr!rr#rGds rG)! functoolsrtypingrrhuggingface_hubrrr!huggingface_hub.inference._commonrr:huggingface_hub.inference._generated.types.chat_completionr huggingface_hub.utilsr r r get_loggerrsrWrr'rw__annotations__r$r(r1ryrrrUrGr!r!r!r#sv          $2D&