PL j%(dZddlmZddlZddlZddlmZmZmZddl m Z ej e Z dd Zdd Z dddZGdde ZdS)uTavily web search + content extraction + crawl — plugin form. Subclasses :class:`agent.web_search_provider.WebSearchProvider`. Three capabilities advertised: - ``supports_search()`` -> True (Tavily ``/search``) - ``supports_extract()`` -> True (Tavily ``/extract``) - ``supports_crawl()`` -> True (Tavily ``/crawl``) — sync HTTP crawl; Firecrawl also advertises ``supports_crawl=True`` (async) All three are sync — the underlying call is ``httpx.post(...)``. The dispatcher in :func:`tools.web_tools.web_crawl_tool` (which is itself async) will run sync providers in a thread when appropriate. Config keys this provider responds to:: web: search_backend: "tavily" # explicit per-capability extract_backend: "tavily" # explicit per-capability crawl_backend: "tavily" # explicit per-capability backend: "tavily" # shared fallback for all three Env vars:: TAVILY_API_KEY=... # https://app.tavily.com/home (required) TAVILY_BASE_URL=... # optional override of https://api.tavily.com Auth note: Tavily uses ``api_key`` in the JSON body for /search and /extract, but **also requires** ``Authorization: Bearer `` for /crawl (body-only auth returns 401 on /crawl). The plugin handles both. ) annotationsN)AnyDictList)WebSearchProviderendpointstrpayloadDict[str, Any]returncddl}tjd}|stdtjdd}t |}||d<|d|d}t d |||dd krd d |ini}| |||d }| | S)zPOST to the Tavily API and return the parsed JSON response. Mirrors :func:`tools.web_tools._tavily_request`. Raises ``ValueError`` when ``TAVILY_API_KEY`` is unset; the caller catches and surfaces as a typed error response. rNTAVILY_API_KEYz\TAVILY_API_KEY environment variable not set. Get your API key at https://app.tavily.com/homeTAVILY_BASE_URLzhttps://api.tavily.comapi_key/zTavily %s request to %scrawl AuthorizationzBearer <)jsonheaderstimeout) httpxosgetenv ValueErrordictlstriploggerinfostrippostraise_for_statusr)rr rrbase_urlurlrresponses ?/home/kuhnn/.hermes/hermes-agent/plugins/web/tavily/provider.py_tavily_requestr',sLLLi())G   >   y*,DEEH7mmG GI  . .,, . .C KK)8S9999As8K8Kw8V8V 3' 3 344\^Gzz#GWbzIIH  ==??r%c g}t|dgD]^\}}||dd|dd|dd|dzd_dd |id S) zEMap Tavily ``/search`` response to ``{success, data: {web: [...]}}``.resultstitler$content)r+r$ descriptionpositionTweb)successdata) enumerategetappend)r% web_resultsiresults r& _normalize_tavily_search_resultsr:KsKx||Ir::;;  6GR00zz%,,%zz)R88E       e[%9 : ::r(r, fallback_urlList[Dict[str, Any]]c g}|dgD]}|d|}|ddp|dd}|||dd||||dddd|d gD]\}||d|ddd|d d d |d|id ]|dgD]F}t|tr|nt|}||dddd d |id G|S)aBMap Tavily ``/extract`` or ``/crawl`` response to standard documents. Documents follow the legacy LLM post-processing shape:: {"url", "title", "content", "raw_content", "metadata"} Failures (``failed_results``, ``failed_urls``) become result entries with an ``error`` field rather than raising. r*r$ raw_contentr,r-r+) sourceURLr+)r$r+r-r>metadatafailed_resultserrorzextraction failedr?)r$r+r-r>rBr@ failed_urls)r5r6 isinstancer ) r%r; documentsr9r$rawfailfail_urlurl_strs r&_normalize_tavily_documentsrJZs')I,,y"--   jj --jj++Hvzz)R/H/HGR00"*- 7B8O8OPP        -r22   xx|44!'+>??($((5,*G*GH    LL33   (377J((S]]!,('2     r(ceZdZdZeddZeddZddZddZdd Z dd Z dddZ ddZ d dZ d!dZdS)"TavilyWebSearchProviderz)Tavily search + extract + crawl provider.r r cdS)Ntavilyselfs r&namezTavilyWebSearchProvider.namexr(cdS)NTavilyrOrPs r& display_namez$TavilyWebSearchProvider.display_namerSr(boolcjttjddS)z@Return True when ``TAVILY_API_KEY`` is set to a non-empty value.rr,)rWrrr rPs r& is_availablez$TavilyWebSearchProvider.is_availables)BI.3399;;<<>+0&+ C4C88 8 9 9 9$s3xx88 8 8 8 8 8 8 O O O NN4c : : :$/M/M/MNN N N N N N N Os5A%A A%% C/BC C"B;5C;Curls List[str]kwargsrr<c  ddlm}|r d|DStdt |t d|dd}t ||r|dnd S#t$rfd |DcYd Sd wt$r3t d fd |DcYd Sd wwxYw)uExtract content from one or more URLs via Tavily. Sync — the underlying call is httpx.post(...). Returns the legacy list-of-results shape; per-URL failures become items with ``error``. rrgcg|]}|ddd S)rir,)r$rBr+rO).0us r& z3TavilyWebSearchProvider.extract..s/HIA CCr(zTavily extract: %d URL(s)extractF)rtrnr,r;c8g|]}|ddtdS)r,r$r+r-rB)r ryrzrss r&r{z3TavilyWebSearchProvider.extract..s,\\\RSArCHHMM\\\r(NzTavily extract error: %sc$g|] }|dddd S)r,zTavily extract failed: rrOrs r&r{z3TavilyWebSearchProvider.extract..s?B2@_Z]@_@_``r() rorhrrlenr'rJrrqrr)rQrtrvrhrFrss @r&r|zTavilyWebSearchProvider.extractsr   6 6 6 6 6 6~ MQ KK3SYY ? ? ?! &+C/T"9$q''r  ] ] ]\\\\W[\\\ \ \ \ \ \ \    NN5s ; ; ;       s5A5AA55 C? B C C(C C Cr$c $ ddlm}|r d|ddddgiS|d}|dd }|d d }td ||||||d }|r||d<t d|}dt ||iS#t$r!} d|ddt| dgicYd} ~ Sd} ~ wt$r2} t d| d|ddd| dgicYd} ~ Sd} ~ wwxYw)uCrawl a seed URL via Tavily's ``/crawl`` endpoint. Accepted kwargs (others ignored for forward compat): - ``instructions``: str — natural-language guidance for the crawl - ``depth``: str — ``"basic"`` (default) or ``"advanced"`` - ``limit``: int — max pages to crawl (default 20) Returns ``{"results": [...]}`` shaped to match what :func:`tools.web_tools.web_crawl_tool` post-processes. rrgr*r,rir instructionsdepthbasicrdrkz%Tavily crawl: %s (depth=%s, limit=%d))r$rd extract_depthrr}NzTavily crawl error: %szTavily crawl failed: ) rorhr5rrr'rJrr rqrr) rQr$rvrhrrrdr rFrss r&rzTavilyWebSearchProvider.crawls$  6 6 6 6 6 6~ g!C"Vc$d$d#eff!::n55LJJw00EJJw++E KK?eU S S S!&''G  7*6'!'733C6sMMM  ^ ^ ^bRRUVYRZRZ [ [\] ] ] ] ] ] ]    NN3S 9 9 9"!##%!>!>!>         s5B*B B** D4C D D'D D DcdddddddgdS) NrUpaidz)Search + extract + crawl in one provider.rzTavily API keyzhttps://app.tavily.com/home)keypromptr$)rRbadgetagenv_varsrOrPs r&get_setup_schemaz(TavilyWebSearchProvider.get_setup_schemas4>,.8   r(N)r r )r rW)rb)rcr rdrer r )rtrurvrr r<)r$r rvrr r )r r )__name__ __module__ __qualname____doc__propertyrRrVrYr\r_rarjr|rrrOr(r&rLrLs33 XX====OOOOO2    D////b       r(rL)rr r r r r )r%r r r )r,)r%r r;r r r<)r __future__rloggingrtypingrrragent.web_search_providerr getLoggerrrr'r:rJrLrOr(r&rs@#""""" """"""""""777777  8 $ $> ; ; ; ; 3500000fP P P P P /P P P P P r(