PL jadZddlmZddlZddlZddlZddlZddlmZddlm Z ddl m Z m Z m Z mZddlmZejeZdd Zdd Zddddd dZd!dZgdZdS)"uContext compression — extract the AIAgent methods that drive summarisation. Three concerns live here: * :func:`check_compression_model_feasibility` — startup probe of the configured auxiliary compression model. Warns when the aux context window can't fit the main model's compression threshold; auto-lowers the session threshold when possible; hard-rejects auxes below ``MINIMUM_CONTEXT_LENGTH``. * :func:`replay_compression_warning` — re-emit a stored warning through the gateway ``status_callback`` once it's wired up (the callback is set after :class:`AIAgent` construction). * :func:`compress_context` — the actual compression call. Runs the configured compressor, splits the SQLite session, rotates the session_id, notifies plugin context engines / memory providers, and returns the compressed message list and freshly-built system prompt. * :func:`try_shrink_image_parts_in_messages` — image-too-large recovery helper that re-encodes ``data:image/...;base64,...`` parts at a smaller size so retries can fit under provider ceilings (Anthropic's 5 MB). ``run_agent`` keeps thin wrappers for each so existing call sites (``self._compress_context(...)``) keep working. Tests that exercise these paths see no behavioural change. ) annotationsN)datetime)Path)AnyListOptionalTuple)estimate_request_tokens_roughagentrreturnNonecp|jsdS ddlm}m}ddlm}m}|d|\}} |d\}}}}}n#t$rd}YnwxYw||sI|r |dkrd |d } nd } | |_ | | t d dStt|d d} tt|dd} ||| | t|dd|r|dkr|nt|dd|j} | r'| |kr!t!d|d| dd|dd|dzd |jj} | | kr| }| }||j_|jj}|r||z |j_|rt+| |z dznd}t|ddpd}t|ddpd}|r|dkr|nd}|s. ddlm}|| jp| }n#t$r| pd}YnwxYw|r|d|dn|}|d|d}d |d!| dd"|d#|dd$|dd%|dd&|d'} | |_ | | t d(|| ||dSdS#t $rt$r&}td)|Yd}~dSd}~wwxYw)*auWarn at session start if the auxiliary compression model's context window is smaller than the main model's compression threshold. When the auxiliary model cannot fit the content that needs summarising, compression will either fail outright (the LLM call errors) or produce a severely truncated summary. Called during ``AIAgent.__init__`` so CLI users see the warning immediately (via ``_vprint``). The gateway sets ``status_callback`` *after* construction, so :func:`replay_compression_warning` re-sends the stored warning through the callback on the first ``run_conversation()`` call. Nr)_resolve_task_provider_modelget_text_auxiliary_client)MINIMUM_CONTEXT_LENGTHget_model_context_length compression) main_runtimeautou/⚠ Configured auxiliary compression provider 'u' is unavailable — context compression will drop middle turns without a summary. Check auxiliary.compression in config.yaml and reauthenticate that provider.u⚠ No auxiliary LLM provider configured — context compression will drop middle turns without a summary. Run `hermes setup` or set OPENROUTER_API_KEY.uLNo auxiliary LLM provider for compression — summaries will be unavailable.base_urlapi_key&_aux_compression_context_length_configprovider)rrconfig_context_lengthrcustom_providerszAuxiliary compression model z has a context window of ,z$ tokens, which is below the minimum zE required by Hermes Agent. Choose a compression model with at least izK context (set auxiliary.compression.model in config.yaml), or set auxiliary.compression.context_length to override the detected value if it is wrong.d2model?)urlparsez ()u⚠ Compression model z context is z tokens, but the main model z's compression threshold was z2 tokens. Auto-lowered this session's threshold to u tokens so compression can run. To make this permanent, edit config.yaml — either: 1. Use a larger compression model: auxiliary: compression: model: 2. Lower the compression threshold: compression: threshold: 0.02duAuxiliary compression model %s has %d token context, below the main model's compression threshold of %d tokens — auto-lowered session threshold to %d to keep compression working.z4Compression feasibility check failed (non-fatal): %s)compression_enabledagent.auxiliary_clientrragent.model_metadatarr_current_main_runtime Exception_compression_warning _emit_statusloggerwarningstrgetattr_custom_providers ValueErrorcontext_compressorthreshold_tokenscontext_lengththreshold_percentint urllib.parser"hostnamedebug)r rrrrclient aux_model_aux_cfg_provider_msg aux_base_url aux_api_key aux_context threshold old_threshold new_thresholdmain_ctxsafe_pct _main_model_main_provider_aux_provider_labelr" _main_label _aux_labelexcs B/home/kuhnn/.hermes/hermes-agent/agent/conversation_compression.py#check_compression_model_feasibilityrN,sB  $b                 65 4466     #,H,H,W,W ) q!Q # # # "    # >> %6&%@%@4)444D *-E &   s # # # NN1    F76:r::;; '&)R8899 .. !")%1Y[_"`"`,=AARV\A\A\''bijoq{}cAcA"4    $  ;)???2y22(H221I22*T1 222   ,=  " "&M'M8EE $ 5/>H !H,(:?GNsK(2c9:::BH "%"55<K$UJ;;ArN%):f)D)D"! ' AA555555 ..7G<('!AAA*6*@&'''A"!;33.3333   &??)<???J 8 8 8> 8 8 8 8!- 8 8 !E 8 82?N 8 8*27 8 8  *-E &   s # # # NN,     E # "X       BC          sg2I;AI; AI;AAI;/D0I; G98I;9H I; H  A-I;;J5J00J5ct|dd}|r1|jr, |d|dS#t$rYdSwxYwdSdS)uRe-send the compression warning through ``status_callback``. During ``__init__`` the gateway's ``status_callback`` is not yet wired, so ``_emit_status`` only reaches ``_vprint`` (CLI). This method is called once at the start of the first ``run_conversation()`` — by then the gateway has set the callback, so every platform (Telegram, Discord, Slack, etc.) receives the warning. r*N lifecycle)r/status_callbackr))r r>s rMreplay_compression_warningrRs %/ 6 6C u$   ! !+s 3 3 3 3 3    DD s4 AAdefault) approx_tokenstask_id focus_topicmessageslistsystem_messager.rT Optional[int]rUrV Optional[str]Tuple[list, str]c t|}td|jpd||r|dnd|j||d|jr, |j|n#t$rYnwxYw |j |||}n,#t$r|j ||}YnwxYwt|j dd }|r6t|d d |kr ||_ |d |d nlt|j d d } t|j dd } | r>| | f} t|dd | kr%| |_|d| d| pdd|j} | r|d| d|||} | |_|jr: |j|j}|||j|jd|j}t5jddt;jjd d|_|jt@j!d< ddl"m#}|$|jn#t$rYnwxYw|j%d|jdz |_&d|_'|j(|j|j)pt@j!*d d!|j|j+|"d#|_'|ru |j,|}|j-|j|n9#t\tf$r%}t/d$|Yd }~nd }~wwxYw|j0|j| d|_1n2#t$r%}t2d%|Yd }~nd }~wwxYw tg*d&}|r9ti|j d'r$|j 5|jpd(d|)n2#t$r%}t/d*|Yd }~nd }~wwxYw tg*d&}|r,|jr%|j6|jpd(|dd+n2#t$r%}t/d,|Yd }~nd }~wwxYw|j j7}|d-kr"|8|j9d.|d/d#0tu|| pd(|j;pd 1}||j _<d|j _= dd2l>m?}||n#t$rYnwxYwtd3|jpd|t||d|| fS)4uCompress conversation context and split the session in SQLite. Args: agent: The owning :class:`AIAgent`. messages: Current message history (will be summarised). system_message: Current system prompt; rebuilt after compression. approx_tokens: Pre-compression token estimate, logged for ops. task_id: Tool task scope (used for clearing file-read dedup state). focus_topic: Optional focus string for guided compression — the summariser will prioritise preserving information related to this topic. Inspired by Claude Code's ``/compact ``. Returns: ``(compressed_messages, new_system_prompt)`` tuple. zPcontext compression started: session=%s messages=%d tokens=~%s model=%s focus=%rnonerunknownuT🗜️ Compacting context — summarizing earlier conversation so I can continue...)current_tokensrV)r`_last_summary_errorN!_last_compression_summary_warningu ⚠ Compression summary failed: z%. Inserted a fallback context marker._last_aux_model_failure_model_last_aux_model_failure_error_last_aux_fallback_warning_keyu"ℹ Configured compression model 'z ' failed (z unknown erroruS). Recovered using main model — check auxiliary.compression.model in config.yaml.user)rolecontentrz %Y%m%d_%H%M%Sr=HERMES_SESSION_IDr) _SESSION_IDsession_z.jsonFHERMES_SESSION_SOURCEcli) session_idsourcer model_configparent_session_idTz,Could not propagate title on compression: %suKSession DB compression split failed — new session will NOT be indexed: %sold_session_idon_session_startr)boundary_reasonrsz1context engine on_session_start (compression): %s)rrresetreasonz2memory manager on_session_switch (compression): %su⚠️ Session compressed u> times — accuracy may degrade. Consider /new to start fresh.)force) system_prompttools)reset_file_dedupz?context compression done: session=%s messages=%d->%d tokens=~%s)@lenr,inforor r+_memory_manageron_pre_compressr)r2compress TypeErrorr/rb _emit_warningre _todo_storeformat_for_injectionappend_invalidate_system_prompt_build_system_prompt_cached_system_prompt _session_dbget_session_titlecommit_memory_session end_sessionrnowstrftimeuuiduuid4hexosenvirongateway.session_contextrksetlogs_dirsession_log_file_session_db_createdcreate_sessionplatformget_session_init_model_configget_next_title_in_lineageset_session_titler1r9update_system_prompt_last_flushed_db_idxr-localshasattrrton_session_switchcompression_count_vprint log_prefixr r{last_prompt_tokenslast_completion_tokenstools.file_toolsr|)r rWrYrTrUrV_pre_msg_count compressed summary_error_aux_fail_model _aux_fail_err_aux_key todo_snapshotnew_system_prompt old_titlersrk new_titlee_old_sid_ce_err_me_err_cc_compressed_estr|s rMcompress_contextrs0]]N KKZ "FN -<=9ek   ^     ! 1 1( ; ; ; ;    D _-66xP]kv6ww ___-66xP]6^^ _ E46KTRRM 5=t D D U U6CE 3   6=666   "%":<[]abb 8:Y[_``  '7Hu>EEQQ7?4##HHH%8HHH %::<BB"3E %m$ m);;EBJ* + ?????? 01111    &+^6XAQ6X6X6X%XE "(-E %   , , +~W8OQV)W)Wk"="0 -   )-E % TT % 1 K KI V VI%778H)TTTT"I.TTTLL!OQRSSSSSSSST   2 253CEV W W W)*E & & m m m NNhjk l l l l l l l l m S88<< 011   8:LMM   $ 5 5 &B -' 6    SSS H'RRRRRRRRS T88<< 011  -   ! 3 3 &B"*$ 4    TTT I7SSSSSSSST  " 4C axx  C CC C C C    4'-2k!TO 3BE/67E3  555555!!!!       KKI "FNC OO  ( ((sA99 BB B((&CC:B=P8 KP K&#P%K&&A>P%:N P O1O PO*P P0 P++P04AR SR;;SAT UT>>U.W W  W  api_messagesboolc |sdS ddlm n3#t$r&}td|Yd}~dSd}~wwxYwd d}d fd }|D]}t |t s|d }t |tsC|D]}t |t s|d }|dvr2|d}t |t r.|dd} || } | r | |d<|dz }t |tr||} | r | |d<|dz }|rt d| dz |dkS)uwRe-encode all native image parts at a smaller size to recover from image-too-large errors (Anthropic 5 MB, unknown other providers). Mutates ``api_messages`` in place. Returns True if any image part was actually replaced, False if there were no image parts to shrink or Pillow couldn't help (caller should surface the original error). Strategy: look for ``image_url`` / ``input_image`` parts carrying a ``data:image/...;base64,...`` payload. For each one whose encoded size exceeds 4 MB (a safe target that slides under Anthropic's 5 MB ceiling with header overhead), write the base64 to a tempfile, call ``vision_tools._resize_image_for_vision`` to produce a smaller data URL, and substitute it in place. Non-data-URL images (http/https URLs) are not touched — the provider fetches those itself and the size limit is different. Fr)_resize_image_for_visionu6image-shrink recovery: vision_tools unavailable — %sNi@urlr.r r[clt|tr|dsdSt| krdS |d\}}}d}|drZ|tddddd}|dr|}ddl}||}d d d d d d d |d }tj d|d} | ||  t| j| } t| jdnN#t"$rYnBwxYw# t| jdw#t"$rYwwxYwxYw| r t| t|krdS| S#t"$r&} t$d| Yd} ~ dSd} ~ wwxYw)z8Return a smaller data URL, or None if shrink can't help.zdata:Nr image/jpeg;rzimage/z.pngz.gifz.webpz.jpgz.bmp)z image/pngz image/gifz image/webprz image/jpgz image/bmphermes_shrink_F)prefixsuffixdelete) mime_typemax_base64_bytesT) missing_oku.image-shrink recovery: re-encode failed — %s) isinstancer. startswithr} partitionsplitstripbase64 b64decodertempfileNamedTemporaryFilewriteclosernameunlinkr)r,r-)rheaderr=datamime mime_part_b64rawrtmpresizedrLr target_bytess rM_shrink_data_urlz._shrink_data_urlsy#s## 3>>'+B+B 4 s88| # #4# !mmC00OFAtD  )) %"3w<<==177Q??BHHJJ ''11%$D ! ! ! !..&&C#&$6c$ -'uC  # 22NN"%1 NN))T):::: DNN))T):::: D c'llc#hh66tN    NNKS Q Q Q44444 szCHA F&(FH FHFHG!(G  G GGGG%HH H3 H..H3rhtype> image_url input_imagerrrzGimage-shrink recovery: re-encoded %d image part(s) to fit under %.0f MBi)rr.r r[) tools.vision_toolsrr)r,r-rdictrrXr.r~) rrL changed_countrr>rhpartptype image_valuerrrrs @@rM"try_shrink_image_parts_in_messagesrs$ u??????? OQTUUUuuuuu#LM*******X''#t$$  '')$$'4((   ' 'DdD)) HHV$$E888((;//K+t,, '!ooeR00**3//')0K&!Q&MK-- '**;77'(/D%!Q&M' '*  U <;7    1 s ?:?)rNrRrr)r rr r )r rrWrXrYr.rTrZrUr.rVr[r r\)rrXr r)__doc__ __future__rloggingrrrrpathlibrtypingrrrr r'r getLogger__name__r,rNrRrr__all__rMrs>8#"""""  ------------>>>>>>  8 $ $r r r r j.$(!%@)@)@)@)@)@)Fnnnnb   r