Pmj9dZddlZddlmZddlmZmZmZedGddZd%d e d e d e d dfdZ d e d e d e fdZ de d ee e ffdZdddde d e de dzd e de de d efdZdeee efd eee effdZdeee efdzd eee effdZdeee efdeee efd efdZdeee efd e d eefdZdd dd d deee efdeee efdzd!e d e de dzd"e d efd#Z d&deee efd!e d e de dzd eee eff d$ZdS)'aAnthropic prompt caching strategy. The default layout uses 4 cache_control breakpoints: the static system prefix, the end of the system prompt, and the last 2 non-system messages. When a static system prefix is unavailable, it falls back to one system breakpoint plus the last 3 messages. All markers use the same TTL (5m or 1h). This preserves intra-session caching while allowing new sessions to reuse the stable system-prompt prefix. Pure functions -- no class state, no AIAgent dependency. N) dataclass)AnyDictListT)frozenc|eZdZUdZeeeefed<eeeefed<e de fdZ dS)PromptCachePlanzARequest-local message and tool sections with their cache markers.messagestoolsreturnc6t|j|jS)zWire-visible cache markers in this plan (computed on demand). Only tests consume this; keeping it lazy avoids walking every message part and tool schema on the per-request hot path. )_count_cache_markersr r )selfs :/home/thesage/.hermes/hermes-agent/agent/prompt_caching.py marker_countzPromptCachePlan.marker_counts$DM4:>>>N) __name__ __module__ __qualname____doc__rrstrr__annotations__propertyintrrrr r ssKK4S>"""" S#X  ?c???X???rr Fmsg cache_markernative_anthropicr c|dd}|d}|dkr |r||d<dS||dkr|dkr|sdS|dkr|sdS||d<dSt|tr d||d g|d<dSt|tr&|r&|d }t|tr ||d<dSdSdSdS) zFAdd cache_control to a single message, handling all format variations.rolecontenttool cache_controlN assistanttexttyper&r$)get isinstancerlistdict)rrrr r"lasts r_apply_cache_markerr/#s! 7762  Dggi  G v~~*~ ,O'R-- 6>>"2> F ;  '7  F+O'3W| L L I '4  1W1r{ dD ! ! 1$0D ! ! !1111 1 1rc|rdS|d}||dkrdSt|tr*t|ot|dtSt|t S)uTrue if a marker on this message is actually honored by the provider. On the native Anthropic layout every message works (top-level markers are relocated by the adapter). On the envelope layout (OpenRouter et al.) only markers inside content parts are honored: empty-content messages (e.g. assistant turns that are pure tool_calls) and empty tool messages would receive a top-level marker the provider ignores — wasting one of the four breakpoints. Skip those so the breakpoints land on messages that count. Tr"Nr!Fr))r*r+r,boolr-r)rrr"s r_can_carry_markerr2Hswtggi  G'R--u'4  ? G}}>GBK!>!>> gs # ##rttlc$ddi}|dkrd|d<|S)zCBuild a cache_control marker dict for the given TTL ('5m' or '1h').r( ephemeral1hr3r)r3markers r _build_markerr8`s#$k2F d{{u Mr) mark_suffixfallback_to_wholemessagestatic_system_prefixr9r:cr|d}t|trv|rtt|tr_||rJ|t |d}|rd|d}|r||d<d||d|g|d<|rdndSt ||| dS|sd St ||| dS) uMark the static system prefix (and optionally the full prompt). The system prompt remains one stored string. Splitting it only in the outgoing request keeps session persistence and non-Anthropic transports unchanged while making the stable prefix independently cacheable. ``mark_suffix=False`` is the tool-cache-plan layout: only the static prefix carries a marker, the volatile suffix rides unmarked (its breakpoint budget is spent on the tools array instead). ``fallback_to_whole=False`` skips marking entirely when the prefix split is not possible (no prefix, mismatched prefix, non-string content) instead of marking the whole message. When the prompt IS exactly the static prefix (empty suffix), the whole message is marked as a single block — never a two-part split with an empty text block, which Anthropic rejects. Returns the number of markers applied (0, 1, or 2). r"Nr&)r(r&r$r'rr)r*r+r startswithlenr/) r;rr<rr9r:r"suffix suffix_parts r_apply_system_cache_markersrEhs:kk)$$G'--  w $ $   3 4 4  122334  +)/ @ @K t|dkp+|dd kot|d k}|r"d d |D|d<|S) uLRemove ``cache_control`` markers and undo decoration-produced list shapes. Used before re-applying decoration after a mid-turn provider failover so the mutated, undecorated shape (image shrink / ASCII cleanup / etc.) is preserved while markers match the *new* provider's cache policy (#72626). Flattening back to a plain string is restricted to the exact shapes :func:`apply_anthropic_cache_control` produces from string content — a single ``{"type": "text"}`` part, or the two-part ``[static, volatile]`` system split — so the ``""``-join is provably byte-exact. Organic multi-part text (merged user turns, imported transcripts) and parts carrying extra keys (``citations`` etc.) keep their structure; only per-part markers are removed. Marker removal is copy-on-write on the part dicts: content parts may alias the persistent conversation history (the per-call copy is shallow), and stripping must never rewrite the stored transcript. Mutates the top-level message dicts of ``api_messages`` in place and returns the same list. r$Nr"c3HK|]}t|tod|vVdS)r$Nr+r-.0parts r z0strip_anthropic_cache_control..s6VVdz$%%A/T*AVVVVVVrcg|];}t|tr"d|vrd|Dn|z..s(GGG$!Q!2F2FA2F2F2Fr)r+r-itemsrJs r z1strip_anthropic_cache_control..sfdD)).=.E.EHG$**,,GGGGrc3K|]}t|toh|dddkoNt|dto&t |ddhkVdS)r(r&N)r+r-r*rsetkeysrJs rrMz0strip_anthropic_cache_control..s+ +  tT " " 5((F2 5488F++S11 5DIIKK  VV$44 + + + + + + rr?r systemr>r!c3&K|] }|dV dS)r&NrrJs rrMz0strip_anthropic_cache_control..s&$F$FdT&\$F$F$F$F$F$Fr) r+r-popr*r,anyallrBjoin)rFrr"decoration_shapes rstrip_anthropic_cache_controlr_sX.GG#t$$   &&&'')$$'4((   VVgVVV V V %$ G %C N"  s+ +  + + + ( (   LLA  C8+AG 0A   GWW$F$Fg$F$F$FFFC N rr ctj|pg}|D]-}t|tr|dd.|S)zBReturn copied tools without request-local Anthropic cache markers.r$N)copydeepcopyr+r-rZ)r cleanedr#s r"strip_anthropic_tool_cache_controlrdsPmEKR((G,, dD ! ! , HH_d + + + Nrr ctd|D}|td|Dz }|td|DzS)z=Count the wire-visible cache markers in a request-local plan.c3LK|]}t|trd|vdV dSr$r?NrI)rKr;s rrMz'_count_cache_markers..sM  gt $ $*9G)C)C )C)C)C)Crc3K|]g}t|trPt|dtr(|dD]}t|trd|vdV hdS)r"r$r?N)r+r-r*r,)rKr;rLs rrMz'_count_cache_markers..s  gt $ $*4GKK 4J4JD)Q)QI&   dD ! !  '6&=&= '>&=&=&=&= rc3LK|]}t|trd|vdV dSrgrI)rKr#s rrMz'_count_cache_markers..sGz$55:IT:Q:Q:Q:Q:Q:Qr)sum)r r counts rrrs   E  SE 3 rcg}d}|t|kr5||}t|tr|ddkr|dz }P|ddkr|dr|dz}|}|t|krO||}t|tr|ddkrn|dz }|t|kO||kr0|dz }t |||r|||}!|ddkrd|t|krO||}t|tr|ddkrn|dz }|t|kO|ddkr|dzt|kr|dz }|ddkr|d d vr|dz } t ||r|||dz }|t|k5|S) z