Pmj NdZddlZddlZddlZddlmZmZddlmZddlm Z m Z da ej Z dZdZde d e efd Zd d d ddedededed ef dZde d efdZdedzdedzde d efdZeddededzdedzde deded eeedzffdZefded efdZdS)uRetry utilities — jittered backoff for decorrelated retries. Replaces fixed exponential backoff with jittered delays to prevent thundering-herd retry spikes when multiple sessions hit the same rate-limited provider concurrently. N)datetimetimezone)parsedate_to_datetime)AnyOptional)g>@gN@gV@^@value_or_headersreturnct|}|rt|tttfsPt |dd}t |r. |d}| |d}n#t $rYdSwxYw|}ndS|dSt|trdSt|ttfrtdt|St| }|sdS tdt|S#ttf$rYnwxYw t|}n#ttf$rYdSwxYw|dS|j |tj}td|t#jtjz S)aParse a ``Retry-After`` value into non-negative seconds. Accepts either a raw header value (numeric string / HTTP-date / number) or a headers mapping, in which case the ``Retry-After`` key is looked up case-insensitively (``.get`` on dict-like objects tries both common casings; real HTTP header containers like httpx/requests are already case-insensitive). Returns: Seconds as a ``float`` (negative deltas clamped to ``0.0``), or ``None`` when the header is absent or unparseable. Ngetz Retry-Afterz retry-afterg)tzinfo) isinstancestrintfloatgetattrcallable Exceptionboolmaxstrip TypeError ValueErrorrrreplacerutcrnow total_seconds)r rawgettervaluetextwhens 7/home/thesage/.hermes/hermes-agent/agent/retry_utils.pyparse_retry_after_secondsr%&s C z#S%/@AAeT** F    }--="F=11E   tt CC4 {t#tt#U|$$$3c ### s88>>  D t 3d $$$ z "     $T** z "tt |t {||8<|00 sTHL666EEGG H HHs6A!! A/.A/1DD"!D"&D66E  E g@rg? base_delay max_delay jitter_ratioattemptr'r(r)cnt5tdz at}dddn #1swxYwYtd|dz }|dks|dkr|}nt|d|zz|}t j|dzz dz}t j|}|d||z} || zS)aCCompute a jittered exponential backoff delay. Args: attempt: 1-based retry attempt number. base_delay: Base delay in seconds for attempt 1. max_delay: Maximum delay cap in seconds. jitter_ratio: Fraction of computed delay to use as random jitter range. 0.5 means jitter is uniform in [0, 0.5 * delay]. Returns: Delay in seconds: min(base * 2^(attempt-1), max_delay) + jitter. The jitter decorrelates concurrent retries so multiple sessions hitting the same provider don't all retry at the same instant. Nr?lyn<l) _jitter_lock_jitter_counterrmintimetime_nsrandomRandomuniform) r*r'r(r)tickexponentdelayseedrngjitters r$jittered_backoffr=Zs. 11gk""H2~~qJ!x-0)<< LNNdZ/ 0J >D -  C [[L50 1 1F 6>s &**errorc|t|ddt|ddt|ddg}dd|DS)zCBest-effort flattened provider error text for retry classification.messageNbodyresponse c38K|]}|t|VdS)N)r).0parts r$ z_error_text..s-DD$43CCII3C3C3C3CDD)rjoinlower)r>partss r$ _error_textrLsj y$''vt$$z4((  E 88DD%DDD D D J J L LLrHbase_urlmodelc|pd}|pd}t|dd}t|}|dkod|vo d|vod|vpd|vS) aLReturn True for Z.AI Coding Plan transient overload 429s. The coding-plan endpoint reports overload as HTTP 429 with body code 1305 and message "The service may be temporarily overloaded...". Treat only that narrow shape specially so ordinary quota/billing 429s still fail fast through the existing classifier. status_codeNizapi.z.ai/api/coding/paas/v4zglm-5.21305ztemporarily overloaded)rJrrL)rMrNr>base model_namestatusr"s r$is_zai_coding_overload_errorrVs N ! ! # #D+2$$&&J UM4 0 0F u  D#  A )T 1 A  # At^ ?74? rHshort_attempts default_waitrXct|||s|dfS||kr|dfSt||z dz ttdz }t|}t d||ddfS)u7Provider-aware rate-limit backoff. For most providers this returns ``default_wait`` unchanged. For Z.AI Coding Plan GLM-5.2 overloads, keep the first ``short_attempts`` retries on the normal short exponential schedule, then switch to progressively longer waits (30s → 60s → 90s → 120s, capped) plus light jitter. ``attempt`` is 1-based, matching the retry loop's logged attempt number. Returns ``(wait_seconds, reason_label)`` where ``reason_label`` is suitable for status/log decoration when a provider-specific policy fired. )rMrNr>Nzai_coding_overload_shortr,g?r&zai_coding_overload_long)rVr1len!_ZAI_CODING_OVERLOAD_LONG_BACKOFFr=)r*rMrNr>rYrXidxr's r$adaptive_rate_limit_backoffr`s( (e T T T"T!!.  888 g&*C0Q,R,RUV,V W WC237J A* Y\ ] ] ]_y yyrHc6|ttzdzS)uRetry-loop ceiling needed for the full Z.AI overload backoff schedule. The adaptive policy runs ``short_attempts`` short retries, then walks the long-backoff table one entry per subsequent attempt. The retry loop gives up as soon as ``retry_count >= ceiling`` — and that check runs *before* the attempt's backoff is computed — so the ceiling must sit one past the final long-backoff entry for every long tier to actually execute. With the default ``api_max_retries`` (3) equal to ``short_attempts`` (3), the loop always gave up before reaching the long tier, leaving the whole long-backoff schedule as dead code. Callers extend the ceiling to this value for Z.AI Coding overload 429s so the 30/60/90/120s waits run. r,)r]r^rWs r$!zai_coding_overload_retry_ceilingrbs C ABB BQ FFrH)__doc__r4 threadingr2rr email.utilsrtypingrrr0Lockr/r^#_ZAI_CODING_OVERLOAD_SHORT_ATTEMPTSrr%rr=rrLrrVtupler`rbrHr$rksL  ''''''''------ y~ %>!'(#1I1I1I1I1I1In &&& && &  &  &&&&RMsMsMMMMcDjtTW\`6>zzz zDjz : z  z  zz 5#* zzzz@=`GGcGdgGGGGGGrH