epj\ @UdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlmZddlmZddlmZddlmZddlmZddlmZdd lmZmZmZdd lmZdd l m!Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*dd l+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAmBZBmCZCmDZDddlEmFZFddlGmHZHddlImJZJ ddlKZKn #eL$rdZKYnwxYwejMeNZOe jPdZQdeRdeRfdZSdeRdeTfdZUde$de$fdZVde&eRe$fde(eRfdZWdsdeRdeRfdZXdeRde)eRe'eRffdZYd eRde)eRe'eRffd!ZZd"e'eRde'eRfd#Z[d"e'eRde'eRfd$Z\e*d%Z]ed&z Z^e^Z_defd'Z`d(Zadabe(eRecd)<e jdZeefZgefeRecd*<e jdZhefZiefeRecd+<e jdZjd,e(eRddfd-Zkde(eRfd.Zld/Zmd,e&eRe$fdeTfd0Znd1e'e&eRe$fde'e&eRe$ffd2Zodtd4eRdeRfd5Zpd6e jqde(eRfd7Zrd6e jqddfd8Zsd6e jqddfd9Zt dud:e(eudeTfd;Z!deRfd<ZvdeRfd=ZwGd>d?e jxZyd&d@dAd6e jqdBeRdCeTdeRfdDZzd@dEd6e jqdBeRdFeTdeRfdGZ{dBeRdHeTddfdIZ|dBeRdJe}ddfdKZ~d&dLd6e jqdBeRddfdMZdNZefZefeRecdO<e jdZdJedeTfdPZdQZdJeeRzdzdeTfdRZdSedeTfdTZdSede(efdUZd&dLdSedBeRddfdVZdSede(eRfdWZdXdYdSedZeTde&eRe$ffd[Zd\Zd]Zdefd^ZdeTfd_Zd6e jqdeTfd`ZGdadbeZGdcddeZGdedfeZdudgZdhd@didjedkedleTdeTfdmZdjede(efdnZGdodpeJeHeFZGdqdrZdS)va} SQLite State Store for Hermes Agent. Provides persistent session storage with FTS5 full-text search, replacing the per-session JSONL file approach. Stores session metadata, full message history, and model configuration for CLI and gateway sessions. Key design decisions: - WAL mode for concurrent readers + one writer (gateway multi-platform) - FTS5 virtual table for fast text search across all session messages - Compression-triggered session splitting via parent_session_id chains - Batch runner and RL trajectories are NOT stored here (separate systems) - Session source tagging ('cli', 'telegram', 'discord', etc.) for filtering N)deque)contextmanager)Path)sanitize_contextActivityProvenance)_sanitize_surrogates)SKILL_EXCERPT_JOINTSKILL_SCAFFOLD_SQL_LIKEdescribe_skill_invocation)get_hermes_home)is_sqlite_wal_reset_vulnerable)AnyCallableDictListOptionalTupleTypeVar)_BRANCH_CHILD_SQL_COMPRESSION_CHILD_SQL_FTS_CJK_TRIGGERS _FTS_TRIGGERS_LISTABLE_CHILD_SQL_PREVIEW_RAW_SELECT_ephemeral_child_sql_shape_preview_sql_session_last_active_sql_session_last_active_by_idDEFERRED_INDEX_SQLFTS_CJK_STALE_KEYFTS_SQLFTS_STORAGE_VERSIONFTS_TRIGRAM_SQLLEGACY_FTS_SQLLEGACY_FTS_TRIGRAM_SQLMAX_FTS5_QUERY_CHARS SCHEMA_SQLSCHEMA_VERSION_PREVIEW_CONTENT_SQL_PREVIEW_HEAD_CHARS_PREVIEW_MAX_CHARS_PREVIEW_SCAFFOLD_WINDOW_PREVIEW_SCAFFOLDED_SQL)SessionPortabilityMixin)SessionSchemaMixin)SessionSearchMixinz(?:^|:)pid=(\d+)(?::|$) system_promptreturncttj|dS)Nzutf-8)hashlibsha256encode hexdigest)r2s 2/home/thesage/.hermes/hermes-agent/hermes_state.py_system_prompt_hashr:Ys, >-..w77 8 8 B B D DDholderct|pd}|dS t|d}n#tt f$rYdSwxYw|dkrdS|t jkrdSt' tj | S#t$rYdSwxYwt j dkrdS t j |dn*#t$rYdSttt f$rYdSwxYwdS)aReturn True only when a structured lock holder's local PID is gone. Compression locks are stored in a host-local SQLite database and holder IDs created by ``conversation_compression`` start with ``pid=``. A process killed during gateway shutdown cannot release its lease, so waiting for the full TTL makes every new turn repeatedly attempt compaction. Reclaim only when the kernel proves that PID no longer exists; legacy/unstructured holders, same-process holders, permission errors, and any probe doubt remain protected until normal TTL expiry (conservative: PID reuse must never steal a live lease, and a wrongly-kept lease self-heals via TTL). NFrntT)_COMPRESSION_LOCK_HOLDER_PID_REsearchintgroup TypeError ValueErrorosgetpidpsutil pid_exists ExceptionnamekillProcessLookupErrorPermissionErrorOSError OverflowError)r<matchpids r9(_compression_lock_holder_process_is_deadrT]sB , 2 26Replace lone surrogates when *value* is text; pass anything else through. sqlite3 encodes bound ``str`` parameters as UTF-8 and raises ``UnicodeEncodeError`` on lone surrogates (U+D800..U+DFFF), so a single such code point anywhere in a message aborts the whole write. No-op for well-formed text. ) isinstancestrr rUs r9_scrub_surrogatesrZs'+5UC*@*@ K  & & &eKr;rowc|dpd}|r|S|dpd}|pdS)uvA session's workspace grouping key: its git repo root when known, else its cwd. Branch is deliberately excluded so checking out a new branch doesn't fragment a workspace's session history. Returns None for cwd-less (unbound) sessions. Both fields are already recorded on ``sessions`` — this just picks the coarser identity for grouping/filtering. git_repo_rootr>cwdN)getstrip)r[rootr^s r9 workspace_keyrbs] GGO $ $ * 1 1 3 3D  775>> R & & ( (C ;$r; model_configcolcd|dS)Nzjson_extract(COALESCE(z, '{}'), '$._delegate_from'))rds r9_delegate_from_jsonrgs GC G G GGr; cwd_prefixcJ|dp|}d||d|dgfS)N/\z+(s.cwd = ? OR s.cwd LIKE ? OR s.cwd LIKE ?)/%\%)rstrip)rhprefixs r9_cwd_prefix_clauseros<   u % % 3F 86f===U[R`R`R`:a aar;keycj|dp|}t|\}}d|d|g|fS)uMatch sessions whose ``workspace_key(row)`` equals ``key``. Mirrors :func:`workspace_key`: a session belongs to workspace ``key`` when its recorded ``git_repo_root`` equals ``key``, or — for rows that predate per-session git metadata — when its ``cwd`` is at or under ``key`` (so a session started in ``repo/src`` still groups with ``repo``). Used by ``hermes -c``/``--resume`` to continue the most recent session in the *current* workspace rather than the global MRU. rjz@(s.git_repo_root = ? OR (COALESCE(s.git_repo_root, '') = '' AND )))rmro)rprn cwd_clause cwd_paramss r9_workspace_key_clauserusOZZ   %#F/77J Y:YYY * r; parent_idsc t}d|Dtt}|rddt |z}|d|d|d|d|d ||z}fd |D}||fd DS) ueDelegate-subagent ids to cascade-delete with *parent_ids*. Only rows carrying the ``_delegate_from`` marker (set at creation, and backfilled by the v16 migration) — generic untagged children keep the orphan-don't-delete contract. Walks marker chains recursively so an orchestrator subagent's own delegate children go too (FK safety). ch|]}||Srfrf.0sids r9 z._collect_delegate_child_ids..s . . .S# .S . . .r;,?zSELECT id FROM sessions WHERE z IN (z) OR (parent_session_id IN (z) AND z IS NOT NULL)c4g|]}|dv |dSidrf)rzr[founds r9 z/_collect_delegate_child_ids..s+UUU#c$iu>T>TCI>T>T>Tr;cg|]}|v| Srfrf)rzr{seedss r9rz/_collect_delegate_child_ids..s# 5 5 5CCu$4$4C$4$4$4r;)rgsetlistjoinlenexecutefetchallupdate)connrvdffrontierphcursorrrs @@r9_collect_delegate_child_idsrs   B . .J . . .E%jjEE{{H  XXcCMM) * * ER E Eb E E)+ E E35 E E E x    VUUU):):UUU X  6 5 5 55 5 5 55r;ct||}|rsddt|z}|d|d||d|d||d|d||S)Nr}r~*DELETE FROM messages WHERE session_id IN ()IUPDATE sessions SET parent_session_id = NULL WHERE parent_session_id IN ("DELETE FROM sessions WHERE id IN ()rrrr)rrvidsrs r9_delete_delegate_childrenrs %dJ 7 7C F XXcCHHn % % G"GGGMMM  1+- 1 1 1    ?"???EEE Jr;Tstate.dbcRttkrtStdz S)uResolve the default state DB path at call time. ``DEFAULT_DB_PATH`` is computed when this module is first imported, which freezes the developer's real ``~/.hermes`` even when a test fixture later redirects ``HERMES_HOME`` — importing this module during collection was enough to point every default ``SessionDB()`` at the real state.db. Precedence: 1. A deliberately re-pointed ``DEFAULT_DB_PATH`` (differs from the import-time snapshot — the established test escape hatch) wins. 2. Otherwise resolve ``get_hermes_home()`` fresh so a runtime ``HERMES_HOME`` redirect takes effect regardless of import order. r)DEFAULT_DB_PATH_IMPORT_DEFAULT_DB_PATHr rfr;r9_default_db_pathrs'111   z ))r;)zlocking protocolznot authorizeddisk i/o error_last_init_error_wal_fallback_warned_paths_wal_reset_bug_warned_pathsmsgcJt5|addddS#1swxYwYdS)uoRecord (or clear) the most recent state.db init failure. Thread-safe via _last_init_error_lock. Callers pass a message to record a failure or None to clear. SessionDB.__init__ only calls this to SET on failure — it deliberately does NOT clear on success, because in a multi-threaded caller (e.g. gateway / web_server per- request SessionDB() instantiation), a concurrent successful open racing past a different thread's failure would erase the cause string that thread's /resume handler is about to format. Explicit clears (e.g. test fixtures) are still supported by passing None. N)_last_init_error_lockr)rs r9_set_last_init_errorrEst s ctS)aKReturn the most recent state.db init failure, if any. Slash-command handlers (``/resume``, ``/title``, ``/history``, ``/branch``) call this to surface the underlying cause in their error messages when ``_session_db is None``. Returns ``None`` if SessionDB initialized successfully (or hasn't been attempted). )rrfr;r9get_last_init_errorrVs  r;)z:Review the conversation above and update the skill libraryz;Review the conversation above and consider saving to memoryc$t|tsdS|ddvrdS|d}t|tsdS|t fdt DS)ayTrue when ``msg`` is a persisted background-review harness prompt. These are user/system turns the forked skill/memory review agent wrote into a real session in older builds (before the ``_persist_disabled`` isolation fix). They instruct the agent to act as the curator under a hard tool restriction, so replaying them as live history hijacks the session. Frole>usersystemcontentc3BK|]}|VdSN) startswith)rzpheads r9 z8_is_background_review_harness_message..zs/DDatq!!DDDDDDr;)rWdictr_rXlstripany_REVIEW_HARNESS_PREFIXES)rrrs @r9%_is_background_review_harness_messagerjs c4 u wwv000uggi  G gs # #u >>  D DDDD+CDDD D DDr;messagesc|s|Sg}d}|D]\}t|rd}|r1d}t|tr|ddkrG||]|S)aSDrop background-review harness messages and the curator-mode assistant reply that immediately followed each one. Walk the list once; when a harness user/system message is found, skip it and also skip the next message if it is the assistant turn that answered it. Everything else passes through untouched and in order. FTr assistant)rrWrr_append)routskip_next_assistantrs r9 _strip_background_review_harnessr}s  "C   0 5 5 "&    "' #t$$ K)G)G 3 Jr;Session database not availablerncts|dSd}tfdtDrd}|d|dS)u~Format a user-facing 'session DB unavailable' message with cause. When ``SessionDB()`` init fails, callers set ``_session_db = None`` and several slash commands (/resume, /title, /history, /branch) previously responded with a bare ``"Session database not available."`` — no indication of WHY. This helper includes the captured cause (typically ``"locking protocol"`` from NFS/SMB) and points users at the known culprit so they can fix it themselves. Example output: Session database not available: locking protocol (state.db may be on NFS/SMB — see https://www.sqlite.org/wal.html). .r>c3DK|]}|vVdSr)lower)rzmarkercauses r9rz0format_session_db_unavailable..s0 G Gv6U[[]] " G G G G G Gr;uN (state.db may be on NFS/SMB/FUSE/ZFS — see https://www.sqlite.org/wal.html): )rr_WAL_INCOMPAT_MARKERS)rnhintrs @r9format_session_db_unavailablersq ! !E ||| D G G G G1F G G GGG`_ & & &t & & &&r;rc |d}n#tj$rYdSwxYw|dS|d}t |t r( |d}n#t$rYdSwxYw|3t| ndS)zRead the journal mode from the SQLite DB header on disk. Returns the mode string (e.g. ``"wal"``, ``"delete"``), or ``None`` if the value cannot be determined (new DB, or PRAGMA read failed). PRAGMA journal_modeNrascii) rfetchonesqlite3OperationalErrorrWbytesdecodeUnicodeDecodeErrorrXr`r)rr[modes r9_on_disk_journal_moders ll011::<<  #tt {t q6D$ ;;w''DD!   44 (,(83t99??   " " $ $ $dBs'*=="A88 BBctjdkrdS |ddS#tj$rYdSwxYw)uEnable ``PRAGMA checkpoint_fullfsync`` on macOS (no-op elsewhere). On Darwin, ``synchronous=FULL`` (the WAL default) issues a plain ``fsync()``, which Apple documents does *not* guarantee that data has reached stable storage or that writes are not reordered — see the ``fsync(2)`` man page. SQLite's WAL corruption-safety guarantee assumes the OS honors the fsync write barrier; macOS does not unless the app uses ``F_FULLFSYNC``. During a launchd *system* shutdown/reboot the OS page cache is dropped (effectively a power-loss event for in-flight pages), so a WAL checkpoint whose ``fsync()`` "reported" durable may never have hit the platter — corrupting ``state.db`` with a malformed image. This is the trigger in issue #30636 ("SIGTERM during launchd shutdown under high load"), distinct from a plain in-session kill (which the page cache survives and SQLite recovers from). ``checkpoint_fullfsync=1`` forces an ``F_FULLFSYNC`` barrier only at checkpoint boundaries — where WAL frames land in the main DB — so the cost amortizes to roughly +0.1 ms/commit (vs ~+4 ms for the broader ``fullfsync=1`` that flushes on every commit's WAL sync). Guarded by ``sys.platform == "darwin"`` because ``F_FULLFSYNC`` is macOS-only; on other platforms the PRAGMA is a no-op, so we skip it entirely. Best-effort: never raises. darwinNzPRAGMA checkpoint_fullfsync=1sysplatformrrrrs r9_apply_macos_checkpoint_barrierrsY6 |x  455555  #     +>>ctjdkrdS |ddS#tj$rYdSwxYw)uSEnforce ``PRAGMA synchronous=FULL`` on macOS to prevent btree corruption. On Darwin, the default ``synchronous=NORMAL`` only calls ``fsync()``, which Apple's fsync(2) man page explicitly states does *not* guarantee data-on-platter or write-ordering. During a WAL checkpoint race with process termination (e.g., launchd shutdown), this can leave the main DB with half-written btree pages → ``btreeInitPage error 11``. WAL mode's durability guarantee assumes the OS honors fsync barriers; macOS does not unless we explicitly set ``synchronous=FULL``, which issues a real ``fsync()`` on every transaction commit. The ``F_FULLFSYNC`` barrier at checkpoint boundaries is handled separately by :func:`_apply_macos_checkpoint_barrier`. This function is called after any successful WAL activation (either from ``apply_wal_with_fallback()`` setting a fresh WAL or when probing an existing WAL mode). It ensures macOS connections always use FULL synchronous mode, even if a prior connection set ``synchronous=NORMAL``. Best-effort: never raises. rNzPRAGMA synchronous=FULLrrs r9_enforce_macos_synchronous_fullrsY, |x  ./////  #     r version_infoc@||n tj}t|S)a:Return True when the linked SQLite library has the WAL-reset bug. Upstream documents the bug in versions 3.7.0 through 3.51.2, fixed in 3.51.3+, with backports 3.50.7 and 3.44.6: https://sqlite.org/wal.html#walresetbug Pre-WAL libraries (< 3.7.0) cannot hit the race and are treated as safe. )rsqlite_version_info_is_sqlite_wal_reset_vulnerable)rinfos r9rrs#(3<<9TD *4 0 00r;cH tjd} |d}|n#|wxYwn#tj$rYdSwxYw|r|ddSt |dS)zCReturn ``sqlite_source_id()``, or an empty string when unavailable.:memory:zSELECT sqlite_source_id()r>r)rconnectrrcloseErrorrX)rr[s r9sqlite_source_idrsz** ,,:;;DDFFC JJLLLLDJJLLLLL =rr #a&.r s1v;;s'A-'AA-A))A--B?Bc\ ddlm}|pi}|di}t|tsdS|dd}n#t $rYdSwxYwt|t sdS|}|dvr|ndS)azReturn the configured journal mode (``wal`` or ``delete``). ``database.journal_mode`` in config.yaml is the canonical operator setting. ``wal`` remains the default; use ``delete`` when the backing filesystem does not provide WAL-safe durability (for example macOS virtiofs, NFS, or SMB). Invalid or malformed values fail safely to the existing default. r)load_config_readonlydatabasewal journal_mode)rdelete) hermes_cli.configrr_rWrrKrXr`r)rconfigrrawrs r9resolve_journal_moder#s ::::::%%''-2::j"--(D)) 5ll>511 uu c3  u 99;;    D,,,44%7s=AA A&%A&ceZdZdZdS)WalUnsupportedErroraRaised by :func:`apply_wal_with_fallback` when ``require_wal=True`` and the filesystem cannot provide WAL journal mode. Covers both shapes of WAL refusal on network filesystems (NFS / SMB / FUSE / the AgentFS NFS overlay): SQLite *raising* ``SQLITE_PROTOCOL`` ("locking protocol"), and the quieter macOS-NFS case where ``PRAGMA journal_mode=WAL`` silently returns the still-effective mode without raising. Subclasses ``sqlite3.OperationalError`` so existing ``except sqlite3.OperationalError`` DB-init handling still catches it, while callers that specifically mandate WAL can catch this narrower type. N__name__ __module__ __qualname____doc__rfr;r9rr=s    r;rF)db_label require_walrrc\ t}trt|||dkS |d}|r,|ddkr t |t |dSn#tj$rYnwxYw|dkrt|d}|r't|d nd}|dkrtjd|pd d |S |d }|rA|d9t|d nd}|dkr t |t |dStd |d }|r|t|||pdS#tj$r} t| trt|  t fdt Dsd vrt#dD]} t%jd |d }nB#tj$r0} dt|  vr| } Yd } ~ yd } ~ wwxYw|rA|d9t|d nd}|dkr%t |t |Yd } ~ dSt)|} | dkr|rtt| | t|| |dYd } ~ dSd } ~ wwxYw)u Set ``journal_mode=WAL`` on ``conn``, falling back to DELETE on failure. Returns the journal mode actually set (``"wal"`` or ``"delete"``). On WAL-incompatible filesystems (NFS, SMB, some FUSE, ZFS), SQLite either raises ``OperationalError("locking protocol")`` / ``OperationalError("disk I/O error")`` or — on macOS NFS / SMB / the AgentFS NFS overlay — silently refuses the switch and leaves the DB in DELETE. Either way the degradation is logged at ERROR level (it is a real loss of concurrency — a write blocks concurrent readers — not a cosmetic warning) and, by default, the function falls back to DELETE (the pre-WAL default, which works on NFS and ZFS) so the feature keeps working. On SQLite builds that still contain the WAL-reset corruption bug (issue #69784), refuse to enable WAL on fresh / non-WAL databases (prefer DELETE). If the on-disk DB is already WAL, keep WAL and warn — never live-downgrade under possible concurrent openers. This gate (#70055) is deliberately RETAINED. An earlier revision of the lock-cancellation fix (#71724) reverted it on the theory that DELETE was "the mode that corrupts", but that comparison was confounded: the clean WAL result came from SQLite 3.53.1, which carries BOTH the WAL-reset fix AND 3.51.0's defenses against close()-broken POSIX locks, so it says nothing about 3.50.4. Re-measured on the actually-bundled 3.50.4 with the lock fix in place, WAL and DELETE are both clean (0/3 each) — i.e. there is no evidence that WAL is safer here, and upstream still documents the WAL-reset bug as real through 3.51.2 with serious consequences. Until a fixed runtime is delivered, keep new databases out of WAL. Callers that genuinely require WAL concurrency (and would rather fail loudly than run silently degraded) pass ``require_wal=True``; the function then raises :class:`WalUnsupportedError` instead of returning ``"delete"``. All current callers deliberately keep the default ``require_wal=False`` so NFS-homed installs keep working. The ERROR is deduplicated per ``db_label``: repeated connections to the same underlying DB (e.g. kanban_db.connect() which is called on every kanban operation) log once per process, not once per call. Different db_labels log independently, so state.db and kanban.db each get one error on the same NFS mount. Shared by :class:`SessionDB` and ``hermes_cli.kanban_db.connect`` so both databases get identical fallback behavior. Never downgrades to DELETE if the on-disk DB header reports WAL — see _on_disk_journal_mode. That holds for both the NFS path and the WAL-reset vulnerability path. r)rrequire_deleterrrPRAGMA journal_mode=DELETEr>2could not set configured journal_mode=delete (got no resultrzPRAGMA journal_mode=WALNz0journal_mode=WAL refused without raising (still c3 K|]}|vV dSrrf)rzrrs r9rz*apply_wal_with_fallback..s'EEV6S=EEEEEEr;rg?)rr_apply_delete_for_wal_reset_bugrrrrrrrXrr`r_log_wal_fallback_oncerWrrrangetimesleepr)rrr configured current_moder[actualr silent_excexc_ retry_excexistingrs @r9apply_wal_with_fallbackrKs,l&''J &'' . %1     ||$9::CCEE  LOu44 +D 1 1 1 +D 1 1 15  #     Xll788AACC(+3SV""$$$ X  *]VEZ{]]]  Jll455>>@@.1Pc!f6Hs3q6{{  ""((***b 5== +D 1 1 1 +D 1 1 15) Ht H H H     x444x  #111 c. / /  #hhnnEEEE/DEEEEE   s " "1XX   4   ,,'@AAJJLLCC/'s9~~/C/C/E/EEE#CHHHH "1v1AKK%%''--/// 5==3D9993D999 555555(.. u     9%c#hh//S 8x--- 1222xxxxxc1sdAB BBBG-*GN+(BN&+'JN&K"&K N& KA-N&AN&&N+)rrcd} |d}|rA|d9t|d}n#t j$rd}YnwxYw|dkr1t|dt|t|dSd} |d}|rA|d9t|d}n#t j$r|rYnwxYw|r |d krt jd |pd d t|d d S)zAvoid enabling WAL when the linked SQLite has the WAL-reset bug. - Already-WAL on disk: leave WAL alone (no live downgrade) and warn. - Otherwise: set DELETE and warn. - For an explicit operator request, verify SQLite accepted DELETE. r>rrNrT)kept_walrrrrrF) rrrXr`rrr_log_wal_reset_bug_oncerr)rrrcurrentr[r s r9rrsGll011::<<  23q6%#a&kk''))//11G  #% 48888'---'---u Fll788AACC  13q6%Q[[&&((..00F  #       &H,,& -)k - - -   Hu5555 8s%A*A//BBA*D++E?Erct5|tvr ddddSt|dddn #1swxYwY|rdnd}td|t j|dS)zHLog once per (process, db_label) about the WAL-reset vulnerability path.Nu\is already in WAL mode — leaving WAL in place (no live downgrade under concurrent openers)z1using journal_mode=DELETE instead of enabling WALuC%s: linked SQLite %s is vulnerable to the WAL-reset corruption bug (https://sqlite.org/wal.html#walresetbug) — %s. Upgrade to SQLite 3.51.3+ (or backports 3.50.7 / 3.44.6); Hermes-managed installs can repair the embedded runtime with `hermes update`. See `hermes doctor`. This warning fires once per process per database.)_wal_reset_bug_warned_lockraddloggerwarningrsqlite_version)rractions r9rr s $22 2 2 2 22222222 $''111222222222222222  A . .A   NN       AAA  A rct5|tvr ddddSt|dddn #1swxYwYtd||dS)uLog a single ERROR per (process, db_label) about WAL fallback. ERROR (not WARNING): a DB silently dropped to DELETE means a real loss of concurrency — under the kanban dispatcher + workers a write blocks readers, surfacing as SQLITE_BUSY/lock contention — so it must be loud, not cosmetic. Without this dedup, NFS users running kanban (which opens a fresh connection on every operation — see hermes_cli/kanban_db.py) would fill errors.log with hundreds of identical errors per hour. Nu%s: WAL journal_mode unsupported on this filesystem (%s) — falling back to journal_mode=DELETE (slower rollback-journal mode; reduces concurrency but works on NFS/SMB/FUSE/ZFS). See https://www.sqlite.org/wal.html for details. This message fires once per process per database.)_wal_fallback_warned_lockrrrerror)rrs r9rr=s #11 1 1 1 11111111 #&&x000111111111111111 LL /  rrc ddlm}m}|}n#t$rYdSwxYwdD]}||d|d}| t t |}n4#ttf$r t d|||YtwxYw | d|d |#tj $rYwxYwdS) u+Apply optional performance and WAL-sizing PRAGMAs from ``config.yaml``. Reads the ``database:`` section and applies configurable PRAGMAs when set to integer values. The journal mode itself is NOT handled here — ``database.journal_mode`` is owned by :func:`resolve_journal_mode` inside :func:`apply_wal_with_fallback`, which layers the operator setting under all the safety guards (never live-downgrading an on-disk WAL DB, filesystem fallback, WAL-reset-bug gating). Supported keys under ``database:`` in config.yaml: * ``cache_size`` — negative value = KiB, positive = pages (e.g. ``-262144`` = 256 MB page cache) * ``mmap_size`` — max bytes for memory-mapped I/O (0 = disabled) * ``temp_store`` — 0=DEFAULT(file), 1=FILE, 2=MEMORY, 3=ALWAYS * ``wal_autocheckpoint`` — WAL auto-checkpoint threshold in pages * ``journal_size_limit`` — max journal/WAL size in bytes Best-effort: config load or pragma failures are ignored so DB init never breaks on a malformed ``database:`` section. r)cfg_getrN) cache_size mmap_size temp_storewal_autocheckpointjournal_size_limitr)defaultz'%s: ignoring non-integer database.%s=%rzPRAGMA =)rr%rrKrCrXr`rErFrrrrr)rrr%rcfg pragma_name raw_valuerUs r9apply_database_pragmasr0ZsL4CCCCCCCC""$$   GC[$GGG    I,,..//EE:&    NN9     H   LL8;8888 9 9 9 9'    D -s/ ##.A,,.BB!B==CC)zmalformed database schemaz database disk image is malformed_repair_attempted_pathsc|ttjsdStfdtDS)zTrue if *exc* is a SQLite 'malformed schema / disk image' error. These are the corruption classes where the schema fails to parse, so targeted ``sqlite_master`` surgery (not an ordinary FTS rebuild) is the only recovery path. Fc3^K|]'}|tvV(dSrrXr)rzrrs r9rz(is_malformed_db_error..s8RRfvS)))RRRRRRr;)rWr DatabaseErrorr_MALFORMED_SCHEMA_MARKERSrs`r9is_malformed_db_errorr8sC c70 1 1u RRRR8QRRR R RRr;)zno space left on deviceznot enough spacezdatabase or disk is fullz disk fullz full diskenospcc,|dSt|tr!t|ddtjkrdSt|t r|nt |}|tfdtDS)aTrue when *exc* (or a stringified error) is a disk-full / ENOSPC failure. Covers: * ``OSError`` with ``errno.ENOSPC`` * SQLite ``OperationalError: database or disk is full`` (SQLITE_FULL) * Plain English / errno strings that survive RPC wrapping NFerrnoTc3 K|]}|vV dSrrf)rzrlowereds r9rz%is_disk_full_error..s(BBVv BBBBBBr;) rWrPgetattrr;ENOSPCrXrr_DISK_FULL_MARKERS)rtextr=s @r9is_disk_full_errorrBs {u#wGC$$?$?5<$O$OtS#&& 433CHHDjjllG BBBB/ABBB B BBr;db_pathct|}t5|tvr ddddSt| ddddS#1swxYwYdS)aClaim the one-shot repair attempt for *db_path* in this process. Returns True for the first caller, False afterwards. Keeps a malformed DB from triggering an unbounded repair/reopen loop and stops concurrent callers from racing surgery on the same file. NFT)rX_repair_attempt_lockr1r)rCrps r9_claim_repair_attemptrFs g,,C  ) ) ) ##C((( s AAAAcddl}ddl} ddlm}n#t$rd}YnwxYw|(||rt d|dS|jd}| |j d|} | ||dD]d}| |j |z}| r1| || |j |ze|S#t$r'}t d||Yd}~dSd}~wwxYw) aCopy a (possibly malformed) DB file to a timestamped backup beside it. Raw file copy on purpose: the DB won't open cleanly, so we preserve the bytes exactly for forensics / manual restore. WAL and SHM sidecars are copied too when present. Returns the backup path, or None on failure. Refuses when a connection to this database is still live in the process: reading the file would ``close()`` a descriptor for it and cancel that connection's POSIX advisory locks (see ``hermes_cli.sqlite_safe_read``). The repair path can be entered by one SessionDB while the gateway holds others, so this is a real possibility rather than a theoretical one. rN)has_live_connectionzRefusing to raw-copy %s for backup: a connection to it is still open in this process and the copy would cancel that connection's POSIX locks. Close all SessionDB handles first.z %Y%m%d_%H%M%Sz.malformed-backup--wal-shmz%Could not back up malformed DB %s: %s)datetimeshutilhermes_cli.sqlite_safe_readrH ImportErrorrr"nowstrftime with_namerLcopy2existsrKr) rCrLrMrHstamp backup_pathsuffixsidecarrs r9_backup_db_filerYsOOOMMM#CCCCCCC ###"#&+>+>w+G+G&  >     t   ! ! # # , ,_ = =E##w|$N$Nu$N$NOOK  Wk***& X XF'' v(=>>G~~ X Wk&;&;KMMMttttts%  A>D E "EE ct|}|dks|drdS ttn#t $rdYnwxYwdtdt ffd dddtd t ddffd }|j}|r ||d d D]B}|r| |j |zn|}| r ||CdS) u5Refuse-or-repair read-only DB files BEFORE the first connection opens. Port of Kilo-Org/kilocode#12508's startup preflight. A stray read-only ``state.db`` / ``-wal`` / ``-shm`` (sudo run, restored backup, copied dotfiles) previously surfaced as an opaque ``sqlite3.OperationalError: attempt to write a readonly database`` raised from deep inside ``_init_schema`` — naming no file and no fix — and the obvious wrong "fix" (deleting the ``-wal``) silently loses committed transactions. This preflight: - **Repairs** permissions with ``chmod u+rw`` when the file lives inside the Hermes home tree (``get_hermes_home()``) — the safe repair scope: Hermes owns those files, and the OS makes ``chmod`` fail on files the user doesn't own, which bounds the repair exactly. - **Fails fast with an actionable error** naming the exact file and the exact ``chmod`` command for anything else (root-owned files, read-only mounts, custom paths outside the home tree). - Never deletes or truncates a WAL sidecar — once writable, the normal open path checkpoints its committed frames into the DB as intended. ``:memory:`` and ``file:`` URI paths are skipped (no plain on-disk files to check). Shared by :class:`SessionDB` and ``hermes_cli.kanban_db``. rfile:Nrr3cdS |S#ttf$rYdSwxYwNF)resolveis_relative_torPrF)rhomes r9_in_repair_scopez2preflight_db_writability.._in_repair_scopeDsU <5 99;;--d33 3$   55 s&.AAF)is_dirrbc nddl}tj|tjtjzrdS|r |j|jz|r|jndz}tj||j |zn#t$rYnwxYwtj|tjtjzr#t d||rdnddS|rdnd}|j drdnd}tjd |d |d |rdndd |d | )Nrz2%s preflight: repaired read-only %s (chmod u+rw%s)xr> directoryfilerJuv Do NOT delete the -wal file — it contains committed data that will be merged into the database once it is writable.z is not writable:  zf is read-only for this user. Hermes needs read-write access to open the database. Fix with: chmod u+rwz 'z4' (files owned by another user may need sudo/chown).)statrGaccessR_OKW_OKS_IRUSRS_IWUSRS_IXUSRchmodst_moderPrrrLendswithrr)rrb_statrkindwal_noterars r9_ensure_writablez2preflight_db_writability.._ensure_writableLs 9Q"') * *  F  A    mem37Uu}}TUVAFFHH,s23333    yBGbg-..  H!)CCr  $0{{&vv&&  D D  & M M4 M M! M M*0#833b M M<= M MCK M M   sA B BBT)r>rJrK) rXrrr r^rKboolparentrbrRrLis_file) rCrrrurwrWrrar`s ` @@r9preflight_db_writabilityrys8 g,,C jCNN733#O$5$566>>@@ DT5:   D T d       B^F }}. ----&  8> KG  glV3 4 4 4G 99;;  Q     s-A A.-A.cTtjt|d} t||d|d}d|D}|r1d|dd|S|dd D]} |d |d |d 2#tj $ru}t |rYd}~`t| }d |vsd|vrYd}~d|d|cYd}~c|Sd}~wtj $r(}d|d|cYd}~c|Sd}~wwxYwdtj} |d|d|dtjf|d|ddtjf|dn#tj $r} |dn#tj$rYnwxYwt| }d |vsd|vrYd}~|dSd|vrYd}~|dSt|cYd}~|Sd}~wwxYw |dS#tj $r-}t|cYd}~|Sd}~wwxYw#|wxYw)u/Probe a DB on a fresh connection. Returns None if healthy, else a reason. Runs the same first-statement (``PRAGMA journal_mode``) that trips the malformed-schema parse, then ``PRAGMA integrity_check`` and a canonical ``sessions`` read, and finally a rolled-back ``messages`` write so that FTS5 index corruption — which leaves base-table reads and ``integrity_check`` passing while every ``INSERT INTO messages`` fails through the FTS triggers — is reported as unhealthy rather than slipping past as a false "ok" (#50502). Nisolation_levelrzPRAGMA integrity_checkcg|]D}|t|ddk/t|dES)rokr4rzrs r9rz%_db_opens_cleanly..sEOOO!qOS1YY__5F5F$5N5NC!II5N5N5Nr;z; zSELECT COUNT(*) FROM sessions messages_ftsmessages_fts_trigrammessages_fts_cjkzSELECT 1 FROM  WHERE z MATCH '""' LIMIT 1z no such tablezno such columnzfts5 read probe failed on r_hermes_fts_health_probe_BEGIN IMMEDIATEz>INSERT INTO sessions (id, source, started_at) VALUES (?, ?, ?) _health_probezOINSERT INTO messages (session_id, role, content, timestamp) VALUES (?, ?, ?, ?)r_fts_health_probeROLLBACK no such tokenizer: cjk_unicode61)rrrXload_fts5_cjk_extensionrrrrrr SessionDB_is_fts5_unavailable_errorrr5r time_nsr)rCrrowsproblems fts_tablerrprobe_session_ids r9_db_opens_cleanlyrysG ?3w<< > > >Do %%% *++44666||455>>@@OOtOOO  +99Xbqb\**F E 455>>@@@ V) G) GI( G WYWWyWWW(****+ G G G77<<HHHH#hhnn&&"c))-=-D-DHHHHFIFFFFFFFFFFF^ ]( G G G GIFFFFFFFFFFFR ] GHt|~~GG  LL* + + + LLP!?DIKK@    LL&!6+> L    LL $ $ $ $'     Z((((=    c((..""C#%%)9S)@)@ttt 2S88 ttt s88OOOOO )     3xx  sBM+M-.DMG+FM )F3M8F?GMG-G4G5MGM/A?I/.M/L8?JL3J'$L3&J'',L3M-L31M L3L8M3L88MN!N /N0N NNN'T)backuprc ddddd}t|}|s |d|d<|St| d|d<d|d <|S|r%t|}|rt |nd|d < t jt |d } t|d D]4} |d |d|d #t j $rY1wxYw | n#| wxYwt|'d|d<d|d <t d||Sn7#t j $r%}t d|Yd}~nd}~wwxYw t jt |d } |d|| n#| wxYwt|'d|d<d|d <t d||Sn7#t j $r%}t d|Yd}~nd}~wwxYw t jt |d } |d|d}|D] \}} } } |d|| | f!|d|| n#| wxYwt|'d|d<d|d <t d||Sn7#t j $r%}t d|Yd}~nd}~wwxYw t jt |d } |d|d|d||d| n#| wxYwt|} | 'd|d<d |d <t d!||S| |d<n.#t j $r}t ||d<Yd}~nd}~wwxYw|ds"td"||d |S)#aRepair a state.db whose ``sqlite_master`` schema is malformed or whose FTS indexes reject writes. Handles two corruption classes: the "duplicate object definition" / malformed-schema class where even ``PRAGMA`` statements fail, and the FTS write-corruption class (#50502) where base tables read fine and ``integrity_check`` passes but writes fail through the ``messages_fts*`` triggers. Tries least-destructive recovery first and escalates: 1. **Rebuild FTS indexes in place** via the FTS5 ``'rebuild'`` command, which rewrites the internal b-tree segments from the canonical ``messages`` rows without dropping or recreating anything. Fixes the FTS write-corruption class while preserving the schema intact. 2. **De-duplicate** ``sqlite_master`` (keep the lowest rowid per ``type``/``name``). Fixes the canonical "table X already exists" case and PRESERVES the existing FTS index intact. 3. **Drop the FTS schema** (every ``messages_fts*`` object) + ``VACUUM``. The next ``SessionDB()`` open rebuilds the FTS indexes from the canonical ``messages`` table. Canonical ``sessions`` / ``messages`` rows are never modified. A timestamped raw backup is taken first unless ``backup=False``. Returns a report dict: ``{repaired: bool, strategy: str|None, backup_path: str|None, error: str|None}``. FN)repairedstrategyrVr"z does not existr"Tralready_healthyrrVr{rz INSERT INTO (z) VALUES('rebuild') rebuild_ftsz 1zFDELETE FROM sqlite_master WHERE type IS ? AND name IS ? AND rowid <> ?zPRAGMA writable_schema=OFF dedup_schemazRstate.db schema repaired by de-duplicating sqlite_master (FTS index preserved): %sz%state.db dedup repair pass failed: %sz9DELETE FROM sqlite_master WHERE name LIKE 'messages_fts%'VACUUMdrop_fts_rebuildzdstate.db schema repaired by dropping FTS schema; indexes will rebuild from messages on next open: %szsstate.db schema repair could not recover %s automatically (backup: %s); manual restore from backup may be required.)rrTrrYrXrrrrrrrrr5commitrr") rCrreportbpathr table_namerdupestype_rL_countkeepreasons r9repair_state_db_schemars8 F7mmG >>  $555w !!)!z.z  >((.3 =E } Ms7||TBBB  $D ) ) )   LLSzSSJSSS/H  JJLLLLDJJLLLL W % % -!%F: !.F:  NNN   M .  MMMFLLLLLLLLM@s7||TBBB  LL # # # KKMMM JJLLLLDJJLLLL W % % -!%F: !0F:  NNA7   M .  @@@93????????@Es7||TBBB  LL4 5 5 5LLFhjj .3  )tVT CD$' LL5 6 6 6 KKMMM JJLLLLDJJLLLL W % % -!%F: !/F:  NN,-4   M .  EEE>DDDDDDDDE#s7||TBBB  LL4 5 5 5 LLT U U U LL5 6 6 6 KKMMM LL " " " JJLLLLDJJLLLL"7++ >!%F: !3F:  NN>?F   M w  ###c((w# *    H VM*   Ms8#EC80C  C8 CC8CC8#E8D8EE<E77E<#H2$)G" H2"G888H22I&I!!I&*#M;BL+M;+M8M;;N/ N**N/3#R+A(Q?R+Q**:R+%R++S:SSa\ CREATE VIEW IF NOT EXISTS messages_fts_cjk_src AS SELECT id, role, content, tool_name, tool_calls FROM messages WHERE role <> 'tool'; CREATE VIRTUAL TABLE IF NOT EXISTS messages_fts_cjk USING fts5( content, tool_name, tool_calls, content='messages_fts_cjk_src', content_rowid='id', tokenize='cjk_unicode61' ); a  CREATE TRIGGER IF NOT EXISTS messages_fts_cjk_insert AFTER INSERT ON messages WHEN new.role <> 'tool' AND (new.id > COALESCE((SELECT CAST(value AS INTEGER) FROM state_meta WHERE key = 'fts_cjk_rebuild_high_water'), -1) OR new.id <= COALESCE((SELECT CAST(value AS INTEGER) FROM state_meta WHERE key = 'fts_cjk_rebuild_progress'), -1)) BEGIN INSERT INTO messages_fts_cjk(rowid, content, tool_name, tool_calls) VALUES (new.id, new.content, new.tool_name, new.tool_calls); END; CREATE TRIGGER IF NOT EXISTS messages_fts_cjk_delete AFTER DELETE ON messages WHEN old.role <> 'tool' AND (old.id > COALESCE((SELECT CAST(value AS INTEGER) FROM state_meta WHERE key = 'fts_cjk_rebuild_high_water'), -1) OR old.id <= COALESCE((SELECT CAST(value AS INTEGER) FROM state_meta WHERE key = 'fts_cjk_rebuild_progress'), -1)) BEGIN INSERT INTO messages_fts_cjk(messages_fts_cjk, rowid, content, tool_name, tool_calls) VALUES ('delete', old.id, old.content, old.tool_name, old.tool_calls); END; CREATE TRIGGER IF NOT EXISTS messages_fts_cjk_update AFTER UPDATE OF content, tool_name, tool_calls, role ON messages WHEN (old.content IS NOT new.content OR old.tool_name IS NOT new.tool_name OR old.tool_calls IS NOT new.tool_calls OR old.role IS NOT new.role) AND (old.id > COALESCE((SELECT CAST(value AS INTEGER) FROM state_meta WHERE key = 'fts_cjk_rebuild_high_water'), -1) OR old.id <= COALESCE((SELECT CAST(value AS INTEGER) FROM state_meta WHERE key = 'fts_cjk_rebuild_progress'), -1)) BEGIN INSERT INTO messages_fts_cjk(messages_fts_cjk, rowid, content, tool_name, tool_calls) SELECT 'delete', old.id, old.content, old.tool_name, old.tool_calls WHERE old.role <> 'tool'; INSERT INTO messages_fts_cjk(rowid, content, tool_name, tool_calls) SELECT new.id, new.content, new.tool_name, new.tool_calls WHERE new.role <> 'tool'; END; ctjd}|r!t|St dz dz S)z1Location of the cjk_unicode61 loadable extension.HERMES_FTS5_CJK_SOlibzlibfts5_cjk.so)rGgetenvr expanduserr )envs r9fts5_cjk_so_pathrsI )( ) )C &Cyy##%%%   u $'7 77r;cxtjdddvS)zBconfig.yaml ``sessions.cjk_fts`` (default on), via its env bridge.HERMES_CJK_FTS1)0falseoffno)rGrr`rrfr;r9_cjk_fts_config_enabledrs: 9%s + + 1 1 3 3 9 9 ; ;D r;ctsdSt}|sdS |d |t ||dn#|dwxYwdS#t $r!td|dYdSwxYw)uFBest-effort load of the cjk_unicode61 tokenizer into ``conn``. Returns False (never raises) when the .so is absent, the feature is disabled via ``sessions.cjk_fts``, or this Python build has extension loading compiled out — every caller treats False as "behave exactly as before the cjk index existed". FTz#fts5_cjk extension load failed (%s)exc_info) rrrTenable_load_extensionload_extensionrXrKrr)rpaths r9rr s # $ $u   D ;;==u  ""4((( .   D * * *  & &u - - - -D & &u - - - -t A non-owner tried to write while compression owns the session.Nrrfr;r9rr/sHHHHr;rceZdZdZdS)!SessionCompressionInProgressErroraA concurrent writer collided with a *live* compression lock. Split out from :class:`CompressionSessionBusyError` because the two conditions that class covers need opposite handling. This one is transient: a healthy compressor holds the session for a few seconds and the lock row carries its own ``expires_at``, so the write can simply wait (see ``_execute_write``'s patience loop). The other case, a compressor discovering its own lease is gone, is permanent and must fail fast rather than spin out the whole patience budget. Subclassing keeps every existing ``except CompressionSessionBusyError`` handler working unchanged. Nrrfr;r9rr3s    r;rc ddlm}nJ#t$r=td|t jt|fi|cYSwxYw||f|t jd|S)a``sqlite3.connect`` that registers the open fd for lock-safety. While a connection is live, byte-level probes of the same file are refused: an ``open()``/``close()`` cancels every POSIX advisory lock this process holds on it -- including a running VACUUM's EXCLUSIVE lock. Released automatically on ``close()``. The ONLY tolerated fallback is the helper being absent entirely (scaffold/embed installs that ship hermes_state without hermes_cli). A real connection failure must propagate: silently retrying an *untracked* connect would disable the guard for the lifetime of that connection, which is precisely the failure mode this module exists to prevent. r)connect_trackedzihermes_cli.sqlite_safe_read unavailable; opening %s untracked (byte-probe guard inactive in this install)) tracking_path connect_fn)rNrrOrdebugrrrX)rrkwargsrs r9_connect_tracked_dbrCs4??????? 444  :    s4yy33F33333 4 ?  #?      s AAAd probe_bytesforcerrrc\ ddlm}||||S#t$rYnwxYw |j}n#t $rYdSwxYw|dkrdSddlm}||td||}|r| drdStd |DS) aDetect the #68474 zeroed state.db signature (size>0, NUL header). Byte-level probe, so it is only safe BEFORE any connection to *path* exists in this process: ``close()`` cancels every POSIX advisory lock the process holds on the file, which can pull the EXCLUSIVE lock out from under a running VACUUM and corrupt the database. The read is routed through ``read_header_bytes_preopen``, which refuses (returning False here) once a connection is live. Pass ``force=True`` only for offline files -- quarantined copies, snapshots, archives. Prefer ``hermes_cli.backup.is_zeroed_sqlite_file`` when available; this local copy keeps SessionDB openable without importing the CLI package in constrained embed paths. r)is_zeroed_sqlite_filerF)read_header_bytes_preopen)lengthrsSQLite format 3c3"K|] }|dkV dS)rNrf)rzbytes r9rz%is_zeroed_state_db..s&**Ttqy******r;) hermes_cli.backuprrKrhst_sizerPrNrmaxrall)rrrrsizerrs r9is_zeroed_state_dbrfs" ;;;;;;$$T{%PPPP      yy{{" uu qyyuEEEEEE $ $ S[))   D 4??#566u **T*** * **s ##A AAc rddl}||jdz}|jdd|d}d} t jdz}|d krddl } | d| | |j d d}n>#t$r0t j|krYnt jd YnwxYwnddl} || |j|jzd}nE#t(tf$r0t j|krYnt jd YnwxYw~|st*d | |r|d krHddl }| d| | |jd n1ddl}|| |jn#tt2f$rYnwxYw|dS#|wxYw|st*d | |r|d krHddl }| d| | |jd n1ddl}|| |jn#tt2f$rYnwxYw|dS#|wxYwt;|st*d| |r|d krHddl }| d| | |jd n1ddl}|| |jn#tt2f$rYnwxYw|dS#|wxYw t jd}n#t>$rd}YnwxYw||jd|dtAj!d} d} | rP| d z } ||jd|dtAj!d| d} | P |"| n#t$r} t*d|| Yd} ~ |r|d krHddl }| d| | |jd n1ddl}|| |jn#tt2f$rYnwxYw|dS#|wxYwd} ~ wwxYwdD]y} tGtI|| z} | rD | "tGtI| | zi#t$rYuwxYwz| |r|d krHddl }| d| | |jd n1ddl}|| |jn#tt2f$rYnwxYw|S#|wxYw# |r|d krHddl }| d| | |jd n1ddl}|| |jn#tt2f$rYnwxYw|w#|wxYwxYw)a`Move a zeroed state.db aside (preserve bytes) and return quarantine path. Uses a cross-process lock (``#68805``) so two concurrent startups cannot race: the first process moves the zeroed file and the second re-checks under the lock, finding the file already gone (or a fresh DB in its place) instead of clobbering the quarantine. rNz.quarantine.lockTparentsexist_okza+bF@Windowsr?{Gz?uquarantine lock for %s not acquired within 5s — refusing to quarantine without the cross-process lock. The zeroed file is left in place. If sessions fail to load, restore from state-snapshots via `hermes snapshot list` / `hermes snapshot restore `.z?quarantine_zeroed_state_db: %s already moved by another processznquarantine_zeroed_state_db: %s is no longer zeroed (another process quarantined it and a fresh DB was created)z %Y%m%d-%H%M%Sunknownz.zeroed--z.bakz"Failed to quarantine zeroed %s: %srI)%rrRrLrwmkdiropenr  monotonicrmsvcrtseeklockingfilenoLK_NBLCKrPr fcntlflockLOCK_EXLOCK_NBBlockingIOErrorrr"LK_UNLCKLOCK_UNAttributeErrorrrTrrrQrKrGrHrenamerrX)rr lock_pathhandleacquireddeadlinerrtsdestnrrWsides r9quarantine_zeroed_state_dbr s OOOty+==>>I 4$777 ^^E " "FHa>##c) ??   ) ) MMM &&KKNNNNN6==??FOQGGG#H&&&~''833Ju%%%%%& & LLL &&KK 1NOOO#H'1&&&~''833Ju%%%%%&  &  LL2     `  @??$$ 11!MMMKKNNNNN6==??FOQGGGG LLLKK ???(    D  LLNNNNNFLLNNNNs{{}}  KKQ   P  @??$$ 11!MMMKKNNNNN6==??FOQGGGG LLLKK ???(    D  LLNNNNNFLLNNNNg"$''  KKE    B  @??$$ 11!MMMKKNNNNN6==??FOQGGGG LLLKK ???(    D  LLNNNNNFLLNNNNW //BB   BBB ~~y 8 8" 8 8ry{{ 8 8 8   kkmm  FA>>9@@b@@29;;@@@@@Dkkmm    KK        LL=tS I I I444  @??$$ 11!MMMKKNNNNN6==??FOQGGGG LLLKK ???(    D  LLNNNNNFLLNNNN3 '  FD F*++D{{}} KKSYY%7 8 89999D    @??$$ 11!MMMKKNNNNN6==??FOQGGGG LLLKK ???(    D  LLNNNNFLLNNNN  @??$$ 11!MMMKKNNNNN6==??FOQGGGG LLLKK ???(    D  LLNNNNFLLNNNNs3_ AC_!D 1_3D _ D  _7E _ (F5_7F _ F!_1BII2II2II22J /__SB_U64_6 ZY<_"BX65Y#6Y Y# Y  Y##Y9<Z:_<2[/._/ [<9_;[<<_B^_^*'_)^**__b6Ba10b1bbbbb6b33b6c4eZdZUdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZd ZdZdZdZdZedeedeefdZeddZedejdeeeffdZdde de!fdZ"deej#fdZ$e%dZ&ed ej'de!fd!Z(ed ej'de!fd"Z)ed#ej*de!fd$Z+d ej'ddfd%Z,d ej'ddfd&Z-dd'Z.ed#ej*ddfd(Z/d#ej*d)ed*ede!fd+Z0 dd,e1ej#ge2fd-ee3de2fd.Z4d/e3d-e3de!fd0Z5ed ej6de!fd1Z7d ej6de!fd2Z8dd3Z9d4Z:d Z;d5Zde!fd8Z? dd9ed:ed;edeed?ed@edAedBedCedDedEeddfdFZ@d9ed:edefdGZAdddddddddHd9ed:ed=ed>ed?ed@edAedIedJedKe!ddfdLZBdd9edNe!ddfdOZCdPdQd>edRedSeddfdTZDdPdQdUeeefdSeddfdVZEdPdQdSedeeeffdWZFdPdQdXeGedSeddfdYZHddMdZd[eed\e!deGeeeffd]ZIddd^d[ed?edAeed=eedeef d_ZJdddddd`d:ed=eed>eed?eed@eedAeedeeeeffdaZKdBedeeeeffdbZLdddddddMdcdBedded:edeeGeeefd;ededeGeeeff$dZdZededefdZededefdZededeefdZd9edfeeddfdZededeeeeffdZ dd9ededededededeXdededededededed e!d eed ed eed eedeeeefdfeedeXf,dZ dd9edeeGeeefdfeedeeXdeXf dZddd9ededed edeeeefde!f dZdZddd9edeXdeededeeGeeeff dZd9edeXdeGeeeffdZddd9ededeGeeeffdZdddMdd9ededeXde!deeXf dZd9edeeXfdZd9ed eXdeefd!Zd9edeeGeeefdeeXeXffd"Z dd9edeeGeeefd\e!ddfd#Zd9ede!fd$Zd9ed%eGeeefdeXfd&Zd9eded edeXfd'Z dd9ed(e!deeXdeXdeGeeeff d)Z dd9ed+eXd,eXdeeeffd-Zd9edefd.Z dd9ed/e!d(e!d0e!d1e!deGeeeff d2Zd3Zdd4d9ed/e!d0e!d1e!deGeeeff d5Zd9edereGeeefeGeeefffd6Zd9edeGeeeffd7Zd9edefd8Zd9edeGefd9ZedeeGeeefd:eeefde!fd;Zd9edeXdeXfd?Z dd:edeXdeXd@edeGeeeff dAZ dd:edeGededeXde!de!dBe!deGedeXfdCZddEeXde!fdFZddddGde!de!dBe!deeeXffdHZdd9edeXfdIZd9edede!fdJZdKeeefde!fdLZdMeeefde!fdNZd9edeGefdOZd9eddfdPZedee d9eddfdQZd9edeGefdRZ dd9edee dSeeGede!fdTZ dd9edee de!fdUZ ddVeGedee deXfdWZdeXfdXZ ddee deXfdYZeddddddddddddddddddddddddZd[ee3d\ee3d]ee3d^ee3d:eed_eedieedeed`eeXdaeeXdee!dbeedeed=eed?eed@eedceeddeeXdeeeXdfee3dgee3dheeXdieeXdereeff0djZ ddkee3d:edeGeeeffdlZ ddkee3d:edeXfdmZdMdndoe3dpe!deXfdqZ ddkee3d:edee deXfdsZːdtedeefduZddvdtedwed#eej*ddfdxZ͐dyedeXfdzZΐdd{Zddd|d?ed=ed}ee!d~ee!ddf dZdMdd?ede!ddfdZd?ed=ede!fdZd?edAedeeeeffdZd?edeGeeeffdZd9edeeeeffdZd?edAedeXfdZ֐ddd?edAed=ed>ed9ededdfdZd9ede!fdZؐddd?ed=edeXdeGeeeffdZِdZdeeXfdZdeXfdZ ddeXdeXde!dee deXdeeeff dZ ddoe3deXdpe!deeeffdZd9ed[ede!fdZd9edeeeeffdZdeGeeeffdZd9ede!fdZd9eddfdZd9edteddfdZdS(rz SQLite-backed session storage with FTS5 search. Thread-safe for the common gateway pattern (multiple reader threads, single writer via WAL mode). Each method opens its own cursor. g4@gN@g?rrg333333?g@g??2ii'iPiPir2r3c\|dSt|}|d||f|S)NzAINSERT OR IGNORE INTO system_prompts (hash, prompt) VALUES (?, ?))r:r)rr2 prompt_hashs r9_store_system_promptzSessionDB._store_system_promptTsC  4)-88  O - (   r;Nc0|ddS)Nz|DELETE FROM system_prompts WHERE NOT EXISTS (SELECT 1 FROM sessions WHERE sessions.system_prompt_hash = system_prompts.hash)rrs r9#_delete_unreferenced_system_promptsz-SessionDB._delete_unreferenced_system_prompts_s'       r;r[cht|}d|vr|d}d|vr||d<|S)N_system_prompt_resolvedr2)rpop)r[dataresolveds r9_session_row_dictzSessionDB._session_row_dictisDCyy $ , ,xx 9::H$&&(0_% r;FrC read_onlyc |p t_|_tj_tj_t_ tj_ d_ d_ d_ d_d_d_d_d_d_d_d_t-_tjtj_d_d_d_ |rt;djdjdddd_t<jj_ tCjd j"}#|d du_jr#|d du_nD#tH$r7jdc}_ |%n#tL$rYnwxYwwxYwdSjj'(dd |stSjd |sj*rtWjr j,j-}n#t\$rd }YnwxYwt_j}jj'dz }d|d|pdd|d}t`1|te||Aj*r(tWjrt=j3|fd fd} | dS#t<j3$r} ti| rtkjst`1d|  jj%n#tL$rYnwxYwtmj} | 7ds| Yd} ~ dSd} ~ wwxYw#tL$r,} tetq| j9d| d} ~ wwxYw)NFrr[?mode=roTr )ruricheck_same_threadtimeoutr|rr#rrrzstate-snapshotszstate.db looks ZEROED (z( bytes, no SQLite header). Preserved at u*(quarantine failed — file left in place)z. Restore from z via `hermes snapshot list` / `hermes snapshot restore ` if available. Opening a fresh empty database so the agent can start.cttjddd_tjj_tjddk_tjdj dtj_ dS)NFr )rr r|rr#rzPRAGMA foreign_keys=ON)rrXrC_connrRow row_factoryr _wal_activer0rr_fts_cjk_loaded _init_schemarsr9_connect_and_initz-SessionDB.__init__.._connect_and_inits0 %%&+ %)    *1 &+DJLLLPUU 'tzJGGGG ""#;<<<'>tz'J'J$!!#####r;c tjjz} dS#tj$r}t |}d|vrd|vr jjn#t$rYnwxYwtj}||krtj ttj jjt!||z dYd}~nd}~wwxYw)NTlockedbusyMbP?)r r_WRITE_PATIENCE_SrrrXrr#rrKr minrandomuniform_WRITE_RETRY_SLOW_MIN_S_WRITE_RETRY_SLOW_MAX_Sr)rrerrrPr*rs r9$_connect_and_init_with_lock_patiencez@SessionDB.__init__.._connect_and_init_with_lock_patiencesO >++d.DD))+++"3!#hhnn..#3..63D3D!!#z5 $ 0 0 2 2 2(!!! D!"n..(??!  &$($@$($@!"!"!$HsNE : :  s9 +D *D% BD BDBA-DD u`state.db schema is malformed (%s) — attempting automatic repair (a backup copy is made first).rr):rrCr threadingLock_locklocal _read_localr _read_conns_read_conns_lock_read_conns_closedr& _write_count_fts_runtime_rebuild_attempted_fts_usermerge_floor_applied _fts_enabled_trigram_availabler'_fts_cjk_available_fts_unavailable_warnedr#r _token_queue Condition_token_queue_cond_token_writer_thread_token_writer_stop_token_writer_busyrrr$r%r0r_fts_table_probe BaseExceptionrrKrwrryrTrrhrrPr rr"rr5r8rFrr_typer) rrCrrrzsizeqpathsnapsrr6rrr*s ` @r9rzSessionDB.__init__rsB4"2"4"4 "^%% %?,, 7:ee ) 0 0#(  /4+-2)!"' %"'',$ $)77!*!4Y^5E5E!F!F@D!"'"'D 0 12DL222"&,&+$(  *1 &*4: KKKK!Z..00F--fnEEM%( 11 & 6 $ $/%'+z4$D$* $  L  % %dT % B B B  L( KKKK 5L'')) 5't|44 5  L--//7EEEEE24<@@ +.??MeMM$)$Y-YMM$)MMM S!!!$S)))=T\%8%8%:%:=?QRVR^?_?_=!/444 $ $ $ $ $,$ $ $ $ $ $ L 74466666( 7 7 7-S119Nt|9\9\ <=@z- ((*** D/ ==zz*--44666666666- 7>    !DII$6!?!?#!?!? @ @ @  sAP#A0G P# H &G;:H ; HH HH  P#A(P#;JP# J)&P#(J))B+P# M!!P 0?P0 OP OPO7PP#P  P## Q-'QQc|jr|jrdSt|jdd}||St|jddrdS t d|jd|jddd }t j|_t|d |j rt||j 5|j r.|d|j_ ddddS|j|dddn #1swxYwYnD#t j$r2d|j_t&d |jd YdSwxYw||j_|S)uwPer-thread read-only connection, or None when unavailable. Only used under WAL: WAL readers see a consistent snapshot and never block on (or get blocked by) the writer, so recall/browse queries can skip self._lock entirely. Under DELETE journal mode (NFS fallback) a reader can hit SQLITE_BUSY storms during writes, so we keep the legacy locked single-connection path there. Fresh read transactions begin per statement (autocommit), so each query observes everything committed so far — read-your-writes holds for the flush-then-search patterns in a turn. NrfailedFr[rTr)rrr r|rr#z'read-only connection open failed for %sr)r&rr>r;rrCrr$r%r0r'rr=r>rrSr<rrrrrrrs r9_get_read_connzSessionDB._get_read_connjs 4> 4t'66  K 4#Xu 5 5 4 &. ..."l $ D '{D  "4* = = = = # .'---& + +* JJLLL.2D$+  + + + + + + + + $$T*** + + + + + + + + + + + + + + +}   '+D  # LLBDL[_L ` ` `44   !% sCA!D %)D D D5 D DD D D =E Ec#K|}||VdS|j5|jVddddS#1swxYwYdS)uYield a connection for read-only statements. WAL: a per-thread read-only connection with NO lock — recall queries never convoy behind writer flushes (the gateway shares one SessionDB across every agent, so this lock was a global choke point). Non-WAL or read-conn failure: the shared writer connection under self._lock, byte-for-byte the legacy behavior. N)rUr9r#rTs r9 _read_ctxzSessionDB._read_ctxs""$$  JJJ F Z  *                     s =AArctt|}d|vrd|vrdSd|vrdSd|vrdSdS)Nzno such modulefts5Tno such tokenizer: trigramrFr4rr5s r9rz$SessionDB._is_fts5_unavailable_errorsX#hhnn s " "v}}4 (3 . .4 . 4 44ur;cTt|}d|vpd|vS)uTrue when only an optional tokenizer is missing (FTS5 itself works). Covers the built-in trigram tokenizer (needs SQLite >= 3.34) and the loadable cjk_unicode61 tokenizer — both mean "this one index can't be served here", never "disable FTS". rZrr4r[s r9_is_trigram_unavailable_errorz'SessionDB._is_trigram_unavailable_errors3#hhnn (C / 91S8 r;rc|d}|dSt|tjs|dn|dpd}d|vS)uTrue when messages_fts exists in ANY pre-v23 shape. v23's messages_fts is external-content over THREE real columns (content, tool_name, tool_calls). Every pre-v23 shape lacks the tool_name/tool_calls columns — whether the old inline single-column form (v11..v22) or the even older external-content single-column form (v10-era, pre-#16751). We therefore detect "needs optimize" as "the stored CREATE lacks the tool_name column", which is the precise v23 marker and correctly catches BOTH legacy variants. Returns False when messages_fts doesn't exist yet (fresh DB mid-init): the post-migration FTS setup block will create it in the v23 shape. zLSELECT sql FROM sqlite_master WHERE type = 'table' AND name = 'messages_fts'NFrsqlr> tool_name)rrrWrr$)rr[r_s r9_db_has_legacy_inline_ftsz#SessionDB._db_has_legacy_inline_ftssjnn =   (**  ;5'W[99Is1vvs5zPb#%%r;ct|ddrdSd|_td|jt j|dS)zHLog once that the trigram tokenizer is missing; base FTS5 stays enabled._trigram_unavailable_warnedFNTzSQLite trigram tokenizer unavailable for %s (requires SQLite >= 3.34, this build is %s); CJK/substring search will fall back to LIKE: %s)r>rcrrrCrrrrs r9_warn_trigram_unavailablez#SessionDB._warn_trigram_unavailablesY 46 > >  F+/(  > L  "       r;cvd|_|jrdSd|_td|j|dS)NFTzSQLite FTS5 unavailable for %s; full-text session search disabled. Run `hermes update` to rebuild the venv with a current Python (managed uv guarantees FTS5). (underlying error: %s))rBrErrrCrds r9_warn_fts5_unavailablez SessionDB._warn_fts5_unavailablesQ!  '  F'+$ % L       r;ct|d}|js|rd|dddt Ddt D}|r`tdt|dtf|D]}|d |d |_ d S | t|s|d tf|d d}|dkr^|dd}dt|fdfD]\}}|d||f|dtf} | r d |_ d S| t|d} | |_ d S#t j$r'tddd |_ Yd SwxYw)uCreate / repair / self-heal the CJK-bigram index surface. ``cursor`` may be a Cursor or a Connection (both expose execute / executescript). Called only for v23-shape DBs with the base FTS surface healthy. Sets ``self._fts_cjk_available``. Never raises; every failure mode degrades to "no cjk index" (trigram/LIKE routing keeps working). Cases: tokenizer loaded, table absent → create. Empty DB: index is complete by construction (triggers cover everything). Populated DB: set the cjk backfill markers so the id-gated triggers stay correct and `optimize-storage` can backfill; the index is NOT served until the backfill completes. tokenizer loaded, table present → ensure triggers (recreates any dropped by a tokenizer-less process), honour the stale breadcrumb (serve only when absent and no backfill pending). tokenizer NOT loaded, table present with live triggers → drop the cjk triggers so message INSERTs don't fail at trigger time, and leave the stale breadcrumb (#self-heal). The table itself stays for a later capable open to rebuild. zNSELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'messages_fts_cjk'cg|] }|d Srrfrs r9rz4SessionDB._ensure_fts_cjk_schema..$ s)AaDr;zCSELECT name FROM sqlite_master WHERE type = 'trigger' AND name IN (r}c3K|]}dVdSr~Nrfrzrs r9rz3SessionDB._ensure_fts_cjk_schema..' s"0P0P0P0P0P0P0P0Pr;ru messages_fts_cjk triggers present but the cjk_unicode61 tokenizer is unavailable (%s) — dropping the cjk triggers so message writes keep working. CJK search falls back to trigram/LIKE; run `hermes sessions optimize-storage` on a host with the extension to rebuild.z^INSERT INTO state_meta (key, value) VALUES (?, '1') ON CONFLICT(key) DO UPDATE SET value = '1'DROP TRIGGER IF EXISTS FNz$DELETE FROM state_meta WHERE key = ?z2SELECT COUNT(*) FROM messages WHERE role <> 'tool'rz)SELECT COALESCE(MAX(id), 0) FROM messagesfts_cjk_rebuild_high_water)fts_cjk_rebuild_progressrgINSERT INTO state_meta (key, value) VALUES (?, ?) ON CONFLICT(key) DO UPDATE SET value = excluded.valuez&SELECT 1 FROM state_meta WHERE key = ?zISELECT 1 FROM state_meta WHERE key = 'fts_cjk_rebuild_high_water' LIMIT 1z@messages_fts_cjk ensure failed; CJK search stays on trigram/LIKETr)rvrrr'rrrrrrr!rD executescriptFTS_CJK_TABLE_SQLrXFTS_CJK_TRIGGER_SQLrr) rr cjk_presentlivetrign_msgshwkvstalebackfill_pendings r9_ensure_fts_cjk_schemaz SessionDB._ensure_fts_cjk_schema s'.6>> ,   (** #  I"(..T(+0P0P>O0P0P0P(P(PTTT)##hjj I NN9 )**NNE*, !%II'G'G'GHHHH&+D # F7 ,  !2 3 3 3 :&( H(**Q A::Chjj$B6s2ww?9!1TF NN8"$hjj   +0'  !4 5 5 5%~~C  hjj +;&:D # # #' , , , NN)-    ',D # # # # ,sDI A I2J  J cvtD]0} |d|#tj$rY-wxYwdS)Nrn)rrrr)rtriggers r9_drop_fts_triggerszSessionDB._drop_fts_triggers| sa$  G BBBCCCC+      s $66rddlcL|||}|dS ||dS#tj$ra}||s||r||n||Yd}~dSd}~wwxYw)NFT)rLrrrrrr]rerg)rrrrstatusrs r9_ensure_fts_schemazSessionDB._ensure_fts_schema s &&vz:: >5    % % %4'   22377 11#66 1..s3333++C00055555 s3B#ABB#fn patience_sct||j}tj|z}d}dtjdt fd} |j5|jd ||j}|j n:#t$r- |j n#t$rYnwxYwwxYw dddn #1swxYwY|xj dz c_ |j |jzdkr||j |jzdkr||S#t$$rK|)t'tj|jz|}|||jrYTtj$r}t/|}d |vsd |vr6|||rYd}~tjd |d d |||r|||rYd}~d}~wtj$rH}||r|||rYd}~#||sYd}~?d}~wtj$r-}||r|||rYd}~xd}~wwxYw)uExecute a write transaction with BEGIN IMMEDIATE and jitter retry. *fn* receives the connection and should perform INSERT/UPDATE/DELETE statements. The caller must NOT call ``commit()`` — that's handled here after *fn* returns. BEGIN IMMEDIATE acquires the WAL write lock at transaction start (not at commit time), so lock contention surfaces immediately. On ``database is locked``, we release the Python lock, sleep a random jitter, and retry — breaking the convoy pattern that SQLite's built-in deterministic backoff creates. *patience_s* is the total time budget for lock retries (default ``_WRITE_PATIENCE_S``). Transcript-critical writes pass ``_TRANSCRIPT_WRITE_PATIENCE_S`` so a sibling process holding the lock for a legitimate long operation (VACUUM, TRUNCATE checkpoint, pre-bounded-merge FTS optimize from an older still-running install) exhausts routine writers' patience without destroying a user turn. Jitter starts small (20-150ms) for fast reclaim on millisecond contention and backs off to 250ms-1s once the lock has been held longer than ``_WRITE_RETRY_SLOW_AFTER_S``. Returns whatever *fn* returns. Nrr3cHdt|vS)Nzno more rows availabler4r7s r9_is_no_more_rowsz2SessionDB._execute_write.._is_no_more_rows s+s3xx~~/?/?? ?r;Trr?rr,r-zQdatabase is locked (another Hermes process held the state.db write lock for over z.0fus — likely a long maintenance operation such as VACUUM, a large WAL checkpoint, or an older pre-update process; the database itself is healthy))r/r rrrrvr9r#rrrMrollbackrKr?_CHECKPOINT_EVERY_N_WRITES_try_wal_checkpoint_FTS_MERGE_EVERY_N_WRITES_try_incremental_merge_ftsrr0_COMPRESSION_BUSY_WAIT_S_sleep_before_write_retryrrXrr5_try_runtime_fts_rebuild) rrrrcompression_deadlinerresultrerr_msgs r9_execute_writezSessionDB._execute_write s:  /J>##j015 @'- @D @ @ @ @R Q Z  J&&'8999!#DJ ))++++(! J//1111(!!! D! ,               !!Q&!!$t'FF!KK,,...$t'EEJJ33555 4   (/+.((4+HH(,,(11($*GH+   c((..**w&&&G*;*;55h KK! "2C8BNCCC  $#C((T-K-KHV`-a-aHHHH(   ##C((T-K-KHV`-a-aHHHH44S99=    $#C((T-K-KHV`-a-aHHHH sD:C)BC B?B-,B?- B:7B?9B::B??C D:CD:CA"D::AJ7 J7?H#!;H#"H##J75!I8I88J7 !J21J22J7rc Ftj}||krdS|||z z }||jkr tj|j|j}ntj|j|j}tj t|t||z ddS)aSleep one jitter interval if the patience budget still allows it. Returns True when the caller should retry, False when *deadline* has passed and the error should propagate. Jitter stays small for the first ``_WRITE_RETRY_SLOW_AFTER_S`` (fast reclaim on millisecond contention) and backs off after that, and never overshoots the deadline by a full slow-jitter. Fr.T) r r_WRITE_RETRY_SLOW_AFTER_Sr1r2r3r4_WRITE_RETRY_MIN_S_WRITE_RETRY_MAX_Sr r0r)rrrrPelapsedjitters r9rz#SessionDB._sleep_before_write_retry! sn (??5J./ d4 4 4^,,FF ^''F 3vs8c>599::;;;tr;cvt|rdSt|}d|vod|vS)aTrue for the error class a corrupt FTS index raises on writes. The message varies by SQLite version: older builds raise the generic ``database disk image is malformed`` (covered by ``is_malformed_db_error``); newer builds (e.g. ubuntu-latest CI) raise the FTS5-specific ``fts5: corrupt structure record for table "messages_fts"``. Both mean the same thing for the write path: the canonical rows are fine, the FTS shadow tables are not. TrYcorrupt)r8rXr)rrs r9_is_fts_write_corruption_errorz(SessionDB._is_fts_write_corruption_error= sB ! % % 4#hhnn}1c!11r;c|jrdS|jsdS||sdSd|_td| |}n3#t $r&}td|Yd}~dSd}~wwxYw|stddStd|dS)ubOne-shot in-place FTS rebuild after a corrupt-index write failure. Returns True when a rebuild was performed and the failed write should be retried; False when the error isn't the FTS-corruption class, FTS is disabled, or a rebuild was already attempted for this instance. Delegates to :meth:`rebuild_fts` (the FTS5 ``'rebuild'`` command — index rewritten from the canonical messages table, zero message-row mutation). Safe to call from ``_execute_write``'s except path: the failed transaction was rolled back and ``self._lock`` released before the exception propagated, and ``rebuild_fts`` re-acquires it. E2E-verified: a corrupted ``messages_fts_data`` shadow table rejects every append; after the in-place rebuild the same append succeeds and search works again. FTustate.db write failed with an FTS-corruption error (%s) — attempting one-shot in-place FTS rebuild; canonical message rows are preserved.zkIn-place FTS rebuild failed (%s); the database needs the full offline repair path (repair_state_db_schema).NzpIn-place FTS rebuild made no progress; the database needs the full offline repair path (repair_state_db_schema).zFstate.db FTS indexes rebuilt in place (%d); retrying the failed write.)r@rBrrrrrKr")rrrebuilt rebuild_excs r9rz"SessionDB._try_runtime_fts_rebuildM s  . 5  522377 5.2+ "#&    &&((GG    LLE    55555    LLI   5 T    ts A"" B,B  Bcr |j5|jd}|r4|ddkr(td|d|dddddS#1swxYwYdS#t $r&}td|Yd}~dSd}~wwxYw)u Best-effort PASSIVE WAL checkpoint. Never raises. Flushes committed WAL frames back into the main DB file without requiring an exclusive lock. PASSIVE is safe for frequent periodic use because it does not block concurrent writers and cannot corrupt B-tree pages under I/O pressure. PASSIVE does not truncate the WAL file — it stays at its high-water mark. WAL truncation happens in :meth:`close` (TRUNCATE) and pre-VACUUM checkpoints, which run infrequently under controlled conditions. Previous TRUNCATE strategy caused B-tree corruption on large databases (65K+ pages) due to the exclusive-lock I/O pressure from checkpointing thousands of frames at once (issue #45383). zPRAGMA wal_checkpoint(PASSIVE)r?rz(WAL checkpoint: %d/%d pages checkpointedrNz#WAL checkpoint (PASSIVE) failed: %s)r9r#rrrrrKr)rrrs r9rzSessionDB._try_wal_checkpoint~ s"" G  ++4(**fQi!mmLLBq 6!9                    G G G NN@# F F F F F F F F F Gs;BA#A9, B9A==BA=B B6B11B6c|tj|j|j5d|_t |j}|jdddn #1swxYwY|D]'} | #t$rY$wxYwd|j _ |j 5|jru|jsN |jdn2#t$r%}t"d|Yd}~nd}~wwxYw|j d|_ddddS#1swxYwYdS)a;Close the database connection. Drains queued token deltas first (the background writer needs the connection). Writable connections then attempt a TRUNCATE WAL checkpoint so exiting writer processes help shrink the WAL file. Read-only connections never request a checkpoint. TNPRAGMA wal_checkpoint(TRUNCATE)z-WAL checkpoint (TRUNCATE) at close failed: %s)_stop_token_writeratexit unregister_drain_token_queue_at_exitr=r>rr<clearrrKr;rr9r#rrrr)r read_connsrrs r9rzSessionDB.close s !!! $9::: " % %&*D #d.//J   " " $ $ $ % % % % % % % % % % % % % % %  D       $ Z " "z "~ **+LMMMM$ K    """!  " " " " " " " " " " " " " " " " " "s_5A66A:=A:B B('B(>EC)(E) D3DED#EE E g@g?fts_v22_trash_c t|d|jdddzfS)zTrue when demoted v22 shadow tables are still awaiting teardown. Caller must hold ``self._lock`` (or pass a migration-time cursor).zSSELECT 1 FROM sqlite_master WHERE type = 'table' AND name LIKE ? ESCAPE '\' LIMIT 1r\_%)rvr_FTS_TRASH_PREFIXreplacerrTs r9_has_fts_trashzSessionDB._has_fts_trash sTDLL 2  # + +C 7 7# = ?   (**  r;rsourcemodelrcuser_id session_keychat_id chat_type thread_idparent_session_idr^ profile_namer]c    fd}|jdS)uInsert a session row, enriching NULL metadata on conflict. The gateway's ``get_or_create_session`` creates a bare row (source + user_id) *before* the agent exists; the agent's later ``create_session`` then carries the real ``model`` / ``model_config`` / ``system_prompt``. A plain ``INSERT OR IGNORE`` silently dropped that enrichment, leaving gateway sessions with NULL model/billing metadata. The ``ON CONFLICT`` upsert backfills those fields via ``COALESCE`` — only filling columns that are still NULL, never overwriting values an earlier writer already set (so a later bare call with source="unknown" can't clobber a real source/model). ``chat_id``/``thread_id`` record the messaging origin (the chat/room and thread the session was started in) so that gateway ``/resume`` can prove a persisted, now-inactive row belongs to the caller's chat/thread before switching to it (IDOR scoping — without them the ``sessions`` table has no chat/thread to compare). When ``parent_session_id`` is set (compression fork, delegate/subagent spawn, branch continuation) and this row's own ``cwd``/``git_repo_root``/ ``git_branch``/``profile_name`` are still NULL after the insert, they are backfilled from the parent row. Callers of ``create_session`` for a child session historically didn't propagate these fields themselves (e.g. the compression-fork path), so a lineage could silently lose its working directory and drop out of the project sidebar every time it forked (#64709), or lose its owning profile and be aggregated as "default" every time it rotated or branched (the cross-profile session-jump bug). This only fills NULLs — an explicit value on the child is never overwritten. For compression forks specifically (parent ended with ``end_reason='compression'``), the gateway origin columns (``user_id``/``session_key``/``chat_id``/``chat_type``/ ``thread_id``/``display_name``/``origin_json``) are inherited too, so a crash before the gateway re-records the peer can't strand the child without a recoverable routing mapping (#59527). c^ |}|d  rtjnd| t jf| |r0|d f|d fdSdS)NaINSERT INTO sessions ( id, source, user_id, session_key, chat_id, chat_type, thread_id, model, model_config, system_prompt, system_prompt_hash, parent_session_id, cwd, profile_name, git_repo_root, started_at ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, NULL, ?, ?, ?, ?, ?, ?) ON CONFLICT(id) DO UPDATE SET model = COALESCE(sessions.model, excluded.model), model_config = COALESCE(sessions.model_config, excluded.model_config), system_prompt_hash = COALESCE( sessions.system_prompt_hash, excluded.system_prompt_hash ), system_prompt = CASE WHEN sessions.system_prompt_hash IS NULL AND excluded.system_prompt_hash IS NOT NULL THEN NULL ELSE sessions.system_prompt END, session_key = COALESCE(sessions.session_key, excluded.session_key), chat_id = COALESCE(sessions.chat_id, excluded.chat_id), chat_type = COALESCE(sessions.chat_type, excluded.chat_type), thread_id = COALESCE(sessions.thread_id, excluded.thread_id), parent_session_id = COALESCE(sessions.parent_session_id, excluded.parent_session_id), cwd = COALESCE(sessions.cwd, excluded.cwd), profile_name = COALESCE(sessions.profile_name, excluded.profile_name), git_repo_root = COALESCE(sessions.git_repo_root, excluded.git_repo_root)aUPDATE sessions SET cwd = COALESCE(sessions.cwd, (SELECT p.cwd FROM sessions p WHERE p.id = sessions.parent_session_id)), git_repo_root = COALESCE(sessions.git_repo_root, (SELECT p.git_repo_root FROM sessions p WHERE p.id = sessions.parent_session_id)), git_branch = COALESCE(sessions.git_branch, (SELECT p.git_branch FROM sessions p WHERE p.id = sessions.parent_session_id)), profile_name = COALESCE(sessions.profile_name, (SELECT p.profile_name FROM sessions p WHERE p.id = sessions.parent_session_id)) WHERE id = ? AND parent_session_id IS NOT NULLaUPDATE sessions SET user_id = COALESCE(sessions.user_id, (SELECT p.user_id FROM sessions p WHERE p.id = sessions.parent_session_id)), session_key = COALESCE(sessions.session_key, (SELECT p.session_key FROM sessions p WHERE p.id = sessions.parent_session_id)), chat_id = COALESCE(sessions.chat_id, (SELECT p.chat_id FROM sessions p WHERE p.id = sessions.parent_session_id)), chat_type = COALESCE(sessions.chat_type, (SELECT p.chat_type FROM sessions p WHERE p.id = sessions.parent_session_id)), thread_id = COALESCE(sessions.thread_id, (SELECT p.thread_id FROM sessions p WHERE p.id = sessions.parent_session_id)), display_name = COALESCE(sessions.display_name, (SELECT p.display_name FROM sessions p WHERE p.id = sessions.parent_session_id)), origin_json = COALESCE(sessions.origin_json, (SELECT p.origin_json FROM sessions p WHERE p.id = sessions.parent_session_id)) WHERE id = ? AND parent_session_id IS NOT NULL AND EXISTS ( SELECT 1 FROM sessions p WHERE p.id = sessions.parent_session_id AND p.end_reason = 'compression' ))rrjsondumpsr r)rsystem_prompt_hashrrr^r]rrcrrrrrrr2rrs r9_doz*SessionDB._insert_session_row.._doD s!%!:!:4!O!O  LLc80<FDJ|,,,$&% !IKK9- - - \"-88>>> ;   G M8 8 M;;; ; r;rN)r_TRANSCRIPT_WRITE_PATIENCE_S)rrrrrcr2rrrrrrr^rr]rs``````````````` r9_insert_session_rowzSessionDB._insert_session_row shm m m m m m m m m m m m m m m m m m m b CD,MNNNNNr;c $|j||fi||S)z4Create a new session record. Returns the session_id.r)rrrrs r9create_sessionzSessionDB.create_session s%  V>>v>>>r;)rrrrr display_name origin_jsoninclude_compression_ancestorsrrrc l rsdS f d} || dS)aPersist the gateway routing peer for an existing session row. ``display_name`` / ``origin_json`` carry the gateway's presentation and full origin metadata (#9006) so consumers (mcp_serve, mirror, channel directory) can read routing data from state.db instead of sessions.json. They are COALESCE'd only in the sense that ``None`` leaves the existing value untouched. ``include_compression_ancestors`` keeps a logical compression lineage on one routing peer when an explicit gateway resume moves its tip to a different lane. Normal per-turn metadata refreshes update only the supplied row. Nc  d}d}g}rd}d}| |  fs| ||d||dS)Nr>z WHERE id = ?a WITH RECURSIVE compression_lineage(id) AS ( SELECT ? UNION SELECT parent.id FROM compression_lineage lineage JOIN sessions child ON child.id = lineage.id JOIN sessions parent ON parent.id = child.parent_session_id WHERE parent.end_reason = 'compression' AND json_extract( COALESCE(child.model_config, '{}'), '$._branched_from' ) IS NULL AND json_extract( COALESCE(child.model_config, '{}'), '$._delegate_from' ) IS NULL AND COALESCE(child.source, '') != 'tool' ) z0WHERE id IN (SELECT id FROM compression_lineage)a9 UPDATE sessions SET session_key = ?, source = ?, user_id = ?, chat_id = ?, chat_type = ?, thread_id = ?, display_name = COALESCE(?, display_name), origin_json = COALESCE(?, origin_json) )rextendr)r lineage_cte target_clause query_paramsrrrrrrrrrrs r9rz2SessionDB.record_gateway_session_peer.._do sK*ML, 0 (!S ##J///        1 0##J/// LL && " &&     r;r) rrrrrrrrrrrrs `````````` r9record_gateway_session_peerz%SessionDB.record_gateway_session_peer s6   F2 2 2 2 2 2 2 2 2 2 2 2 2 2 h C     r;T finalizedcHsdSfd}||dS)zMark a gateway session's expiry-finalization flag in state.db. Mirrors ``SessionEntry.expiry_finalized`` (sessions.json) so the flag survives even if the JSON index is pruned or lost (#9006). Nc@|drdndfdS)Nz5UPDATE sessions SET expiry_finalized = ? WHERE id = ?r?rr)rrrs r9rz+SessionDB.set_expiry_finalized.._do s7 LLG&Q 3     r;r)rrrrs `` r9set_expiry_finalizedzSessionDB.set_expiry_finalized sM   F       C     r;r>)scope entry_jsonrcPrsdSfd}||dS)u Upsert one gateway routing entry (session_key -> SessionEntry JSON). The gateway_routing table is the durable replacement for sessions.json: one row per routing key, holding the full serialized ``SessionEntry`` so the gateway can rehydrate exactly what it wrote. ``scope`` namespaces the index the way separate sessions.json files did (one per sessions_dir) — callers pass their sessions_dir path so two stores with different directories never share routing state. Nc^|dtjfdS)Na!INSERT INTO gateway_routing (scope, session_key, entry_json, updated_at) VALUES (?, ?, ?, ?) ON CONFLICT(scope, session_key) DO UPDATE SET entry_json = excluded.entry_json, updated_at = excluded.updated_atrr )rrrrs r9rz1SessionDB.save_gateway_routing_entry.._do3 s< LL;  Z=      r;r)rrrrrs ``` r9save_gateway_routing_entryz$SessionDB.save_gateway_routing_entry# sZ *  F        C     r;entriescjtjfd}||dS)a7Atomically replace the routing index for *scope* with *entries*. Mirrors the sessions.json full-rewrite semantics: keys absent from *entries* are removed (pruned/reset sessions disappear from the index). Runs as a single write transaction. Other scopes are untouched. c|dfr7|dfdDdSdS)Nz+DELETE FROM gateway_routing WHERE scope = ?z\INSERT INTO gateway_routing (scope, session_key, entry_json, updated_at) VALUES (?, ?, ?, ?)c*g|]\}}|| ||fSrfrf)rzrzr{rPrs r9rzJSessionDB.replace_gateway_routing_entries.._do..Q s0OOODAqqOQOeQ3'OOOr;)r executemanyitems)rrrPrs r9rz6SessionDB.replace_gateway_routing_entries.._doK su LLF Q Q Q   *OOOOOGMMOOOOO  r;Nr r)rrrrrPs `` @r9replace_gateway_routing_entriesz)SessionDB.replace_gateway_routing_entries? sQikk        C     r;c|j5|jd|f}dddn #1swxYwYd|DS)z>Load routing entries for *scope* as {session_key: entry_json}.zCSELECT session_key, entry_json FROM gateway_routing WHERE scope = ?Nc,i|]}|d|dS)rrrfrs r9 z:SessionDB.load_gateway_routing_entries..] s#@@@a- !L/@@@r;)r9r#rr)rrrs r9load_gateway_routing_entriesz&SessionDB.load_gateway_routing_entriesV s Z  :%%Uhjj                 A@4@@@@/AA A session_keyscHsdSfd}||dS)z=Remove routing entries for the given session keys in *scope*.NcL|dfdDdS)Nz?DELETE FROM gateway_routing WHERE scope = ? AND session_key = ?cg|]}|fSrfrf)rzrzrs r9rzISessionDB.delete_gateway_routing_entries.._do..i s222%222r;)r)rrrs r9rz5SessionDB.delete_gateway_routing_entries.._dof s?   Q2222\222     r;r)rrrrs `` r9delete_gateway_routing_entriesz(SessionDB.delete_gateway_routing_entries_ sM  F       C     r;)r active_onlyrrcZdtdd}g}|r|dz }|||r|dz }|dz }j5j||}dddn #1swxYwYfd|DS) uFList gateway sessions (rows with a session_key) from state.db. Returns the newest row per session_key — the same shape consumers got from sessions.json: one live mapping per routing key. ``platform`` filters on ``source``; ``active_only`` restricts to sessions that have not ended. z SELECT sessions.*, COALESCE(sp.prompt, sessions.system_prompt) AS _system_prompt_resolved, sessionsag AS last_active FROM sessions LEFT JOIN system_prompts sp ON sp.hash = sessions.system_prompt_hash WHERE session_key IS NOT NULL AND started_at = ( SELECT MAX(s2.started_at) FROM sessions s2 WHERE s2.session_key = sessions.session_key ) z AND LOWER(source) = LOWER(?)z AND ended_at IS NULLz ORDER BY last_active DESCNc:g|]}|Srfrrzrrs r9rz3SessionDB.list_gateway_sessions.. s'888a&&q))888r;)flush_token_countsrrr9r#rr)rrrqueryparamsrs` r9list_gateway_sessionszSessionDB.list_gateway_sessionsn s/ !!!  -Z88      $ 4 4E MM( # # #  - , ,E -- Z @ @:%%eV44==??D @ @ @ @ @ @ @ @ @ @ @ @ @ @ @888848888s.BBB)rrcn|r|dvrdSd}|t|g}|'|dz }|t||dz }|j5d|j||D}dddn #1swxYwY|sdSrAfd|D}|rt|dd St |d krdSn4t |d kr!d |D} t | d krdSt|dd S) aFind the most recent live session_id for a platform + chat origin. Equivalent of gateway/mirror's sessions.json scan: matches on source + chat_id (+ thread_id when provided). When ``user_id`` is provided, exact sender matches are preferred; if multiple distinct users share the chat and none matches, returns None rather than contaminating another participant's session. )Nr>Nz SELECT id, user_id, started_at FROM sessions WHERE LOWER(source) = LOWER(?) AND session_key IS NOT NULL AND chat_id = ? AND ended_at IS NULL z AND COALESCE(thread_id, '') = ?z ORDER BY started_at DESCc,g|]}t|Srfrrs r9rz4SessionDB.find_session_by_origin.. sRRRDGGRRRr;c~g|]9}t|dpdtk7|:Srr>)rXr_)rzrrs r9rz4SessionDB.find_session_by_origin.. sBTTT1AEE),<,<,B(C(Cs7||(S(SQ(S(S(Sr;rrr?ch|]n}t|dpd8t|dpdoSr)rXr_r`rs r9r|z3SessionDB.find_session_by_origin.. sxquuY''-2..4466AEE)$$*++1133r;)rXrr9r#rrr) rrrrrrrrexactdistinct_userss ` r9find_session_by_originz SessionDB.find_session_by_origin s  7j004 !#g,,/  7 7E MM#i.. ) ) ) ,, Z S SRRTZ%7%7v%F%F%O%O%Q%QRRRD S S S S S S S S S S S S S S S 4  TTTTTTTE +58D>***4yy1}}t YY]]N >""Q&&t474=!!!s8BBB)rrrrrc |sdS|j5|jd||f}|!||cdddS|| ddddS|jd|||||f}dddn #1swxYwY|r||ndS)aFind the latest recoverable gateway session for a routing peer. ``sessions.json`` is the fast routing index, but it can be missing or pruned after process-level restart bugs. New gateway sessions persist the deterministic ``session_key`` on the durable session row so the mapping can be rebuilt exactly. Rows ended only by older gateway cleanup's ``agent_close`` bug or a mistaken TUI ``ws_orphan_reap`` (dashboard viewer disconnect before #60609) are treated as recoverable; explicit conversation boundaries such as /new, /resume switches, and compression splits are not. Na SELECT s.*, COALESCE(sp.prompt, s.system_prompt) AS _system_prompt_resolved FROM sessions s LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hash WHERE s.session_key = ? AND s.source = ? AND (s.ended_at IS NULL OR s.end_reason IN ('agent_close', 'ws_orphan_reap')) AND (COALESCE(s.message_count, 0) > 0 OR EXISTS ( SELECT 1 FROM messages WHERE messages.session_id = s.id LIMIT 1 )) ORDER BY s.started_at DESC LIMIT 1 a SELECT s.*, COALESCE(sp.prompt, s.system_prompt) AS _system_prompt_resolved FROM sessions s LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hash WHERE s.source = ? AND COALESCE(s.user_id, '') = COALESCE(?, '') AND COALESCE(s.chat_id, '') = COALESCE(?, '') AND COALESCE(s.chat_type, '') = COALESCE(?, '') AND COALESCE(s.thread_id, '') = COALESCE(?, '') AND (s.ended_at IS NULL OR s.end_reason IN ('agent_close', 'ws_orphan_reap')) AND (COALESCE(s.message_count, 0) > 0 OR EXISTS ( SELECT 1 FROM messages WHERE messages.session_id = s.id LIMIT 1 )) ORDER BY s.started_at DESC LIMIT 1 )r9r#rrr)rrrrrrrr[s r9$find_latest_gateway_session_for_peerz.SessionDB.find_latest_gateway_session_for_peer sv* 4 Z/ / *$$f%!"hjj# $--c22)/ / / / / / / / 2)"35/ / / / / / / / 6*$$$'9i@'(hjj) 7/ / / / / / / / / / / / / / / `/2;t%%c***t;sAB/B/12B//B36B3c|sdS|j5|jd|f}||d |ddkr ddddS|jd|f}dddn #1swxYwYt |dkr||dndS) aReturn the unique live direct child of a compression-ended session. A stale agent may observe that another compression path already rotated its parent. Recovery is safe only when the durable lineage identifies exactly one live direct continuation. Multiple children are treated as ambiguous and fail closed rather than guessing which transcript owns subsequent messages. N6SELECT ended_at, end_reason FROM sessions WHERE id = ?ended_at end_reason compressiona SELECT s.*, COALESCE(sp.prompt, s.system_prompt) AS _system_prompt_resolved FROM sessions s LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hash WHERE s.parent_session_id = ? AND s.ended_at IS NULL AND json_extract(COALESCE(s.model_config, '{}'), '$._branched_from') IS NULL AND json_extract(COALESCE(s.model_config, '{}'), '$._delegate_from') IS NULL AND COALESCE(s.source, '') != 'tool' ORDER BY s.started_at ASC LIMIT 2 r?r)r9r#rrrrr)rrrwrs r9find_live_compression_childz%SessionDB.find_live_compression_child sJ! 4 Z  Z''H"$hjj  *%-,'=88        :%% #$ hjj!                836d))q..t%%d1g...dJsAB.BB B)rrcr2r^rcompression_lock_holderrequire_compression_leasechild_session_idrr r c h   f d} | dS)a:Atomically close a parent and publish its durable compression child. The parent closure, child row, and compacted handoff become visible in one transaction. Readers can therefore observe either the live parent or a complete child, never an ended parent with a missing/empty child. c |d f}rL|8r6|dks*t|dtjkrt d |d f}|t d |dt d  st d |}|d  rtj nd| p|d |d |d p|d|d|d|d|d|d|d|dtjf | \}}|d||f|dtj f}|j dkrt d dS)NESELECT holder, expires_at FROM compression_locks WHERE session_id = ?r< expires_atz+Compression lease lost before publication: zSELECT ended_at, cwd, git_branch, git_repo_root, user_id, session_key, chat_id, chat_type, thread_id, display_name, origin_json, profile_name FROM sessions WHERE id = ?zCompression parent not found: rz"Compression parent already ended: z+Compression child handoff must not be emptyaINSERT INTO sessions ( id, source, model, model_config, system_prompt, system_prompt_hash, parent_session_id, cwd, git_branch, git_repo_root, profile_name, user_id, session_key, chat_id, chat_type, thread_id, display_name, origin_json, started_at ) VALUES (?, ?, ?, ?, NULL, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)r^ git_branchr]rrrrrrrrGUPDATE sessions SET message_count = ?, tool_call_count = ? WHERE id = ?z^UPDATE sessions SET ended_at = ?, end_reason = 'compression' WHERE id = ? AND ended_at IS NULLr?z/Compression parent changed during publication: ) rrfloatr r RuntimeErrorrrr_insert_message_rowsrowcount)rlock_rowrwrtotal_messagestotal_tool_callsupdatedr r r^rrrcrrr rrr2s r9rz0SessionDB.publish_compression_child.._doR s||W"$hjj )  .!H%)@@@,/00DIKK??1UBSUU\\1#$  hjj ~"#WDU#W#WXXXj!-"#[HY#[#[\\\ R"#PQQQ!%!:!:4!O!O  LLY%0<FDJ|,,,$&%(6%=<(?+ !:F>$:9%=)9%;';'>*=)IKK/! ! ! D04/H/H&00 ,N, LLY!13CD   ll4/0G 1$$"YFWYY%$r;Nr) rrr rrrrcr2r^rr r rs ```````````` r9publish_compression_childz#SessionDB.publish_compression_child= s*N N N N N N N N N N N N N N N N ` C     r;rc@fd}||dS)aMark a session as ended. No-ops when the session is already ended. The first end_reason wins: compression-split sessions must keep their ``end_reason = 'compression'`` record even if a later stale ``end_session()`` call (e.g. from a desynced CLI session_id after ``/resume`` or ``/branch``) targets them with a different reason. Use ``reopen_session()`` first if you intentionally need to re-end a closed session with a new reason. c\|dtjfdS)NzRUPDATE sessions SET ended_at = ?, end_reason = ? WHERE id = ? AND ended_at IS NULLr)rrrs r9rz"SessionDB.end_session.._do s8 LL4j*5     r;Nr)rrrrs `` r9 end_sessionzSessionDB.end_session s>       C     r;c<fd}||dS)z6Clear ended_at/end_reason so a session can be resumed.c6|dfdS)NzCUPDATE sessions SET ended_at = NULL, end_reason = NULL WHERE id = ?rrrs r9rz%SessionDB.reopen_session.._do s+ LLU      r;Nrrrrs ` r9reopen_sessionzSessionDB.reopen_session s8      C     r; session_resetrcsdStjfd} ||}t|S#t$rYdSwxYw)uDurably mark a session as ended by an intentional reset boundary. Promotes *only* live rows (``ended_at IS NULL``) or rows carrying an accidental end_reason that the recovery query (``find_latest_gateway_session_for_peer``) treats as recoverable: ``agent_close`` (older gateway cleanup bug) and ``ws_orphan_reap`` (mistaken TUI reaper). Explicit conversation boundaries such as ``compression``, ``session_reset``, ``session_switch``, etc. are preserved — the first writer wins for those, and a later expiry finalization must not silently overwrite them. Plain ``end_session()`` is NOT sufficient for reset boundaries: it no-ops on an already-ended row, so a row that agent cleanup already closed as ``agent_close`` would stay recoverable and stale-route recovery would resurrect the reset session with its full history (#61220, #61993, #63539). Keep this promotion set in sync with the recoverable set in ``find_latest_gateway_session_for_peer`` — any reason recovery would reopen must be promotable here. ``reason`` lets reset paths keep their auditable specific reasons (``idle``, ``daily``, ``suspended``, ``resume_pending_expired``). Returns ``True`` when the row was promoted, ``False`` when skipped (already has a different explicit end_reason, or row not found). FcD|df}|jS)NzUPDATE sessions SET ended_at = ?, end_reason = ? WHERE id = ? AND (ended_at IS NULL OR end_reason IN ('agent_close', 'ws_orphan_reap'))rr)rrrPrrs r9rz/SessionDB.promote_to_session_reset.._do s2\\Ffj) F ? "r;)r rrvrK)rrrrrrPs `` @r9promote_to_session_resetz"SessionDB.promote_to_session_reset s< 5ikk # # # # # # # &&s++D::    55 s#A AArc |r|sdS|pd}|pd}dg |g|r* d||r* d|| fd}||dS)u!Persist the session working directory when a frontend knows it. ``git_branch`` records the git branch checked out in ``cwd`` at the time the session started/resumed. The sidebar groups main-checkout sessions by this so feature-branch work doesn't pile under a single "main" row (the main checkout's *current* branch is transient and would misattribute past sessions). ``git_repo_root`` records the git repo this cwd belongs to — the authoritative project key. Resolving it here, at the lowest level, means every surface reads the same membership instead of re-probing git in the GUI over a partial page. Each field is only written when non-empty so a probe failure never clobbers a previously-captured value. Nr>zcwd = ?zgit_branch = ?zgit_repo_root = ?cb|ddddS)NzUPDATE sessions SET z, z WHERE id = ?)rr)rrsetss r9rz)SessionDB.update_session_cwd.._dos4 LLN $NNNPV W W W W Wr;)r`rr) rrr^rr]branch repo_rootrrr+s @@r9update_session_cwdzSessionDB.update_session_cwd s"   F"))++"(b//11 { E  " KK( ) ) ) MM& ! ! !  % KK+ , , , MM) $ $ $ j!!! X X X X X X C     r; cwd_to_rootcd|DsdSfd}||dS)a$Persist resolved git repo roots for cwds that don't have one yet. Backfills history so projects light up for sessions created before the column existed, without clobbering an already-recorded root. Only non-empty roots are written (a non-git cwd stays NULL). c$g|] \}}|| ||fSrfrf)rzr^ras r9rz1SessionDB.backfill_repo_roots..s*RRRdTRcR$RRRr;NcHD]\}}|d||fdS)NzXUPDATE sessions SET git_repo_root = ? WHERE cwd = ? AND COALESCE(git_repo_root, '') = ''r)rrar^pairss r9rz*SessionDB.backfill_repo_roots.._do!sH"   c I3K  r;)rr)rr/rr3s @r9backfill_repo_rootszSessionDB.backfill_repo_rootssgSRk.?.?.A.ARRR  F      C     r;cooldown_untilr"csdSfd} ||dS#tj$r'}td|Yd}~dSd}~wwxYw)z>Persist the active compression-failure cooldown for a session.Nc:|dfdS)NfUPDATE sessions SET compression_failure_cooldown_until = ?, compression_failure_error = ? WHERE id = ?r)rr5r"rs r9rz:SessionDB.record_compression_failure_cooldown.._do5s2 LL= 3     r;z2record_compression_failure_cooldown(%s) failed: %srrrrr)rrr5r"rrs ``` r9#record_compression_failure_cooldownz-SessionDB.record_compression_failure_cooldown+s  F            $ $ $ $ $}    NNDC          s'AAAc|sdStj}|j5|jd|f}dddn #1swxYwY|dSt |t jr|dn|d}|dSt|}||krdSt |t jr|dn|d}|||z |dS)zBReturn the active compression-failure cooldown for ``session_id``.N_SELECT compression_failure_cooldown_until, compression_failure_error FROM sessions WHERE id = ?"compression_failure_cooldown_untilrcompression_failure_errorr?)r5remaining_secondsr") r r9r#rrrWrr$r)rrrPr[r5r"s r9 get_compression_failure_cooldownz*SessionDB.get_compression_failure_cooldownDsW  4ikk Z  *$$- hjj                 ;4#w{++ C4 5 5Q   !4~.. S 4#w{++ C+ , ,Q  -!/#!5   s/AA!Ac|sddddS|j5|jd|f}dddn #1swxYwY|ddddSt |t jr|dn|d}t |t jr|dn|d}d |t|nd|dS) aGReturn the exact stored cooldown columns without expiry filtering. Compression cancellation uses this under its session lease so rollback can preserve an expired row, a partially-null row, or an absent session exactly instead of converting those states through the active-cooldown API. FNsession_existsr5r"r<r=rr>r?T)r9r#rrrWrr$r)rrr[r5r"s r9$get_compression_failure_cooldown_rowz.SessionDB.get_compression_failure_cooldown_rowisF T&+tdSS S Z  *$$- hjj                 ;&+tdSS S#w{++ C4 5 5Q #w{++ C+ , ,Q  #)7)Cn%%%    s/A  AAsnapshotct|dd}|s<|}|ddrtddS|d|dfd}|||}dt ndd }||krtd |d |dS) a(Restore and verify an exact cooldown-row snapshot. Unlike the ordinary record/clear helpers, this transactional rollback API deliberately propagates write and verification failures. A caller must not report cancellation as mutation-free when compensation failed. rCFzBcannot restore absent compression cooldown row: session now existsNr5r"ct|df}|jdkrtddS)Nr8r?z/compression cooldown rollback session missing: )rrr)rrrr"rs r9rz?SessionDB.restore_compression_failure_cooldown_row.._dosY\\=5*-F !##"RjRR$#r;TrBz>zJJFzz*E22 "X F<< 011 W%%        C   :::FF"191EeHooo4   X  <$<<17<<   r;csdSfd} ||dS#tj$r'}td|Yd}~dSd}~wwxYw)z?Clear any persisted compression-failure cooldown for a session.Nc6|dfdS)NzlUPDATE sessions SET compression_failure_cooldown_until = NULL, compression_failure_error = NULL WHERE id = ?rr!s r9rz9SessionDB.clear_compression_failure_cooldown.._dos. LL@      r;z1clear_compression_failure_cooldown(%s) failed: %sr9)rrrrs ` r9"clear_compression_failure_cooldownz,SessionDB.clear_compression_failure_cooldowns  F          $ $ $ $ $}    NNCC          s#AAAc|sdS|j5|j}| ddddS|d|f}dddn #1swxYwY|dSt |t jr|dn|d} tdt|pdS#ttf$rYdSwxYw)z3Return the persisted deterministic-fallback streak.rNz=SELECT compression_fallback_streak FROM sessions WHERE id = ?compression_fallback_streak r9r#rrrWrr$rrCrErFrrrr[rUs r9get_compression_fallback_streakz)SessionDB.get_compression_fallback_streaksM 1 Z  :D|        ,,O hjj                  ;1#w{++ C- . .Q   q#ejq//** *:&   11 ( A)AA AB22CCstreakcsdStdt|fd}||dS)z:Persist the deterministic-fallback streak for one session.Nrc8|dfdS)Nz@UPDATE sessions SET compression_fallback_streak = ? WHERE id = ?rr normalizedrs r9rz6SessionDB.set_compression_fallback_streak.._dos. LLRZ(     r;rrCr)rrrTrrXs ` @r9set_compression_fallback_streakz)SessionDB.set_compression_fallback_streaks`  FCKK((        C     r;c|sdS|j5|j}| ddddS|d|f}dddn #1swxYwY|dSt |t jr|dn|d} tdt|pdS#ttf$rYdSwxYw)aReturn the persisted ineffective-compaction strike count. Mirrors ``get_compression_fallback_streak``: this is the durable half of the anti-thrash guard (``_ineffective_compression_count`` on the built-in compressor), persisted so that a fresh compressor bound to a resumed session inherits an armed/tripped guard instead of starting from zero across process restarts (#54923). rNz?SELECT compression_ineffective_count FROM sessions WHERE id = ?compression_ineffective_countrPrQs r9!get_compression_ineffective_countz+SessionDB.get_compression_ineffective_countsO 1 Z  :D|        ,,Q hjj                  ;1#w{++ C/ 0 0Q   q#ejq//** *:&   11 rScountcsdStdt|fd}||dS)z@Persist the ineffective-compaction strike count for one session.Nrc8|dfdS)NzBUPDATE sessions SET compression_ineffective_count = ? WHERE id = ?rrWs r9rz8SessionDB.set_compression_ineffective_count.._dos. LLTZ(     r;rY)rrr^rrXs ` @r9!set_compression_ineffective_countz+SessionDB.set_compression_ineffective_counts`  FCJJ''        C     r;r@r< ttl_secondsc rsdStj}||zfd} t||S#tj$r'}t d|Yd}~dSd}~wwxYw)uExtend the compression lock lease if ``holder`` still owns it. Ownership is decided by the ``holder`` column alone, deliberately NOT by ``expires_at``: a live owner whose refresher thread was starved (GC pause, loaded CI runner, a slow write escaping ``_execute_write``'s retry budget) past its own TTL must be able to revive its still-unclaimed row on the next tick. Requiring ``expires_at >= now`` here made such a stall permanent — every later refresh matched 0 rows, so the owner kept compressing and rotating with no lease at all, which is exactly the unprotected window a competing path can fork the session lineage in. This does not resurrect a lock somebody else already took: SQLite serialises writes, so a reclaim (DELETE-expired + INSERT-or-IGNORE in :meth:`try_acquire_compression_lock`) and this UPDATE never interleave. Reclaim-first replaces ``holder``, so this UPDATE matches nothing and returns False; refresh-first pushes ``expires_at`` into the future, so the reclaimer's DELETE-expired matches nothing and its acquire fails. FcL|df}|jdkS)NzOUPDATE compression_locks SET expires_at = ? WHERE session_id = ? AND holder = ?rr')rcurrr<rs r9rz/SessionDB.refresh_compression_lock.._doXs4,,6Z0C ._doso# ,,' hjj  %/W[%A%AMCMMs1v*4C)E)EQC %%3q6#',,?OO-LL>#^4 (6$ LL&VS*5     ,,K hjj $!+C!=!=IH 3q6,H-- -r;z;Reclaimed stale compression lock for session=%s (holder=%s)z+try_acquire_compression_lock(%s) failed: %sN)r rrrrvrr) rrr<rcrrrkrrrPs `` @@r9try_acquire_compression_lockz&SessionDB.try_acquire_compression_lockis. 5ikk;& ' .' .' .' .' .' .' .' .R )-)<)> !}    NN=C    55555 sAA//B%>B  B%csdSfd} ||dS#tj$r'}td|Yd}~dSd}~wwxYw)aKRelease the compression lock for ``session_id`` iff we own it. Idempotent: no-op when the lock has already expired and been reclaimed by a different holder, or when no lock exists. The ``holder`` check prevents a late-returning compressor from clobbering a fresh lock held by someone else. Nc8|dfdS)Nrjr)rr<rs r9rz/SessionDB.release_compression_lock.._dos0 LL6V$     r;z'release_compression_lock(%s) failed: %sr9)rrr<rrs `` r9release_compression_lockz"SessionDB.release_compression_locks  F           $ $ $ $ $}    NN9C          s%AAAc|sdStj}|jd||f}|dSt |t jr|dn|dS)uReturn the current (non-expired) holder for ``session_id``, or None. Diagnostic helper — not used by the locking protocol itself. NzMSELECT holder FROM compression_locks WHERE session_id = ? AND expires_at >= ?r<r)r r#rrrWrr$)rrrPr[s r9get_compression_lock_holderz%SessionDB.get_compression_lock_holders}  4ikkj   7     (** ;4 *3 < <Hs8}}#a&Hr;) description provenancerrtruc sdSddlm}m}t||nt j ||||j   fd}|||jdS)a-Stamp durable mid-turn session activity (observation-only). Called (rate-limited) from ``AIAgent._touch_activity`` so gateway/CLI surfaces and stall consumers observe API/tool/compaction activity even when no new message row has been written yet (#72016 / #72039). Never moves ``last_activity_at`` backwards. When the timestamp advances, bounded ``last_activity_description`` / ``last_activity_provenance`` are written with it. No-ops when ``session_id`` is empty or the row does not exist. Nr)bound_activity_descriptionnormalize_activity_provenancec>|dfdS)NzUPDATE sessions SET last_activity_at = ?, last_activity_description = ?, last_activity_provenance = ? WHERE id = ? AND (last_activity_at IS NULL OR last_activity_at < ?)r)rdescprovrwhens r9rz-SessionDB.touch_session_activity.._do s9 LLV tT:t4      r;r)agent.session_activityrwrxrr rUr_ACTIVITY_WRITE_PATIENCE_S) rrrrtrurwrxrrzr{r|s ` @@@r9touch_session_activityz SessionDB.touch_session_activitys&  F        2>RRty{{;;))+66,,Z88>         CD,KLLLLLr;csdSddlm |jdf}n#t j$rd}YnwxYw|jt|t js|dn|d}t|t js|dn|d}|s|r|j j krdSfd}| ||j dS) aClear mid-turn activity labels after a turn ends. Keeps ``last_activity_at`` intact so idle / watchdog clocks stay continuous. Description and provenance are observation labels for *what was happening at* that timestamp during an active turn; once the turn is idle they must not keep advertising "compressing" / "executing tool" (#72039). Response-critical-path contract (#76354 review S1): runs in the turn's ``finally``; a no-op clear (labels already empty) skips the write transaction entirely, and a real clear uses the same short sub-second busy budget as :meth:`touch_session_activity` instead of the full routine write patience. NrrzUSELECT last_activity_description, last_activity_provenance FROM sessions WHERE id = ?last_activity_descriptionr?last_activity_provenancecN|ddjjfdS)Nz\UPDATE sessions SET last_activity_description = ?, last_activity_provenance = ? WHERE id = ?r>)rUNKNOWNrU)rrrs r9rz4SessionDB.clear_session_activity_labels.._do@s; LL'/5zB      r;r) r}rr#rrrrrWr$rrUrr~)rrr[rzr{rrs ` @r9clear_session_activity_labelsz'SessionDB.clear_session_activity_labelss9  F====== *$$- hjj C }   CCC  ?!+C!=!=c3q663GbCcD!+C!=!=b3q663GaCbD   $6$>$DDD       CD,KLLLLLs.=AAc|sdS||}|sdSddlm}||d|d|dS)z?Return the durable activity snapshot for *session_id*, or None.Nr)build_activity_snapshotlast_activity_atrr)rrr) get_sessionr}rr_)rrr[rs r9get_session_activityzSessionDB.get_session_activityKs 4z** 4BBBBBB&& WW%788&)gg.I&J&J%(WW-G%H%H    r;model_config_jsoncl|fd}||dS)aUpdate model_config and optionally model for an existing session. Uses COALESCE so that passing model=None leaves the stored model column unchanged. Routes through _execute_write for the standard BEGIN IMMEDIATE + jitter-retry + lock guarantee. c:|dfdS)NzMUPDATE sessions SET model_config = ?, model = COALESCE(?, model) WHERE id = ?r)rrrrs r9rz*SessionDB.update_session_meta.._dois0 LL_"E:6     r;Nrr)rrrrrs ``` r9update_session_metazSessionDB.update_session_metaZsX !!!        C     r;cDfd}|dS)z0Store the full assembled system prompt snapshot.c|}|d|f|dS)NzMUPDATE sessions SET system_prompt_hash = ?, system_prompt = NULL WHERE id = ?)rrr)rrrrr2s r9rz+SessionDB.update_system_prompt.._dotsY!%!:!:4!O!O  LLP#Z0     4 4T : : : : :r;Nr)rrr2rs``` r9update_system_promptzSessionDB.update_system_promptpsD ; ; ; ; ; ; ; C     r;clfd}|dS)a@Update the model for a session after a mid-session switch. Unlike ``update_token_counts`` which uses ``COALESCE(model, ?)`` (only filling in NULL), this unconditionally sets the model column so that the dashboard reflects the user's latest /model choice. Also nulls ``system_prompt`` so stale ``Model:`` / ``Provider:`` footer metadata is rebuilt on the next turn. A successful /model switch explicitly replaces any confirmed Browser runtime lock while preserving unrelated lineage markers in ``model_config``. cb|df|dS)NaUPDATE sessions SET model = ?, model_config = CASE WHEN model_config IS NULL THEN NULL WHEN json_valid(model_config) THEN json_remove(model_config, '$.browser_model_lock') ELSE model_config END, system_prompt = NULL, system_prompt_hash = NULL WHERE id = ?rr)rrrrs r9rz+SessionDB.update_session_model.._dosD LL # #     4 4T : : : : :r;Nr)rrrrs``` r9update_session_modelzSessionDB.update_session_model~sX" !!! ; ; ; ; ; ; ; C     r;)rprovider model_options route_source confirmedrrrrc|pdpd|pi|pdt|tjdfd}|dS)aKPersist a Browser / API client runtime lock without clobbering lineage markers. Merges ``browser_model_lock`` into the existing ``model_config`` JSON so ``_branched_from`` / ``_delegate_from`` survive. Nulls ``system_prompt`` so cached ``Model:`` / ``Provider:`` footers cannot lie after a switch. r>)rrrrr updated_atc\|df}|dSt|tjr|dn|d}i}t|t rS|r? tj|}t|tr|}n6#t$ri}Yn(wxYwt|trt|}|d<|dtj |f |dS)N.SELECT model_config FROM sessions WHERE id = ?rcrbrowser_model_lockzUPDATE sessions SET model_config = ?, model = COALESCE(?, model), system_prompt = NULL, system_prompt_hash = NULL WHERE id = ?) rrrWrr$rXr`rloadsrrKrr) rr[rrparsedlockrrrs r9rz2SessionDB.update_session_runtime_lock.._dosE,,@ hjj {)3C)E)EQ#n%%3q6C%'F#s## #  # !Z__F!&$//(!'    FFF C&& #c+/F' ( LL# F##UJ7     4 4T : : : : :+B11 C?CN)rvr r) rrrrrrrrrs ``` @r9update_session_runtime_lockz%SessionDB.update_session_runtime_locks"!B[b*0b(.Bi)++    ; ; ; ; ; ; ; ;: C     r;enabledcHsdSfd}||dS)uYPersist the per-session YOLO bypass flag into ``model_config``. Merges ``yolo_mode`` into the existing ``model_config`` JSON (same merge discipline as ``update_session_runtime_lock`` so lineage markers like ``_branched_from`` / ``_delegate_from`` survive). The CLI resume paths read this flag back so a ``/yolo ON`` toggle — or a ``--yolo`` launch — survives ``hermes --resume`` into a fresh process. No-op when the session row doesn't exist yet; the creation-time ``model_config`` carries the flag for ``--yolo`` launches. NcJ|df}|dSt|tjr|dn|d}i}t|t rS|r? tj|}t|tr|}n6#t$ri}Yn(wxYwt|trt|}t|d<|dtj |fdS)Nrrcr yolo_modez1UPDATE sessions SET model_config = ? WHERE id = ?) rrrWrr$rXr`rrrrKrvr)rr[rrrrrs r9rz'SessionDB.set_session_yolo.._dos2,,@ hjj {)3C)E)EQ#n%%3q6C%'F#s## #  # !Z__F!&$//(!'    FFF C&& #c"&w--F;  LLCF##Z0     rr)rrrrs `` r9set_session_yolozSessionDB.set_session_yolosM  F      . C     r; session_metac|pid}t|tr' tj|}n#t $rYdSwxYwt|t sdSt|dS)uRead the persisted YOLO flag off a session row dict. Accepts the dict returned by ``get_session`` (``model_config`` is a JSON string) or an already-parsed dict. Returns False on any parse failure — resume must never enable the bypass by accident. rcFr)r_rWrXrrrKrrv)rrs r9session_yolo_enabledzSessionDB.session_yolo_enableds!r&&~66 c3    joo   uu #t$$ 5CGGK(()))sA AA) billing_modebase_urlrctfd}|dS)uUnconditionally update the billing provider/base_url for a session. Unlike ``update_token_counts`` which uses ``COALESCE(billing_provider, ?)`` (only filling in NULL), this unconditionally sets the billing fields so that the dashboard reflects the user's latest /model switch. Also nulls ``system_prompt`` so the cached snapshot (which embeds a stale ``Model:`` / ``Provider:`` header) is rebuilt — matching the behavior of ``update_session_model`` (see #48173, #48248). cf|df|dS)NaUPDATE sessions SET billing_provider = ?, billing_base_url = ?, billing_mode = COALESCE(?, billing_mode), system_prompt = NULL, system_prompt_hash = NULL WHERE id = ?r)rrrrrrs r9rz3SessionDB.update_session_billing_route.._do*sH LL#8\:>     4 4T : : : : :r;Nr)rrrrrrs````` r9update_session_billing_routez&SessionDB.update_session_billing_routesd& !!! ; ; ; ; ; ; ; ; ; C     r;) input_tokens output_tokenscache_read_tokenscache_write_tokensreasoning_tokensapi_call_count)estimated_cost_usdactual_cost_usd)r cost_status cost_sourcepricing_versionbilling_providerbilling_base_urlrc |j5|j}|jo|dup| }|s|j||f||sPt j|jdd}||_| tj |j |j dddn #1swxYwY|r|j|fi|dSdS)uEnqueue a token/cost delta for the background writer. Accepts the same keyword arguments as :meth:`update_token_counts` and applies them asynchronously with identical semantics. Cheap (append + notify) — safe to call on the turn thread after every API call. After close() has stopped the writer, falls back to the synchronous path and may raise like :meth:`update_token_counts`. Nzsession-db-token-writerT)targetrLdaemon)rHrIrJis_aliverFrr7Thread_token_writer_loopstartrregisterr notify_allupdate_token_counts)rrrthreadwriter_stoppeds r9queue_token_countszSessionDB.queue_token_countsSsg # 4 4.F!4$7foo&7&7"7 " 4!((*f)=>>>>):):>'-#66#F 17D-LLNNNOD$CDDD&113331 4 4 4 4 4 4 4 4 4 4 4 4 4 4 42  ; %D $Z : :6 : : : : : ; ;sCCC Cr c>|js |jsdSd}|j5tj|z}|js|jr|j}||s<|js5d|_t|j}|jnR|tjz }|dkr ddddS|j ||j|jdddn #1swxYwY|r | ||j5d|_|j dddn #1swxYwYnD#|j5d|_|j dddw#1swxYwYwxYwdS)uOBlock until every queued token delta has been applied. Returns True when the queue is fully drained, False on timeout (callers then read totals that are stale by the still-queued deltas — no worse than reading before the flush existed). Never raises: apply failures are logged by the writer. TNrF) rFrKrHr rrIrrrwait_apply_token_batchr)rr batchrr remainings r9rzSessionDB.flush_token_countss  )@ 4  # 7 7~'''1H# 7t'> 72^6??+<+<^ 3$/3D+ !233E%++---$t~'7'77 >> 9 7 7 7 7 7 7 7 7:&++I6667# 7t'> 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7<  8 8''...+88.3D+*55777888888888888888T+88.3D+*557778888888888888888tsZBC6(C66C:=C:E !E  EEF!!F FF FF Fcv |j5|js.|js'|j|js|j'|js ddddSd|_t |j}|jdddn #1swxYwY |||j5d|_|jdddn #1swxYwYnD#|j5d|_|jdddw#1swxYwYwxYw9)NTF) rHrFrJrrKrrrr)rrs r9rzSessionDB._token_writer_loops( 8' * *+2D4K2*//111+2D4K2( * * * * * * * *+/'T.//!''))) * * * * * * * * * * * * * * * 8''...+88.3D+*55777888888888888888T+88.3D+*557778888888888888888! 8sY>B4BBB C5>v>>>>   K   s, A AA A## B-BBc.g}|D]\}d}ds$|ftfd|jDz}|r||dd|kr|dd}|jD]2}||d|dz||<3|jD]6}|}|||pd|z||<7|||t fd|DS) aMerge consecutive incremental deltas with an identical route. Only adjacent deltas merge, so ordering across sessions and across a mid-session /model switch is preserved exactly. absolute=True deltas (cumulative overwrites) never merge. Nabsolutec3BK|]}|VdSr)r_)rzfrs r9rz3SessionDB._coalesce_token_deltas..s>,,&'FJJqMM,,,,,,r;r!rrcg|] \}}}||f Srfrf)rzrr{kws r9rz4SessionDB._coalesce_token_deltas..s"333jabb 333r;)r_tuple_TOKEN_DELTA_ROUTE_FIELDS_TOKEN_DELTA_SUM_FIELDS_TOKEN_DELTA_COST_FIELDSrr) rrgroupsrrpmergedrrUrs @r9rz SessionDB._coalesce_token_deltassnEG"' ? ? JC::j)) !me,,,,+/+I,,,'' ?#/fRjms.B.BA5DDA & 1a 0 06::a3C3C CF1II6CCA"JJqMME(&,ZZ]]%9cU$Bq C sJV =>>>>33F3333r;$@ join_timeoutc|j5d|_|j|j}dddn #1swxYwY|n|rZ|||r0t d|t|j dS|j5tj |z}|j ry|tj z }|dkr>NNP$c$*;&<&<  * * * * * * * *&++I666) 7 *++E **.'!'')))+ * * * * * * * * * * * * * * *,  8 8''...+88.3D+*55777888888888888888888T+88.3D+*557778888888888888888  8 8sk(<AA>A)F4AFFF%G<!G//G36G3<H=!H1% H=1H5 5H=8H5 9H=cR |dS#t$rYdSwxYwr)rrKr)s r9rz$SessionDB._drain_token_queue_at_exit3sA   # # % % % % %    DD s  &&rrrrrrrrrrrrrrrch d|rdndtp p p pppp  | r ndrndrndrndf| o p p pppp   fd}|dS)uUpdate token counters and backfill model if not already set. When *absolute* is False (default), values are **incremented** — use this for per-API-call deltas (CLI path). When *absolute* is True, values are **set directly** — use this when the caller already holds cumulative totals (gateway path, where the cached agent accumulates across messages). r)raUPDATE sessions SET input_tokens = ?, output_tokens = ?, cache_read_tokens = ?, cache_write_tokens = ?, reasoning_tokens = ?, estimated_cost_usd = COALESCE(?, 0), actual_cost_usd = CASE WHEN ? IS NULL THEN actual_cost_usd ELSE ? END, cost_status = COALESCE(?, cost_status), cost_source = COALESCE(?, cost_source), pricing_version = COALESCE(?, pricing_version), billing_provider = COALESCE(billing_provider, ?), billing_base_url = COALESCE(billing_base_url, ?), billing_mode = COALESCE(billing_mode, ?), model = COALESCE(model, ?), api_call_count = ? WHERE id = ?a^UPDATE sessions SET input_tokens = input_tokens + ?, output_tokens = output_tokens + ?, cache_read_tokens = cache_read_tokens + ?, cache_write_tokens = cache_write_tokens + ?, reasoning_tokens = reasoning_tokens + ?, estimated_cost_usd = COALESCE(estimated_cost_usd, 0) + COALESCE(?, 0), actual_cost_usd = CASE WHEN ? IS NULL THEN actual_cost_usd ELSE COALESCE(actual_cost_usd, 0) + ? END, cost_status = COALESCE(?, cost_status), cost_source = COALESCE(?, cost_source), pricing_version = COALESCE(?, pricing_version), billing_provider = COALESCE(billing_provider, ?), billing_base_url = COALESCE(billing_base_url, ?), billing_mode = COALESCE(billing_mode, ?), model = COALESCE(model, ?), api_call_count = COALESCE(api_call_count, 0) + ? WHERE id = ?Nc|df}||dnd}||dnd}t||dndpd}|dko+o)tot o |kp| k}|r|d  f|r'|    dSdS)NzISELECT model, billing_provider, api_call_count FROM sessions WHERE id = ?rrrrzUPDATE sessions SET model = ?, billing_provider = ?, billing_base_url = ?, billing_mode = ? WHERE id = ?)rrrrrrrrrrrrrr)rrrCrv_record_model_usage)rr[existing_modelexisting_providerexisting_api_callsfirst_accounted_routerrrrrrrrrrhas_accounted_usagerrrrrrecord_model_usagerrr_s r9rz*SessionDB.update_token_counts.._dos,,[ hjj .1_S\\$N;>?$6 7 7PT !$sc*:&;&;TU%[Z[!\!\ #a'W'WKKW)**W$u,U0AEU0U "%  ',.> jY  LLf % % %! ((%5%5!-!-"/&7'9%5'9$3 + +#1!)  r;)rrvr)rrrrrrrrrrrrrrrrrrrrrrr_s```````````` ```` @@@@r9rzSessionDB.update_token_counts9sB   Ye DDD ) ##CC*#C(#  5M 5-> 5! 5%5 59G 5! 5%4              3 =   3 =  / 9LLT( 2EEd  # @#+l  "M "-> "! "%5 "9G "!  / / / / / / / / / / / / / / / / / / / / / / / / ` C     r;)taskrc|d|f}||dnd}||dnd}||dnd}||dnd}|r|pd}|pd}|pd}|pd}n|p|pd}|p|pd}|p|pd}|p|pd}tj}|d ||||||pd|pd |pd |pd | pd | pd | pd t| pd t| pd ||||fdS) uAccumulate a per-API-call usage delta into session_model_usage. Runs inside the caller's write transaction (after the ``sessions`` UPDATE) so the per-model rows stay consistent with the summary row. When the caller omits the model/provider (some paths only pass token deltas), fall back to the values already recorded on the session row — the same COALESCE-from-session behaviour the summary update uses. ``task`` distinguishes what kind of work consumed the tokens: ``''`` (empty) is the main agent loop; auxiliary calls record their task name (``vision``, ``compression``, ``title_generation``, ...) via :meth:`record_auxiliary_usage` (issue #23270). zYSELECT model, billing_provider, billing_base_url, billing_mode FROM sessions WHERE id = ?Nrrrrrr>aINSERT INTO session_model_usage ( session_id, model, billing_provider, billing_base_url, billing_mode, task, api_call_count, input_tokens, output_tokens, cache_read_tokens, cache_write_tokens, reasoning_tokens, estimated_cost_usd, actual_cost_usd, cost_status, cost_source, first_seen, last_seen ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(session_id, model, billing_provider, billing_base_url, billing_mode, task) DO UPDATE SET api_call_count = api_call_count + excluded.api_call_count, input_tokens = input_tokens + excluded.input_tokens, output_tokens = output_tokens + excluded.output_tokens, cache_read_tokens = cache_read_tokens + excluded.cache_read_tokens, cache_write_tokens = cache_write_tokens + excluded.cache_write_tokens, reasoning_tokens = reasoning_tokens + excluded.reasoning_tokens, estimated_cost_usd = estimated_cost_usd + excluded.estimated_cost_usd, actual_cost_usd = actual_cost_usd + excluded.actual_cost_usd, cost_status = COALESCE(excluded.cost_status, cost_status), cost_source = COALESCE(excluded.cost_source, cost_source), last_seen = excluded.last_seenrr)rrr r)rrrrrrrrrrrrrrrrrrr[ sess_model sess_provider sess_base_urlsess_billing_mode eff_model eff_provider eff_base_urleff_billing_moderPs r9rzSessionDB._record_model_usagesDll ) M   (** &)_S\\$ 36?.// 36?.// 36?C//  G*I+1rL+1rL+1r  88yI+B}BL+B}BL+F/@FB ikk  5*  #!!"!&Q"'a %A(/C00o,--% +) ) ) ) ) r;rc (|j||fd|i||S)zHEnsure a session row exists (INSERT OR IGNORE). Accepts optional kwargs.rr)rrrrrs r9ensure_sessionzSessionDB.ensure_sessionHs, ! VKK5KFKKKr;) rrrrrrrrrc  rsdSd  f d} | dS)usRecord an auxiliary LLM call's usage against *session_id* (issue #23270). Auxiliary calls (vision, compression, title_generation, web_extract, session_search, ...) historically discarded their usage, leaving the dashboard's per-model analytics blind to aux model spend. This writes a per-(model, provider, task) delta into ``session_model_usage`` — the same table the main loop's ``update_token_counts`` feeds — WITHOUT touching the ``sessions`` summary row. That separation is deliberate: the gateway overwrites session counters with absolute main-loop totals, so folding aux tokens into the summary row would either be clobbered or double-counted. Insights/analytics read the union of both. Best-effort by contract: callers must never fail an aux call because accounting failed. Nrch | dpdpdpdpd pddddd dS)Nrr?)rrrrrrrrrrrrrrr)r) rrrrrrrrrrrrrs r9rz-SessionDB.record_auxiliary_usage.._doxsv  $ $!1!1!).Q+0q"3"8q#5#:!1!6Q#5 $   # %     r;)rr) rrrrrrrrrrrrrs ```````````` r9record_auxiliary_usagez SessionDB.record_auxiliary_usageSs<   F   Y777                ( C     r; sessions_dirzOptional[Path]ctjdz fd}|pg}|r|r|D]}||t|S)zCRemove empty TUI ghost sessions (no messages, no title, >24hr old).Qc|df}d|D}|rTddt|z}|d|d|||S)NaZ SELECT id FROM sessions WHERE source = 'tui' AND title IS NULL AND ended_at IS NOT NULL AND started_at < ? AND NOT EXISTS ( SELECT 1 FROM messages WHERE messages.session_id = sessions.id ) cfg|].}t|ttfr|dn|d/S)rr)rWrrrs r9rzESessionDB.prune_empty_ghost_sessions.._do..s7SSS:a%77D1Q44QtWSSSr;r}r~rr)rrrrr)rrr placeholderscutoffrs r9rz1SessionDB.prune_empty_ghost_sessions.._dos<< !  %HJJ TSdSSSC ?"xxc#hh77  HHHH#88>>>Jr;)r r_remove_session_filesr)rrr removed_idsr{rs` @r9prune_empty_ghost_sessionsz$SessionDB.prune_empty_ghost_sessionssu$      ())#..4"  >K >" > >**<====;r;chtjdz fd}||pdS)ajMark orphaned compression continuation sessions as ended. Targets child sessions that were never finalized: parent is ended with reason='compression', child has messages but no end_reason/ended_at and api_call_count=0. Non-destructive: preserves all messages and sets end_reason='orphaned_compression'. Fix for #20001. i: chtj}|d|f}|jS)Na UPDATE sessions SET ended_at = ?, end_reason = 'orphaned_compression' WHERE api_call_count = 0 AND end_reason IS NULL AND ended_at IS NULL AND started_at < ? AND parent_session_id IS NOT NULL AND EXISTS ( SELECT 1 FROM sessions p WHERE p.id = sessions.parent_session_id AND p.end_reason = 'compression' AND p.ended_at IS NOT NULL ) AND EXISTS ( SELECT 1 FROM messages m WHERE m.session_id = sessions.id ) )r rr)rrPrrs r9rz=SessionDB.finalize_orphaned_compression_sessions.._dos9)++C\\(f +F.? "r;rr)rrrs @r9&finalize_orphaned_compression_sessionsz0SessionDB.finalize_orphaned_compression_sessionssIv% # # # # #6""3'',1,r;c ||5}|d|f}|}dddn #1swxYwY|r||ndS)zGet a session by ID.zSELECT s.*, COALESCE(sp.prompt, s.system_prompt) AS _system_prompt_resolved FROM sessions s LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hash WHERE s.id = ?N)rrWrrr)rrrrr[s r9rzSessionDB.get_sessions !!! ^^   $\\!  F//##C $ $ $ $ $ $ $ $ $ $ $ $ $ $ $/2;t%%c***t;s,A!!A%(A%session_id_or_prefixc||}|r|dS|dddddd}|j5|jd|df}d |D}d d d n #1swxYwYt |d kr|d Sd S) a*Resolve an exact or uniquely prefixed session ID to the full ID. Returns the exact ID when it exists. Otherwise treats the input as a prefix and returns the single matching session ID if the prefix is unambiguous. Returns None for no matches or ambiguous prefixes. r\\\rrlrrzSSELECT id FROM sessions WHERE id LIKE ? ESCAPE '\' ORDER BY started_at DESC LIMIT 2cg|] }|d Srrfrzr[s r9rz0SessionDB.resolve_session_id..s>>>Ss4y>>>r;Nr?r)rrr9r#rrr)rr rescapedrmatchess r9resolve_session_idzSessionDB.resolve_session_ids'  !566  ;  ! WT6 " " WS% WS%  Z ? ?Z''f F?>FOO,=,=>>>G  ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? w<<1  1: ts%>B//B36B3rtitlecp|sdSt|}tjdd|}tjdd|}tjdd|}|sdSt |t jkr-tdt |dt jd |S) aValidate and sanitize a session title. - Strips leading/trailing whitespace - Removes ASCII control characters (0x00-0x1F, 0x7F) and problematic Unicode control chars (zero-width, RTL/LTR overrides, etc.) - Collapses internal whitespace runs to single spaces - Normalizes empty/whitespace-only strings to None - Enforces MAX_TITLE_LENGTH Returns the cleaned title string or None. Raises ValueError if the title exceeds MAX_TITLE_LENGTH after cleaning. Nz [\x00-\x08\x0b\x0c\x0e-\x1f\x7f]r>zB[\u200b-\u200f\u2028-\u202e\u2060-\u2069\ufeff\ufffc\ufff9-\ufffb]z\s+rgzTitle too long (z chars, max r)r resubr`rrMAX_TITLE_LENGTHrF)rcleaneds r9sanitize_titlezSessionDB.sanitize_titles 4%U++ &z*UPDATE sessions SET title = ? WHERE id = ?)rrr#rFr) rrrconflict conflict_id predicater$rrrs r9rz)SessionDB._set_session_title.._dobsF ,,=M(**?gg&6&B1 IJ'"??,,"*4.K44+Z5  K(N )YeYYKYY1>E,,2I\\HYHH #F? "r;r)rr)rrrr$rrs```` r9_set_session_titlezSessionDB._set_session_titleYsc##E**- #- #- #- #- #- #- #- #^&&s++!|r;c2|||dS)aLSet or update a session's title. Returns True if session was found and title was set. Raises ValueError if title is already in use by another session, or if the title fails validation (too long, invalid characters). Empty/whitespace-only strings are normalized to None (clearing the title). Fr$r+rrrs r9set_session_titlezSessionDB.set_session_titles&&z5&NNNr;c2|||dS)a Set an auto-generated title only when the current title is NULL. The predicate and write run in one transaction so a concurrent manual rename cannot be overwritten. Validation and uniqueness behavior match :meth:`set_session_title`. Tr-r.r/s r9set_auto_title_if_emptyz!SessionDB.set_auto_title_if_emptys&&z5&MMMr;c|j5|jd|f}|}dddn #1swxYwY|r|dndS)z%Get the title for a session, or None.r'Nrr9r#rr)rrrr[s r9get_session_titlezSessionDB.get_session_titles Z $ $Z''9J=F//##C  $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ #,s7||,1AA  A archivedcHfd}||}|dkS)uArchive or unarchive a session. Archived sessions are hidden from the default session list but keep all their messages — this is a soft hide, not a delete. For compression chains, archive the whole logical conversation. Desktop lists compression roots projected forward to their latest continuation; updating only the displayed tip lets the still-unarchived root resurrect it on refresh. Returns True when at least one row was updated. c|drdndf}|j}||dkr-|dd}|S)Na[ WITH RECURSIVE ancestors(id) AS ( SELECT ? UNION SELECT parent.id FROM ancestors a JOIN sessions child ON child.id = a.id JOIN sessions parent ON parent.id = child.parent_session_id WHERE parent.end_reason = 'compression' ), descendants(id) AS ( SELECT ? UNION SELECT child.id FROM descendants d JOIN sessions parent ON parent.id = d.id JOIN sessions child ON child.parent_session_id = parent.id WHERE parent.end_reason = 'compression' ), lineage(id) AS ( SELECT id FROM ancestors UNION SELECT id FROM descendants ) UPDATE sessions SET archived = ? WHERE id IN (SELECT id FROM lineage) r?rSELECT changes()rrr)rrrr7rs r9rz+SessionDB.set_session_archived.._dosp\\:Zh)=A>=F@H8a<<<<(:;;DDFFqIOr;rr)rrr7rrs `` r9set_session_archivedzSessionDB.set_session_archivedsB$ $ $ $ $ $ J&&s++!|r;pinnedcHfd}||}|dkS)ulPin or unpin a session (and its whole compression lineage). ``pinned`` is a durable "keep" flag: pinned sessions are exempt from the ``sessions.auto_archive`` stale sweep (see :meth:`archive_stale_sessions`). Desktop is the current writer — its sidebar pins mirror here so a backend/other-surface sweep honours them. Like :meth:`set_session_archived` the whole compression chain is flipped as a unit, so pinning the surfaced tip protects the root (and vice-versa) no matter which id the caller holds. Returns True when at least one row changed. c|drdndf}|j}||dkr-|dd}|S)NaY WITH RECURSIVE ancestors(id) AS ( SELECT ? UNION SELECT parent.id FROM ancestors a JOIN sessions child ON child.id = a.id JOIN sessions parent ON parent.id = child.parent_session_id WHERE parent.end_reason = 'compression' ), descendants(id) AS ( SELECT ? UNION SELECT child.id FROM descendants d JOIN sessions parent ON parent.id = d.id JOIN sessions child ON child.parent_session_id = parent.id WHERE parent.end_reason = 'compression' ), lineage(id) AS ( SELECT id FROM ancestors UNION SELECT id FROM descendants ) UPDATE sessions SET pinned = ? WHERE id IN (SELECT id FROM lineage) r?rr:r;)rrrr=rs r9rz)SessionDB.set_session_pinned.._dosp\\:Zf);!<=F@H8a<<<<(:;;DDFFqIOr;rr)rrr=rrs `` r9set_session_pinnedzSessionDB.set_session_pinnedsB$ $ $ $ $ $ J&&s++!|r;c|5}|d|f}|}dddn #1swxYwY|r||ndS)z?Look up a session by exact title. Returns session dict or None.zSELECT s.*, COALESCE(sp.prompt, s.system_prompt) AS _system_prompt_resolved FROM sessions s LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hash WHERE s.title = ?N)rWrrr)rrrrr[s r9get_session_by_titlezSessionDB.get_session_by_titles ^^   $\\$  F//##C $ $ $ $ $ $ $ $ $ $ $ $ $ $ $/2;t%%c***t;s,A  AAc||}|dddddd}|5}|d|df}|}d d d n #1swxYwY|r|d d S|r|d Sd S) adResolve a title to a session ID, preferring the latest in a lineage. If the exact title exists, returns that session's ID. If not, searches for "title #N" variants and returns the latest one. If the exact title exists AND numbered variants exist, returns the latest numbered variant (the most recent continuation). rrrrlrrzaSELECT id, title, started_at FROM sessions WHERE title LIKE ? ESCAPE '\' ORDER BY started_at DESC #%Nrr)rBrrWrr)rrrrrrnumbereds r9resolve_session_by_titlez"SessionDB.resolve_session_by_title$s))%00--f--55c5AAII#uUU ^^   )\\J"F ((H  ) ) ) ) ) ) ) ) ) ) ) ) ) ) )  A;t$ $  ; ts(/B##B'*B' base_titlec Ntjd|}|r|d}n|}|dddddd}|j5|jd ||d f}d |D}d d d n #1swxYwY|s|Sd}|D]I}tjd |} | r0t|t| d}J|d|dzS)uGenerate the next title in a lineage (e.g., "my session" → "my session #2"). Strips any existing " #N" suffix to find the base name, then finds the highest existing number and increments. z^(.*?) #(\d+)$r?rrrrlrrzESELECT title FROM sessions WHERE title = ? OR title LIKE ? ESCAPE '\'rDcg|] }|d S)rrfrs r9rz7SessionDB.get_next_title_in_lineage..VsBBBG BBBr;Nz ^.* #(\d+)$z #) rrRrDrr9r#rrrrC) rrGrRbaserrrmax_numtms r9get_next_title_in_lineagez#SessionDB.get_next_title_in_lineageAs*J77  ;;q>>DDD,,tV,,44S%@@HHeTT Z C CZ''X''FCB0A0ABBBH  C C C C C C C C C C C C C C C K 8 8A++A 8gs1771::77'''A+'''s5?CCCcr|}|r|hn t}tdD]}|j5|jdt dd|f}|}dddn #1swxYwY||cS|d}|r||vr|cS|||}|S)aWalk the compression-continuation chain forward and return the tip. A compression continuation is a child of a session whose ``end_reason = 'compression'``. Older builds tried to distinguish continuations from branches/subagents by requiring ``child.started_at >= parent.ended_at``. That ordering is too brittle: gateway + compression races can insert the real continuation row before the parent row's ``ended_at`` is written, while a stale websocket later creates/reuses a sibling that *does* satisfy the timestamp test. The visible symptom is brutal: desktop resume follows the stale sibling and the user's latest messages look "lost" even though they are persisted in the real continuation chain. Instead, only follow children of compression-ended parents, exclude explicit branch/delegate/tool children, and prefer children that are themselves continuing the compression chain (``end_reason='compression'``) or still live over stale closed siblings such as ``ws_orphan_reap``. Returns the latest continuation tip, or the input id when no continuation exists. ra SELECT child.id FROM sessions parent JOIN sessions child ON child.parent_session_id = parent.id WHERE parent.id = ? AND parent.end_reason = 'compression' AND json_extract(COALESCE(child.model_config, '{}'), '$._branched_from') IS NULL AND json_extract(COALESCE(child.model_config, '{}'), '$._delegate_from') IS NULL AND COALESCE(child.source, '') != 'tool' ORDER BY CASE WHEN child.end_reason = 'compression' THEN 0 WHEN child.ended_at IS NULL THEN 1 ELSE 2 END, rz DESC, child.started_at DESC, child.id DESC LIMIT 1 Nr)rrr9r#rrrr)rrrseenrrr[child_ids r9get_compression_tipzSessionDB.get_compression_tipds3*#.yys  A ( (++088(J+.oo''1 ( ( ( ( ( ( ( ( ( ( ( ( ( ( (2{4yH x4// HHX   GGsAA<<B B r_session_compact_cols_sqlsourcesexclude_sourcesrhlimitoffsetinclude_childrenmin_message_countproject_compression_tipsorder_by_last_activeinclude_archived archived_onlyid_query search_query compact_rowsinclude_pinnedc@|g}g}|s?|t|tdd|r|gnt |pg}|rMdd|D}|d|d|||r*|d|||rMdd|D}|d |d|||r. "#A#AAC#A#A#A#A#A#Ar; s.source IN (rzs.session_key = ?c3K|]}dVdSrlrfrms r9rz/SessionDB.list_sessions_rich..rgr;s.source NOT IN (rs.message_count >= ?s.archived = 1s.archived = 0zWHERE  AND r>zA, COALESCE(sp.prompt, s.system_prompt) AS _system_prompt_resolvedz=LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hashneedler3c|dddddd}d|dS)Nrrrrlrr)r)rors r9 _like_patternz3SessionDB.list_sessions_rich.._like_patternLsINN40088eDDLLSRWXX&7~~~%r;znEXISTS (SELECT 1 FROM chain cq WHERE cq.root_id = s.id AND LOWER(cq.cur_id) LIKE ? ESCAPE '\')z[\W_]+z`REPLACE(REPLACE(REPLACE(REPLACE(LOWER(COALESCE({0}, '')), '-', ''), '_', ''), '.', ''), ' ', '')zEXISTS (SELECT 1 FROM chain cq JOIN sessions cs ON cs.id = cq.cur_id WHERE cq.root_id = s.id AND (LOWER(COALESCE(cs.title, '')) LIKE ? ESCAPE '\' OR LOWER(cq.cur_id) LIKE ? ESCAPE '\'rz OR zcs.titlez LIKE ? ESCAPE '\'rrzs.*zr WITH RECURSIVE chain(root_id, cur_id) AS ( SELECT s.id, s.id FROM sessions s a UNION ALL SELECT c.root_id, child.id FROM chain c JOIN sessions parent ON parent.id = c.cur_id JOIN sessions child ON child.parent_session_id = c.cur_id WHERE parent.end_reason = 'compression' AND json_extract(COALESCE(child.model_config, '{}'), '$._branched_from') IS NULL AND json_extract(COALESCE(child.model_config, '{}'), '$._delegate_from') IS NULL AND COALESCE(child.source, '') != 'tool' ), chain_max AS ( SELECT root_id, MAX(cur_idz) AS effective_last_active FROM chain GROUP BY root_id ) SELECT z@, COALESCE( (SELECT a FROM messages m WHERE m.session_id = s.id AND m.role = 'user' AND m.content IS NOT NULL ORDER BY m.timestamp, m.id LIMIT 1), '' ) AS _preview_raw, sz AS last_active, COALESCE(cm.effective_last_active, s.started_at) AS _effective_last_active FROM sessions s LEFT JOIN chain_max cm ON cm.root_id = s.id z z ORDER BY _effective_last_active DESC, s.started_at DESC, s.id DESC LIMIT ? OFFSET ? z SELECT z@ AS last_active FROM sessions s zY ORDER BY s.started_at DESC LIMIT ? OFFSET ? N _preview_rawpreview_effective_last_activech|] }|d Srrf)rzrss r9r|z/SessionDB.list_sessions_rich..s222A$222r;z AND s.pinned = 1zWHERE s.pinned = 1a FROM messages m WHERE m.session_id = s.id AND m.role = 'user' AND m.content IS NOT NULL ORDER BY m.timestamp, m.id LIMIT 1), '' ) AS _preview_raw, COALESCE( (SELECT MAX(m2.timestamp) FROM messages m2 WHERE m2.session_id = s.id), s.started_at ) AS last_active FROM sessions s z8 ORDER BY s.started_at DESC rrr)ra) rrr message_counttool_call_countr last_activerurr2r^rr]_lineage_root_id)rrrrgrrrror`rrXrrr!_compact_session_colsrrrrWrrrrrrr_rR_get_session_rich_rows_batchrvaluesr):rrrUrVrhrWrXrYrZr[r\r]r^r_r`rarbr where_clausesrinclude_sourcesrclause clause_params where_sqlbase_where_params prompt_select prompt_join id_needle search_needle outer_where id_paramsfilter_clausesrqcompact_needle compact_sql search_clausecombined_selrrrrrr[rsseen_ids pinned_where pinned_query pinned_cursor pinned_rowstip_ids_by_roottip_idtip_rows projectedtip_rowrrps: r9list_sessions_richzSessionDB.list_sessions_richs T !!!  U  !4 5 5 5  $78H$I$I!S!S!S T T T&,E6(($w}"2E2E  +88#A#A#A#A#AAAL  !@!@!@!@ A A A MM/ * * *  '  !4 5 5 5 MM+ & & &  +88#A#A#A#A#AAAL  !D\!D!D!D E E E MM/ * * *  )$6z$B$B !FM   ( ( ( MM- ( ( ( q  !7 8 8 8 MM+ , , ,  3  !1 2 2 2 2! 3  !1 2 2 2>KS:W\\-88:::QS !LL UBBT   QBBP ^**,,2244 %+2244::<< { +$K#%I(*N &c &c & & & &  ; %%I   y!9!9::: <"$ 2}!E!E> >  -- ">">!?!!CDDD!D!R{11*==RRRM$$]]>%B%BCCC%%md&:;;; "<<775>Wy11x111DWXDWDW4@J4--///UD$7@$$  ?Nr<rz& is being compressed by another writerrrrr)rr rrr)rrrr  active_locksessions r9_check_transcript_write_guardsz(SessionDB._check_transcript_write_guardstsll 6  %   (**   #H%)@@@3O:OOO ,, D M   (**    #/ %66/ ;; ; //66r;rc|dS t|trtj|n|}t|trtj|}n7#tjt f$rt dYdSwxYwt|tst ddS|S)aDecode a ``display_metadata`` column into the dict every reader expects. Every message read path must go through this. Returning the raw TEXT instead reaches the desktop as a string, where ``'task_count' in meta`` throws and fails the whole resume. Rows written before the encode guard landed are double-encoded, so unwrap a second layer when we find one. Nz0Ignoring invalid display metadata on message rowz3Ignoring non-object display metadata on message row) rWrXrrrrErrr)rmetas r9_decode_display_metadataz"SessionDB._decode_display_metadatas ;4 &0c&:&:C4:c???D$$$ (z$''$i0    NNM N N N44 $%%  NNP Q Q Q4 sAA0BBrr` tool_calls tool_call_id token_count finish_reason reasoningreasoning_contentreasoning_detailscodex_reasoning_itemscodex_message_itemsplatform_message_idobservedeffect_disposition timestamp api_content display_kindc || rtj| nd| rtj| nd| rtj| ndt|tr4 tj|}n#tjtf$rg}YnwxYw|rtj|nd|tj |u t|dr"t| nt|n2#ttf$rtd|YnwxYwd|&t|t rt#|nd fd}|jS)u Append a message to a session. Returns the message row ID. Also increments the session's message_count (and tool_call_count if role is 'tool' or tool_calls is present). ``platform_message_id`` is the external messaging platform's own message ID (e.g. Telegram update_id, Yuanbao msg_id). It is independent of the SQLite autoincrement primary key and is used by platform-specific flows like yuanbao's recall guard to redact a message by its platform-side identifier. ``api_content`` is the exact content string sent to the API for this message when it differs from ``content`` (ephemeral memory/plugin injections, persist overrides). It is a byte-fidelity sidecar for prompt-cache-stable replay — stored as sent, except lone surrogates (which sqlite3 cannot bind and which the conversation loop scrubs from every outgoing payload anyway, so the scrubbed form IS the wire bytes). Nr/Ignoring invalid explicit message timestamp: %rrr?c||dt  tt rdnddttrtndttrtndf}|j} dkr|d fn|df|S)NINSERT INTO messages (session_id, role, content, tool_call_id, tool_calls, tool_name, effect_disposition, timestamp, token_count, finish_reason, reasoning, reasoning_content, reasoning_details, codex_reasoning_items, codex_message_items, platform_message_id, observed, active, api_content, display_kind, display_metadata) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)r?rzUPDATE sessions SET message_count = message_count + 1, tool_call_count = tool_call_count + ? WHERE id = ?zBUPDATE sessions SET message_count = message_count + 1 WHERE id = ?)rrrZrWrX lastrowid)rrmsg_idrcodex_items_jsoncodex_message_items_jsonr rdisplay_metadata_jsonrrmessage_timestampnum_tool_callsrrrrreasoning_details_jsonrrrstored_contentrrtool_calls_jsonr`s r9rz%SessionDB.append_message.._dosJ  / /j"9   \\] " #%i00&%!%i00%&788*$,'!(AAq6@c6R6R\%k222X\7A,PS7T7T^%l333Z^)+ F<%F!! M#Z0  XMMr;r)rrrrWrXrrrErr hasattrrrrFrrrrrr)rrrrr`rrrrrrrrrrrrrrrrr rrrrrrrrrs``` ` ````` ``` `` ` @@@@@@@@r9append_messagezSessionDB.append_messages\!% = =>N O O! +DJ( ) ) )&*  % /DJ, - - -*.  # -DJ* + + +(, ! j# & &  !Z 33 ()4     4>H$*Z000D--g66 IKK   [9k229(-i.A.A.C.C(D(D%%(-i(8(8%z* [ [ [ NPYZZZZZ [  !0::t0L0LSS___RSN0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 n"" D=#   s% B B;:B;AE,E21E2 chunk_rowsc sdS|^t|krKd}tdt|D](}||||zz })|Sfd}|jS)uAppend multiple messages atomically in ONE write transaction. ``messages`` is a list of dicts in the same shape :meth:`_insert_message_rows` already consumes for replace/compact/ import (role, content, tool_name, tool_calls, tool_call_id, finish_reason, reasoning*, codex_*, timestamp, api_content, display_kind, display_metadata, ...). Reusing that helper keeps ONE row-serialization path for every multi-row writer. A turn-boundary flush writes the whole turn (user + assistant + tool rows, typically 3-8 messages) as one BEGIN IMMEDIATE / commit pair instead of one transaction (and, off WAL, one fsync) per row. Atomicity contract: all rows land or none do (the caller re-flushes unstamped messages on the next attempt). The same admission guards as :meth:`append_message` run once for the batch — same session, same instant. ``chunk_rows`` bounds the transaction size for LARGE copies (branch seeds can be thousands of rows; measured: 10k rows ≈ 2.4s inside one BEGIN IMMEDIATE because the FTS triggers run per row, which would monopolize the write lock and starve concurrent writers). When set, the batch commits in chunks of at most that many rows — same recovery semantics as the old per-row loops (a mid-copy failure leaves a partial seed), just with bounded lock holds. A turn flush never needs it. Returns the inserted row count. rN)r c||\}}|dkr|d||fn|d|f|S)NrzUPDATE sessions SET message_count = message_count + ?, tool_call_count = tool_call_count + ? WHERE id = ?zBUPDATE sessions SET message_count = message_count + ? WHERE id = ?)rrr)rinsertedtool_calls_totalr rrrs r9rz,SessionDB.append_messages_batch.._dons  / /j"9   *.)B)Bj(** &H& !## M/<  Xz*Or;r)rrappend_messages_batchrr)rrrr rinserted_totalrrs```` r9rzSessionDB.append_messages_batch?sD 1  !c(mmj&@&@Nq#h--<<  $"<"<U5:#556,C#=## " !        ,"" D=#   r;)rcvrrsdSfd}t|S)a_Stamp presentation metadata on this turn's freshly persisted row. The model still receives ``role`` and ``content`` unchanged. Gateway and CLI synthetic inputs call this immediately after their serial turn has flushed, preserving producer provenance without classifying by content during transcript rendering. Fc |df}|dS|dt|dfdS)NzqSELECT id FROM messages WHERE session_id = ? AND role = ? AND content = ? AND active = 1 ORDER BY id DESC LIMIT 1FzGUPDATE messages SET display_kind = ?, display_metadata = ? WHERE id = ?rT)rrrrZr)rr[rrrrrrs r9rz?SessionDB.set_latest_matching_message_display_kind.._dos,,JT4#7#7#@#@Ahjj  {u LLY%l33112BCCF   4r;)rvr)rrrrrrrs`````` r9(set_latest_matching_message_display_kindz2SessionDB.set_latest_matching_message_display_kindsu    5          $D'',,---r; reactionsrauthormessage_row_idemojircTrdSfd}|S)apSet (or with ``emoji=None`` clear) *author*'s reaction on one message. iOS Tapback semantics: one reaction per author per message. Re-sending the same emoji clears it, a different emoji replaces it. Returns the message's full reaction list after the write, or ``None`` when the row doesn't exist or isn't part of *session_id*. Nc|d f}|dS |dpi}| j}fdt |t r|ngD}tfdt |t r|ngDd}duo|duo|dk}r9|s7|ttj d|r || j<n| jd|d|r |nd f|S)NESELECT display_metadata FROM messages WHERE id = ? AND session_id = ?rcpg|]2}t|tr|dk0|3SrrWrr_rzrrs r9rz?SessionDB.set_message_reaction.._do..sLa&&,-55??f+D+D+D+D+Dr;c3xK|]4}t|tr|dk0|V5dS)rNrrs r9rz>SessionDB.set_message_reaction.._do..sX!!T**01uuX&/H/H/H/H/H/Hr;r)rrat5UPDATE messages SET display_metadata = ? WHERE id = ?) rrrr_REACTIONS_METADATA_KEYrWrnextrrZr rr) rr[rrrprevious toggling_offrrrrrs r9rz+SessionDB.set_message_reaction.._dos,,W,hjj {t00Q88>BDxx ;<.s'<<)row_idrrrAr) rrrr_rrWrrrrrXr) rrpendingr[rrchangedreactionrrrrs r9rz,SessionDB.take_unseen_reactions.._dos<<  hjj G  44S9K5LMM HHT%@AA !)T22 )H&x66!#<<11V;;#<<//<!'+HV$"G"223y>BBGNN&)$i$'K%-\\'%:%:%@b/9'3/G/G$OGGR LLO66t<rrrNz0AND content IS NOT NULL AND TRIM(content) != '' r>zISELECT id FROM messages WHERE session_id = ? AND role = ? AND active = 1 z!ORDER BY id DESC LIMIT 1 OFFSET ?r9r#rrCr)rrrrXr text_filterr[s r9latest_message_row_idzSessionDB.latest_message_row_id>s$ T)>>>&1**4CO V > >TV Z  *$$Q"-QQQT3v;;/hjj                &s1vv$&sAA**A.1A.c0||dS)aRow id of the most recent active user message, or ``None``. The agent's default reaction target: "the message that triggered me", so the model never has to thread row ids through a tool call (mirrors the photon adapter's ``_record_last_inbound``). r)r)r)rrs r9latest_user_message_row_idz$SessionDB.latest_user_message_row_id`s))*6)BBBr;rc|sdS|j5|jdt||f}dddn #1swxYwY|r|dndS)zRole of the active message at *row_id* in *session_id*, or ``None``. Lets a reaction event carry the target's role so a renderer can match a live message that doesn't know its durable row id yet. NzHSELECT role FROM messages WHERE id = ? AND session_id = ? AND active = 1rr)rrrr[s r9get_message_rolezSessionDB.get_message_roleis  4 Z  *$$ZVj)hjj                 &s1vv$&s=AAAc~tj}d}d}|D]}|dd}|d} |} |d |d} t| dr"t| } nt| } nE#t t f$r1td|dYnwxYw|dkr|d nd} |dkr|d nd} |dkr|d nd}| rtj | nd}| rtj | nd}|rtj |nd}t| tr4 tj | } n#tjt f$rg} YnwxYw| rtj | nd}|d p|d }|d}|d||||d|d|t#|d|d| |d|d|dkr"t#|dnd|dkr"t#|dnd|||||drdnddt|trt#|ndt|dtr"t#|dnd||df|dz }| )|t| t&rt)| ndz }t+|dz| dz}!||fS)uInsert *messages* as fresh active rows for *session_id*. Shared by :meth:`replace_messages` (delete-then-insert) and :meth:`archive_and_compact` (soft-archive-then-insert). Runs inside the caller's write transaction (takes the live ``conn``). Returns ``(inserted_count, tool_call_count)``. Does NOT touch sessions.* counters — the caller owns that, since the two flows reconcile counts differently. rrrrrNrrrrrr message_idrrrrr`rrrrrrr?rrgư>)r r_rrrrErFrrrrrWrXrrrrrZrrrr)rrrrnow_tsrrrrrrts_valuerrrrrrrplatform_msg_idrs r9rzSessionDB._insert_message_rowszsvR BR BC7769--D..J & ww{##/j"ww{33Hx55<,1(2D2D2F2F,G,G)),1(OO)!:.jjjLL!RTWT[T[\gThThiiiiij@D @S@S(; < < >>pt11#'':L2M2MNN+    < MH% '1*d'C'CJC OOO $(9D(@AAFF)))s% AB77?C98C91GG! G!cT|rdndfd}|dS)uyAtomically replace the stored messages for a session. Used by transcript-rewrite flows such as /retry, /undo, and /compress. The delete + reinsert sequence must commit as one transaction so a mid-rewrite failure does not leave SQLite with a partial transcript. DESTRUCTIVE by default: every row for the session is DELETEd (and drops out of the FTS index). For compaction that must preserve the pre-compaction transcript under the same id, use :meth:`archive_and_compact` instead. Pass ``active_only=True`` to replace ONLY the live (``active = 1``) rows, leaving soft-archived rows (``active = 0`` — e.g. the ``compacted = 1`` turns that :meth:`archive_and_compact` keeps on disk for #38763 durability, or rewind/undo rows) untouched. Callers that share a session id with an agent already running in-place compaction must use this so a full-history rewrite doesn't wipe the rows the agent deliberately archived. ``message_count``/``tool_call_count`` then track the live set, matching :meth:`archive_and_compact`.  AND active = 1r>cl|df}|#|d|ddkrt|df|df|\}}|d||fdS)Nrrrr)DELETE FROM messages WHERE session_id = ?GUPDATE sessions SET message_count = 0, tool_call_count = 0 WHERE id = ?r)rrrr)rrrr active_clauserrrs r9rz'SessionDB.replace_messages.._dosllH hjj  #J'3L)]::3J??? LLKMKK     LLY    04/H/Hj(00 ,N, LLY!1:>     r;Nr)rrrrrrs``` @r9replace_messageszSessionDB.replace_messagess\6.9@))b         6 C     r;c|j5|jd|f}|ducdddS#1swxYwYdS)u?Return True if the session has any soft-archived (``active = 0``) rows. Used by callers (e.g. the ACP adapter's ``_persist``) that must decide whether a full-history :meth:`replace_messages` would destroy durable compaction-archived turns. Cheap existence probe — does not load rows. zBSELECT 1 FROM messages WHERE session_id = ? AND active = 0 LIMIT 1Nr4rrrs r9has_archived_messageszSessionDB.has_archived_messagessZ 1 1Z''T F??$$D0  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1s2AA A compacted_messagesc@fd}|S)ujNon-destructive in-place compaction for a single durable session id. Soft-archives every currently-active message (``active = 0``) and inserts *compacted_messages* as fresh active rows — atomically, in one write transaction. The conversation keeps ONE session id for life (#38763) WITHOUT destroying history: - The live-context load (:meth:`get_messages_as_conversation`, :meth:`get_messages`) filters ``active = 1`` by default, so the model reloads ONLY the compacted set. - The archived pre-compaction turns stay on disk (active=0) and stay DISCOVERABLE: they are marked compacted=1, and search_messages() includes compacted=1 rows by default — so session_search still finds them, unlike rewind/undo rows (active=0, compacted=0) which stay hidden. They remain in the FTS index (the messages_fts* triggers index on INSERT / drop on DELETE and don't key on active/compacted; flipping to active=0 is a content-preserving UPDATE) and are recoverable via get_messages(..., include_inactive=True). This is the durability-preserving alternative to :meth:`replace_messages` for compaction. ``message_count`` is set to the ACTIVE (compacted) count, matching what the live load returns. Returns the new active count. c|df|\}}|d||f|S)NzQUPDATE messages SET active = 0, compacted = 1 WHERE session_id = ? AND active = 1r)rr)rrrr rrs r9rz*SessionDB.archive_and_compact.._do>sw LL6     *.)B)Bj"4** &H& LLY+Z8   Or;r)rrr rs``` r9archive_and_compactzSessionDB.archive_and_compact#s>6       .""3'''r;cj||fd}||S)uBackfill the ``api_content`` sidecar onto the newest ACTIVE user row. In-place preflight compaction (:meth:`archive_and_compact`) inserts the current turn's user row BEFORE the turn prologue composes the prefetch/plugin sidecar, and the subsequent crash persist identity-skips every compacted dict — without this backfill the stamped sidecar would never land in the DB and any reload would replay clean content, re-introducing the prompt-cache divergence the sidecar exists to close. The ``content`` match is a defensive guard: if the newest active user row is not the message the caller stamped (racing rewrite, unexpected tail shape), nothing is written. Returns the number of rows updated (0 or 1). c^|dtf}|jS)NzUPDATE messages SET api_content = ? WHERE id = (SELECT id FROM messages WHERE session_id = ? AND role = 'user' AND active = 1 ORDER BY id DESC LIMIT 1) AND content IS ?)rrZr)rrrencodedrs r9rz2SessionDB.set_latest_user_api_content.._dojs9\\% #;//WE F? "r;)rr)rrrrrrs ` ` @r9set_latest_user_api_contentz%SessionDB.set_latest_user_api_contentWsS"&&w// # # # # # # #""3'''r;include_inactivec|rdnd}d|d}|g}||r |dz }||dn||g|5}|||} | } dddn #1swxYwYg} | D]} t | } d| vr|| d| d<| d rZ tj| d | d <n;#tj tf$r"t d g| d <YnwxYw| d | | d | d <| | | S) u=Load messages for a session in insertion order. By default only active messages are returned. Pass ``include_inactive=True`` to load soft-deleted rows (e.g. for audit / debug views of rewound history). See :meth:`rewind_to_message` for the soft-delete mechanic. Ordered by AUTOINCREMENT id (true insertion order) rather than timestamp — see c03acca50 for the WSL2 clock-regression rationale. When ``limit`` is provided, returns at most ``limit`` messages starting from ``offset`` (0-based, in insertion order). Enables pagination for the API endpoint to avoid loading entire transcripts. ``offset`` alone (without ``limit``) also pages — SQLite requires a LIMIT clause for OFFSET, so it's emitted as ``LIMIT -1`` (unbounded). r>rz+SELECT * FROM messages WHERE session_id = ? ORDER BY idNz LIMIT ? OFFSET ?r!rrzDFailed to deserialize tool_calls in get_messages, falling back to []r)rrWrrrrr_rrrrErrrr)rrrrWrXrr_rrrrrr[rs r9 get_messageszSessionDB.get_messagesws./E4E  + + + + #|    & &C MM22E6B C C C ^^   %\\#v..F??$$D % % % % % % % % % % % % % % %  Cs))CC!%!5!5c)n!E!EIww|$$ ++(, 3|3D(E(EC %%,i8+++NN#ijjj(*C %%%+ww)**6*.*G*GL^H_*`*`&' MM#     s$+A??BBC555D-,D-around_message_idwindowc8|dkrd}|5}|d||f}|sgdddcdddS|d|||dzf}|d|||f}dddn #1swxYwYt t |t |z}g} |D]} t | } d| vr|| d| d<| d rZ tj | d | d <n;#tj tf$r"td g| d <YnwxYw| d || d | d <| | t#dt%|dz } t%|} | | | dS) uLoad a window of messages anchored on a specific message id. Returns a dict with: - ``window``: up to ``window`` messages before the anchor, the anchor itself, and up to ``window`` messages after, ordered by id ascending. - ``messages_before``: count of messages strictly before the anchor still in the session (== window unless we hit the start). - ``messages_after``: count of messages strictly after the anchor still in the session (== window unless we hit the end). Used by ``session_search`` for both the discovery shape (anchored on the FTS5 match) and the scroll shape (anchored on any message id). The ``messages_before`` / ``messages_after`` counts let the caller detect session boundaries: when either is less than ``window``, the agent has reached one end of the session. Returns an empty window when ``around_message_id`` is not a real id in ``session_id`` — callers decide how to surface that. rz>SELECT 1 FROM messages WHERE id = ? AND session_id = ? LIMIT 1)rmessages_beforemessages_afterNzPSELECT * FROM messages WHERE session_id = ? AND id <= ? ORDER BY id DESC LIMIT ?r?zNSELECT * FROM messages WHERE session_id = ? AND id > ? ORDER BY id ASC LIMIT ?rrzKFailed to deserialize tool_calls in get_messages_around, falling back to []r)rWrrrrreversedrrr_rrrrErrrrrr)rrrrr anchor_exists before_rows after_rowsrrr[rrrs r9get_messages_aroundzSessionDB.get_messages_arounds2 A::F ^^    LLP"J/hjj ! Q"$aPP        ,,+. ;  hjj  *.7  hjj #               2H[))**T*-=-==  Cs))CC!%!5!5c)n!E!EIww|$$ ++(, 3|3D(E(EC %%,i8+++NNe)+C %%% + ww)**6*.*G*GL^H_*`*`&' MM#    a[!1!1A!566Z.,   s*2CACCCE##5FFc|s|S ||}n#t$r|}YnwxYw|r||kr|}|j5|}|h}d}tdD]} |jd|f}n #t$r|cYccdddSwxYw||} |jd|f}n #t$r|cYccdddSwxYw|n@t|dr|dn|d} | r| |vrn|| | }||n|cdddS#1swxYwYdS)uRedirect a resume target to the descendant session that holds the messages. Context compression ends the current session and forks a new child session (linked via ``parent_session_id``). The flush cursor is reset, so the child is where new messages actually land — the parent ends up with ``message_count = 0`` rows unless messages had already been flushed to it before compression. See #15000. This helper walks ``parent_session_id`` forward from ``session_id`` and returns the descendant in the chain that has the **most recent** messages. Unlike the original logic, it does NOT short-circuit when the starting session already has messages — a descendant that was created by compression may hold the continuation content and should be preferred by the WebUI and gateway for ``--resume`` and session loading. If no descendant (including the starting session) has any messages, the original ``session_id`` is returned unchanged. The chain is always walked via the child whose ``started_at`` is latest; that matches the single-chain shape that compression creates. A depth cap (32) guards against accidental loops in malformed data. N z3SELECT 1 FROM messages WHERE session_id = ? LIMIT 1aSELECT id FROM sessions WHERE parent_session_id = ? AND json_extract(COALESCE(model_config, '{}'), '$._branched_from') IS NULL AND json_extract(COALESCE(model_config, '{}'), '$._delegate_from') IS NULL AND COALESCE(source, '') != 'tool' ORDER BY started_at DESC, id DESC LIMIT 1keysrr) rRrKr9rr#rrrr) rrtiprrPbestrr[ child_rowrQs r9resolve_resume_session_idz#SessionDB.resolve_resume_session_ids.   **:66CC   CCC   3*$$J Z+ <+ < G9DD2YY$ #$ #&*,,M hjjC!&&&%%%%%+ <+ <+ <+ <+ <+ <+ <+ <&?"D & $ 2 2D ! !!hjjI!&&&%%%%%E+ <+ <+ <+ <+ <+ <+ <+ rr}c3K|]}dVdSrlrfrms r9rz9SessionDB.get_messages_as_conversation..t"#=#=AC#=#=#=#=#=#=r;zSELECT $ FROM messages WHERE session_id IN (rrNrr*r+r,_session_lineage_root_to_tiprWrr_CONVERSATION_ROW_COLUMNSrr_rows_to_conversation) rrr*rr+r, session_idsrrrrs r9get_messages_as_conversationz&SessionDB.get_messages_as_conversationTsI4"l  H;;JGGK.E4E ^^   88#=#=#=#=#===L<< /$8 / /6B / /! / / /k""  hjj                ")) !/1+ *   sA$B%%B),B)a id, role, content, tool_call_id, tool_calls, tool_name, effect_disposition, finish_reason, reasoning, reasoning_content, reasoning_details, codex_reasoning_items, codex_message_items, platform_message_id, observed, timestamp, api_content, display_kind, display_metadata)r,cg}|D]7}||d}|ddvr6t|tr!t|}|d|d} |r|d |d| d<|dr |d| d<|d r |d | d <|d r"||d } | | | d <|d r |d | d <|d r |d | d <|d r |d | d <|dr |d| d<|drZ t j|d| d<n;#t jtf$r"t dg| d<YnwxYw|dr |d| d<|drd| d<|ddkr_|dr |d| d<|dr |d| d<|d |d| d<|drZ t j|d| d<n;#t jtf$r"t dd| d<YnwxYw|drZ t j|d| d<n;#t jtf$r"t dd| d<YnwxYw|drZ t j|d| d<n;#t jtf$r"t dd| d<YnwxYw|r| || r"| | 9t|}|r2|r0dd lm} | d|} | rtd!| ||S)"apDecode fetched message rows into the OpenAI conversation format. Extracted from get_messages_as_conversation so get_resume_conversations can build the model-fed and display views from one SELECT. ``rows`` must already be ordered by ``id`` (insertion order) and filtered to the desired session set / active state by the caller. rr>rr)rrrN_row_idrrrrrr`rrzKFailed to deserialize tool_calls in conversation replay, falling back to []rrrTrrrrrz=Failed to deserialize reasoning_details, falling back to NonerzAFailed to deserialize codex_reasoning_items, falling back to Nonerz?Failed to deserialize codex_message_items, falling back to Noner)repair_message_sequenceuRepaired %d message-alternation violation(s) while restoring session %s — durable transcript kept them, see repair_message_sequence)rrWrXrr`rrrrrErr#_is_duplicate_replayed_user_messagerragent.agent_runtime_helpersr:r) rrrr*r+r,rr[rrdecodedr:rs r9r5zSessionDB._rows_to_conversations P !P !C**3y>::G6{333 7C8P8P3*73399;;v;7;;C +3t9#8!$TI=! 8%(%7M">" :&).&9N#%& 677" :&).&9N#; 4#&{#3K '( F,/0D,E()<  ++(, 3|3D(E(EC %%,i8+++NN#pqqq(*C %%%+() ?$'(=$>L!: '"&J6{k))'@+.+?C({#8'*;'7C $*+7/23F/GC+,*+8837:cBU>V3W3W/00 0)<888'fggg37/0008./<<7;z#F]B^7_7_344 0)<<<<'jkkk7;3444<,-::59ZDY@Z5[5[122 0)<:::'hiii591222:! T%M%MhX[%\%\  OOC 4H==  (  L K K K K K..tX>>H  2 sH(E5E>=E>0H5III005J('J(4K5L  L c|}|5}dd|D}|d|jd|dt |}dddn #1swxYwYfd|D}||dd d }||d dd }||fS) uReturn ``(model_history, display_history)`` for a session resume in ONE SELECT. ``session.resume`` needs two projections of the same lineage: - ``model_history`` — the tip session's active rows, alternation-repaired (the live-replay working conversation). Equivalent to ``get_messages_as_conversation(session_id, repair_alternation=True)``. - ``display_history`` — the full lineage (ancestors → tip), verbatim, with replayed-user dedup. Equivalent to ``get_messages_as_conversation(session_id, include_ancestors=True)``. The display fetch already reads a superset of the model fetch (the tip rows are part of the lineage), so serving both from one lineage SELECT halves the resume's DB work versus two separate calls, with byte-identical output (see test_get_resume_conversations_matches_separate_reads). r}c3K|]}dVdSrlrfrms r9rz5SessionDB.get_resume_conversations..)r/r;SELECT session_id, r0) AND active = 1 ORDER BY idNc,g|]}|dk|Srrfrzrrs r9rz6SessionDB.get_resume_conversations..6s'EEE!q*'D'DA'D'D'Dr;FTr1r2) rrr6rrrr model_historydisplay_historys ` r9get_resume_conversationsz"SessionDB.get_resume_conversationssk&77 CC ^^   88#=#=#=#=#===L<<d&D6B k"" hjj                FEEEtEEE22 !## 3  44 !"$ 5  o--sA!BBBc|}t|dkrgS|5}dd|D}|d|jd|dt |}dddn #1swxYwYfd|D}|sgS||d d S) uGReturn the ancestor-only display messages for a session lineage. These are messages from parent/grandparent sessions (compression ancestors) that appear in the display transcript but NOT in the tip session's model-fed history. Used by ``session.resume`` to build the ``display_history_prefix`` that ``_live_session_payload`` prepends to the live model history. Previously the prefix was calculated as ``display_history[:len(display) - len(raw)]``, but that overcounts when ``repair_message_sequence`` removes messages from the MIDDLE of the tip history (e.g. verification candidates collapsed by the consecutive-assistant merge) — the length difference includes both ancestor messages AND repair-removed tip messages, but the slice only captures the first N display messages (which are tip messages when there are no ancestors), causing duplication. This method returns ONLY the genuine ancestor messages, identified by ``session_id != tip_session_id``. (#65919) r?r}c3K|]}dVdSrlrfrms r9rz8SessionDB.get_ancestor_display_prefix.._r/r;r@r0rANc,g|]}|dk|SrCrfrDs r9rz9SessionDB.get_ancestor_display_prefix..fs'JJJqAlOz,I,I,I,I,Ir;TF)rr*r+) r3rrWrrr4rrr5)rrr6rrr ancestor_rowss ` r9get_ancestor_display_prefixz%SessionDB.get_ancestor_display_prefixGsX(77 CC {  q I ^^   88#=#=#=#=#===L<<d&D6Bk""  hjj                KJJJDJJJ  I)) !"$ *   sA!B--B14B1cT||}|r|dr|dn|S)u'Return the ROOT id of *session_id*'s lineage chain. The root is the stable "conversation id": context compression rotates ``session_id`` to a new segment linked via ``parent_session_id``, and delegate subagents hang off their parent the same way. Walking to the root gives every segment of one user-facing conversation (and its delegation tree) a single identifier — used for Nous Portal ``conversation=`` usage tagging. Returns *session_id* unchanged when it has no recorded parent. r)r3)rrchains r9get_conversation_rootzSessionDB.get_conversation_rootps311*==!>eAh>aJ?r;c|s|gSg}|}t}|5}tdD]}|r||vrnx|||||d|f}|n!t|dr|dn|d}dddn #1swxYwYtt|p|gS)Nrz3SELECT parent_session_id FROM sessions WHERE id = ?r%rr) rrWrrrrrrrr)rrrNrrPrrr[s r9r3z&SessionDB._session_lineage_root_to_tip~sa < uu ^^   W3ZZ W W'T//E!!! W%%%llIJ(**;E6=c66J6JV#122PSTUPV W W W W W W W W W W W W W W WHUOO$$4 4sBC  CCrc|ddkrdS|d}t|tr|sdSt|D]}}|ddkr|d|krdS|ddkr-|ds|drdS~dS)NrrFrTrr)r_rWrXr)rrrprevs r9r;z-SessionDB._is_duplicate_replayed_user_messages 776??f $ $5'')$$'3'' w 5X&&  Dxx6))dhhy.A.AW.L.Lttxx;..DHHY4G4G.488T`KaKa.uuur;target_message_idc|j5|jdf}dddn #1swxYwY|t ddt |}|ddkr)t d|ddd ||d |d <g}fd }||}|j5|jd f}dddn #1swxYwY|r|d |d nd}t|||dS)uSoft-delete all messages with id >= ``target_message_id`` in *session_id*. The target message itself becomes inactive as well so the caller can pre-fill it as the next user prompt without it appearing twice in the replayed transcript. Rewound rows are kept on disk with ``active=0`` for audit / forensic inspection — use :meth:`get_messages` with ``include_inactive=True`` to see them. Returns a dict:: { "rewound_count": int, # number of rows newly flipped to active=0 "target_message": dict, # full row dict of the target "new_head_id": int|None # id of the last still-active row, or None } Raises ``ValueError`` if the target message does not exist in *session_id* or if its role is not ``"user"``. Always increments ``sessions.rewind_count`` — even when the target is already inactive — so the counter accurately reflects the number of rewind operations performed against the session. Idempotent on the ``active`` flag: re-rewinding past the same target is a no-op on row state but still bumps the counter. z6SELECT * FROM messages WHERE id = ? AND session_id = ?Nzmessage z not found in session rrz1rewind target must be a 'user' message (got role=z, id=rrc|df}d|D}|r9dd|D}|d|d||df|S)NzGSELECT id FROM messages WHERE session_id = ? AND id >= ? AND active = 1cg|] }|d Srjrfrs r9rz._do..333A1Q4333r;r}c3K|]}dVdSrlrfrms r9rz;SessionDB.rewind_to_message.._do.."'9'9'9'9'9'9'9'9r;z,UPDATE messages SET active = 0 WHERE id IN (rzMUPDATE sessions SET rewind_count = COALESCE(rewind_count, 0) + 1 WHERE id = ?)rrr)rrrrrrSs r9rz(SessionDB.rewind_to_message.._dos\\B./F 43!2!2333C "xx'9'9S'9'9'999  R<RRR LL     Jr;z@SELECT MAX(id) FROM messages WHERE session_id = ? AND active = 1r) rewound_counttarget_message new_head_id) r9r#rrrFrr_rrr) rrrSr[ target_rowrewoundrhead_rowr\s `` r9rewind_to_messagezSessionDB.rewind_to_messagesS<Z  *$$H"J/hjj                 ;P,PPJPP #YY >>& ! !V + +G>>&))GG2CGGG  !% 4 4Z^^I5N5N O O 9      (%%c**Z  z))R hjj                 &.S(1+2Ihqkkt !\\(&   s#0AA  A /EE Esince_message_idc<fd}||S)zMark inactive messages with id >= *since_message_id* active again. Returns the number of rows flipped back to ``active=1``. Intended for undo-of-rewind and test cleanup; not wired to a slash command in v1. c|df}d|D}|r9dd|D}|d|d|t|S)NzGSELECT id FROM messages WHERE session_id = ? AND id >= ? AND active = 0cg|] }|d Srjrfrs r9rz:SessionDB.restore_rewound.._do.. rWr;r}c3K|]}dVdSrlrfrms r9rz9SessionDB.restore_rewound.._do.. rYr;z,UPDATE messages SET active = 1 WHERE id IN (r)rrrr)rrrrrras r9rz&SessionDB.restore_rewound.._dos\\B-.F 43!2!2333C "xx'9'9S'9'9'999  R<RRRs88Or;r)rrrars `` r9restore_rewoundzSessionDB.restore_rewounds8      ""3'''r;rbc.dtdd}g}g}|r*|d|||rzI ORDER BY last_active DESC, s.started_at DESC, s.id DESC LIMIT ? OFFSET ?c:g|]}|Srfr)rzr[rs r9rz-SessionDB.search_sessions..Ds'MMMCD**3//MMMr;N) rrrurrr9r#rr) rrrWrXrbselect_with_last_activerr ws_clause ws_paramsrrs ` r9search_sessionszSessionDB.search_sessionss& M',, M M M    "   0 0 0 MM& ! ! !  %#8#G#G Iy   + + + MM) $ $ $?LT;gll=99;;;RT  ufo&&& Z N NZ''*[[[[ F NMMM6??;L;LMMM N N N N N N N N N N N N N N N N N Ns=AD  DDexclude_childrenc g} g} |r?| t| tdd|r|gnt|pg} | rMdd| D} | d| d| | |rMdd|D} | d| d| ||r.nrgr;rhrc3K|]}dVdSrlrfrms r9rz*SessionDB.session_count..rrgr;rjrrkrlrmrrnr>zSELECT COUNT(*) FROM sessions sN) rrrgrrrror9r#rr)rrrUrhrZr]r^rnrVrrrrrrrrs r9 session_countzSessionDB.session_countJs2   U  !4 5 5 5  $78H$I$I!S!S!S T T T&,E6(($w}"2E2E  +88#A#A#A#A#AAAL  !@!@!@!@ A A A MM/ * * *  +88#A#A#A#A#AAAL  !D\!D!D!D E E E MM/ * * *  )$6z$B$B !FM   ( ( ( MM- ( ( ( q  !7 8 8 8 MM+ , , ,  3  !1 2 2 2 2! 3  !1 2 2 2?LT;gll=99;;;RT Z ( (Z''(U)(U(UW]^^F??$$Q' ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (s:8G??HHr?rc|j5|jd|f}|}dddn #1swxYwYt ||kS)uCheck if at least N sessions exist (archived included). Short-circuits via LIMIT — much cheaper than ``session_count()``, which pays a full index scan for its default ``archived = 0`` filter (measured 543us vs 4us on a 20k-session DB). Archived sessions count: every caller so far asks "has this install ever had sessions", and an archived session is still a created one. Use this instead of ``session_count() >= n`` when the exact count is irrelevant. zSELECT 1 FROM sessions LIMIT ?N)r9r#rrr)rrrrs r9session_count_gezSessionDB.session_count_gesZ % %Z''(H1$OOF??$$D % % % % % % % % % % % % % % %4yyA~r6)r]r^rncg}g}|r?|t|tdd|r|dn|s|d|rdd|nd}|j5|jt d |jd |d |}dddn #1swxYwYd |DS) aReturn a ``{source: count}`` dict via a single ``GROUP BY`` query. Replaces the O(N) ``list_sessions_rich`` histogram loop with an aggregate query. When ``exclude_children`` is False the query uses ``idx_sessions_source``; when True, the child-exclusion predicates require a full table scan (same as ``session_count`` and ``list_sessions_rich``). ``exclude_children=True`` mirrors ``list_sessions_rich`` visibility (roots + branch sessions, excluding sub-agent runs, delegates, and compression continuations) so the source counts match what the Sessions page actually lists. rdrerlrmrrnr>NzSessionDB connection is closedzYSELECT COALESCE(NULLIF(s.source, ''), 'cli') AS source, COUNT(*) AS count FROM sessions szC GROUP BY COALESCE(NULLIF(s.source, ''), 'cli') ORDER BY count DESCcdi|]-}t|dt|dpd.S)rr^r)rXrCrs r9rz5SessionDB.session_count_by_source..s7KKKsCM""CG (9$:$:KKKr;) rrrgrr9r#rrr)rr]r^rnrrrrs r9session_count_by_sourcez!SessionDB.session_count_by_sources(   U  !4 5 5 5  $78H$I$I!S!S!S T T T  3  !1 2 2 2 2! 3  !1 2 2 2?LT;gll=99;;;RT Z  z!"#CDDD:%%&"+&&&  hjj                LKdKKKKsAC,,C03C0c|j5|r|jd|f}n|jd}|dcdddS#1swxYwYdS)z2Count messages, optionally for a specific session.z2SELECT COUNT(*) FROM messages WHERE session_id = ?zSELECT COUNT(*) FROM messagesrNr4r s r9rxzSessionDB.message_counts Z ( ( M++H:-++,KLL??$$Q' ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (sAA((A,/A,c|j5|jd||f}|ducdddS#1swxYwYdS)abCheck if a message with the given platform_message_id exists. Uses the idx_messages_platform_msg_id partial index for efficient lookup. Used by the gateway's transient-failure dedupe guard (#47237) to skip re-persisting a user message that was already saved on a prior retry of the same inbound platform message. zOSELECT 1 FROM messages WHERE session_id = ? AND platform_message_id = ? LIMIT 1Nr4)rrrrs r9has_platform_message_idz!SessionDB.has_platform_message_idsZ 1 1Z''K01F ??$$D0  1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1s3AA A rc |d}|sdS t|trtj|n|}n#t tjf$rYdSwxYwt|to|dduS)NrcF_branched_from)r_rWrXrrrErr)rrrr-s r9_is_branch_child_rowzSessionDB._is_branch_child_rowskk.)) 5 %/S%9%9B$*S///sCC4/0   55 #t$$N1A)B)B$)NNs+AA! A!rc|d}|r||rdS||}t|o|ddkS)NrFrr)r_r}rrv)rr parent_idrws r9_is_compression_child_rowz#SessionDB._is_compression_child_rowsmII122  D55e<< 5!!),,FHvzz,77=HIIIr;c||}|r||r|r|gngS|}|dh}||rZ||d}|r |d|vrn2|}||d||Z|dg}|dh}|}|ddkr|j5|jd|df} dddn #1swxYwYd} | D]*} t| } || s| } n+| r | d|vrn^| | d|| d| }|d|kr|ddk||vr|n|gS)z@Return compression ancestors through tip in chronological order.rrrrz SELECT * FROM sessions WHERE parent_session_id = ? ORDER BY started_at ASC N) rr}rrr_r9r#rrrr) rrrra ancestorsrwlineagerPrr next_childr[ candidates r9get_compression_lineagez!SessionDB.get_compression_lineageso"":.. 3$33G<< 3#*2J<< 2$ZL ,,T22 &%%d+>&?@@F VD\Y66D MM$t* % % % ,,T22 &:,T |kk,''=88  z)) T]$ (**               J   II 00;;!*JE D!1T!9!9 NN:d+ , , , HHZ% & & & Gt} **1kk,''=882%//wwj\As"5D##D'*D'c<fd}||dS)z9Delete all messages for a session and reset its counters.cd|df|dfdS)Nrrrr!s r9rz%SessionDB.clear_messages.._do%sJ LL;j]    LLY      r;Nrr"s ` r9clear_messageszSessionDB.clear_messages#s8      C     r;c"|dSdD]2}|||z } |d##t$rY/wxYw |d|dD])} |d#t$rY&wxYwdS#t$rYdSwxYw)aHRemove on-disk transcript files for a session. Cleans up ``{session_id}.json``, ``{session_id}.jsonl``, and any ``request_dump_{session_id}_*.json`` files left by the gateway. Silently skips files that don't exist and swallows OSError so a filesystem hiccup never blocks a DB operation. N)z.jsonz.jsonlT) missing_ok request_dump_z_*.json)unlinkrPglob)rrrWrs r9rzSessionDB._remove_session_files/s   F)  F*6f666A D))))     !&&'Jz'J'J'JKK  HHH----D       DD sA* 77BA.-B. A;8B:A;;B B Bc |j5|jd|f}|sgcdddSt |j|g}dddn #1swxYwY|gt |S)aEReturn every session row that :meth:`delete_session` would remove. The requested session is first, followed by its recursively discovered delegate/subagent children. Branch and compression children are not included because deletion preserves them by orphaning their parent reference. +SELECT 1 FROM sessions WHERE id = ? LIMIT 1N)r9r#rrrsorted)rrrT delegate_idss r9get_session_delete_targetsz$SessionDB.get_session_delete_targetsJsZ Q QZ''= }hjj    Q Q Q Q Q Q Q Q 7tzJ<PPL  Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q2VL1122s2A)A))A-0A-expected_delete_idscg|t|ndfd}|}|r1D]}|||t|S)uDelete a session and all its messages. Delegate subagent children (``model_config._delegate_from``) are cascade-deleted with the parent so they never resurface in session pickers as orphaned rows. Branch / compression children are orphaned (``parent_session_id → NULL``) so they remain accessible independently. When *sessions_dir* is provided, also removes on-disk transcript files (``.json`` / ``.jsonl`` / ``request_dump_*``) for every deleted session. When *expected_delete_ids* is provided, deletion proceeds only if the parent plus delegate cascade still matches that exact set. This lets export-before-delete callers fail closed if a new delegate appears after they materialize their archive. The delegate tree is re-walked inside the write transaction on purpose (TOCTOU guard); the cost is accepted for correctness. Returns True if the session was found and deleted. Nc|df}|dSht|g}|krdSt |g|df|df|df|dS)NrFzHUPDATE sessions SET parent_session_id = NULL WHERE parent_session_id = ?r!DELETE FROM sessions WHERE id = ?T)rrrrrr)rr actual_ids expected_idsremoved_delegate_idsrrs r9rz%SessionDB.delete_session.._dovs\\= }F  (u'0 |DD -- 5 ' '(A$ (U(U V V V LL.     LLDzm T T T LL._dosR\\  F""88>>>?Q& &r;)rrrv)rrrrrs`` r9delete_session_if_emptyz!SessionDB.delete_session_if_emptysb& ' ' ' ' ' '(%%c**  A  & &|Z @ @ @G}}r;r6c|sdStd|DsdSggfd}|}D]}||D]}|||S)uDelete every session in *session_ids* in a single transaction. Backs the dashboard's bulk-select-then-delete flow on the sessions page (``POST /api/sessions/bulk-delete``). Mirrors the single-session :meth:`delete_session` contract per row: * Unknown IDs are silently skipped (no 404) — selection state in the UI can race against another tab's delete, and we'd rather succeed-on-the-rest than fail-the-whole-batch. * Delegate subagent children (``model_config._delegate_from``) are cascade-deleted with their parent; branch children are orphaned (``parent_session_id → NULL``) so they stay accessible. * Messages and the session row both go in one ``_execute_write`` call so a partial failure can't leave the DB in a "messages gone but session row still there" state. * On-disk transcript / ``request_dump_*`` files are cleaned up outside the DB transaction when *sessions_dir* is provided, matching :meth:`prune_sessions` and :meth:`delete_empty_sessions`. Returns the count of sessions that actually existed and were deleted (may be less than ``len(session_ids)`` if some IDs were already gone). rc@h|]}t|t||Srf)rWrXrys r9r|z,SessionDB.delete_sessions..s-VVV3C9M9MVRUV3VVVr;cdddtz}|d|d}d|D}|sdSddt|z}t |||d|d||d|d||d |d|||t|S) Nr}r~z%SELECT id FROM sessions WHERE id IN (rcg|] }|d Srrfrs r9rz:SessionDB.delete_sessions.._do..s???cD ???r;rrrr)rrrrrrr) rrrrexisting_placeholdersrrr unique_idss r9rz&SessionDB.delete_sessions.._dosn88C#j//$9::L\\G GGGF@?V__->->???H q$'HHS3x==-@$A$A ! ' '(A$(Q(Q R R R LLH/DHHH    LLU=RUUU    LLM5JMMM     4 4T : : :   x ( ( (x== r;)rrr) rr6rrr^r{rrrs ` @@@r9delete_sessionszSessionDB.delete_sessionss: 1VV+VVVWW  1!# *," !" !" !" !" !" !" !" !H##C((' : :C  & &|S 9 9 9 9 : :C  & &|S 9 9 9 9 r;c|j5|jd}|dcdddS#1swxYwYdS)uReturn the count of empty, non-active, non-archived sessions. "Empty" = ``message_count = 0`` AND the session has ended (``ended_at IS NOT NULL``) AND is not archived. The ``ended_at`` guard matches the safety contract used by :meth:`prune_sessions`: only ended sessions are candidates for bulk deletion, so a freshly spawned session whose first message hasn't landed yet — or one held open by the live agent — is never sniped out from under the runtime. Backs the ``GET /api/sessions/empty/count`` endpoint that lets the web dashboard hide its "Delete empty" button when there's nothing to clean up, and pre-populate the confirm dialog with the actual count. z_SELECT COUNT(*) FROM sessions WHERE message_count = 0 AND ended_at IS NOT NULL AND archived = 0rNr4)rrs r9count_empty_sessionszSessionDB.count_empty_sessionss Z ( (Z''#F ??$$Q' ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (s4A  A A czgfd}|}D]}|||S)uJDelete every empty, ended, non-archived session. Mirrors :meth:`prune_sessions`' transactional shape: * Selects candidate IDs first (``message_count = 0`` AND ``ended_at IS NOT NULL`` AND ``archived = 0``) so we never touch a live session or one the user deliberately archived. * Orphans any child whose parent is in the kill list — children of an empty parent are kept and re-parented to ``NULL`` rather than cascade-deleted, matching ``delete_session`` / ``prune_sessions`` semantics so branch/subagent transcripts survive an inadvertent parent cleanup. * Deletes the rows in a single ``_execute_write`` callback so the operation is atomic — a partial failure (e.g. SIGKILL mid-loop) doesn't leave the DB in a "messages-deleted but session-row-still-there" half-state. * Cleans up on-disk transcript files (``.json`` / ``.jsonl`` / ``request_dump_*``) outside the DB transaction when ``sessions_dir`` is provided. Empty sessions don't typically have transcript files, but the gateway can leave a stub ``request_dump_*`` if it crashed before the first reply — so we still sweep, matching ``prune_sessions``. Returns the number of sessions deleted. c|d}d|D}|sdSddt|z}|d|dt ||D]E}|d|f|d |f|F|t|S) NzYSELECT id FROM sessions WHERE message_count = 0 AND ended_at IS NOT NULL AND archived = 0ch|] }|d Srrfrs r9r|z?SessionDB.delete_empty_sessions.._do..UBBB3t9BBBr;rr}r~rrrrrrrrrrr)rrr6rr{rrs r9rz,SessionDB.delete_empty_sessions.._doNs \\#F CB0A0ABBBK q88C#k*:*:$:;;L LL?/;???[!!    # ( (  ?# @3&III""3''''  4 4T : : :{## #r;)rr)rrrr^r{rs` @r9delete_empty_sessionszSessionDB.delete_empty_sessions/sl:"$  $ $ $ $ $ $@##C(( : :C  & &|S 9 9 9 9 r;)last_active_beforelast_active_afterstarted_before started_afterr title_likerrh min_messages max_messagesr7 model_likerrrr branch_like min_tokens max_tokensmin_costmax_costmin_tool_callsmax_tool_callsrrrrrrrrrrrrrrrcdg}g}|*|d|||*|d|||*|d|||*|d|||r*|d|||r@|d|d |d |r*|d |||r= ?zs.started_at < ?zs.started_at >= ?rhz#LOWER(COALESCE(s.title, '')) LIKE ?rzs.end_reason = ?rkzs.message_count <= ?z#LOWER(COALESCE(s.model, '')) LIKE ?z+LOWER(COALESCE(s.billing_provider, '')) = ?z s.user_id = ?z s.chat_id = ?zs.chat_type = ?z(LOWER(COALESCE(s.git_branch, '')) LIKE ?zA(COALESCE(s.input_tokens, 0) + COALESCE(s.output_tokens, 0)) >= ?zA(COALESCE(s.input_tokens, 0) + COALESCE(s.output_tokens, 0)) <= ?z9COALESCE(s.actual_cost_usd, s.estimated_cost_usd, 0) >= ?z9COALESCE(s.actual_cost_usd, s.estimated_cost_usd, 0) <= ?z#COALESCE(s.tool_call_count, 0) >= ?z#COALESCE(s.tool_call_count, 0) <= ?TrlFrmrn)rrrorr)rrrrrrrrhrrr7rrrrrrrrrrrrclausesrrrs r9_prune_filter_wherezSessionDB._prune_filter_wheressZ,,  ) NN    MM, - - -  ( NN    MM+ , , ,  % NN- . . . MM. ) ) )  $ NN. / / / MM- ( ( (  " NN> * * * MM& ! ! !  5 NN@ A A A MM3j..00333 4 4 4  & NN- . . . MM* % % %  )$6z$B$B !FM NN6 " " " MM- ( ( (  # NN1 2 2 2 MM, ' ' '  # NN1 2 2 2 MM, ' ' '  5 NN@ A A A MM3j..00333 4 4 4  , NNH I I I MM(..** + + +  # NN? + + + MM' " " "  # NN? + + + MM' " " "  % NN, - - - MM) $ $ $  6 NNE F F F MM4k//11444 5 5 5  ! NNS    MM* % % %  ! NNS    MM* % % %   NNK    MM( # # #   NNK    MM( # # #  % NN@ A A A MM. ) ) )  % NN@ A A A MM. ) ) ) t   NN+ , , , ,    NN+ , , ,||G$$f,,r;older_than_daysc r|d3|d|tj|dzz |d<|jd d|i|\}}|j5|jd|d|}d|DcdddS#1swxYwYdS) azReturn the sessions a matching :meth:`prune_sessions` / :meth:`archive_sessions` call would touch, without modifying anything. Backs ``--dry-run`` and pre-confirmation counts. Accepts the same keyword filters as :meth:`_prune_filter_where` (unknown names raise ``TypeError`` there). Rows are ordered oldest-first and carry ``id, source, title, model, started_at, last_active, ended_at, message_count, archived``. ``older_than_days`` is an inactivity threshold: it uses the latest message timestamp, falling back to ``started_at`` for sessions without messages. rNrrraSELECT s.id, s.source, s.title, s.model, s.started_at, COALESCE( (SELECT MAX(m.timestamp) FROM messages m WHERE m.session_id = s.id), s.started_at ) AS last_active, s.ended_at, s.message_count, s.archived FROM sessions s WHERE z? ORDER BY last_active ASC, s.started_at ASCc,g|]}t|Srfrrs r9rz3SessionDB.list_prune_candidates..& s;;;#DII;;;r;rf)r_r rr9r#rr)rrrfilterswhererrs r9list_prune_candidateszSessionDB.list_prune_candidatess3$ KK, - - 5 ,--5+,0IKK%'-G( )10JJJ'JJ v Z < <Z''B,1BBB  F<;):):;;; < < < < < < < < < < < < < < < < < E   7 7C  % %c$i 6 6 6 64yyr;exclude_pinned idle_daysrc ||dkrdStjt|dzz }|rdnd}|j5|jd|dt dd |f}dddn #1swxYwYd |D}|D]}||d t|S) ujArchive every session untouched for at least ``idle_days`` days. "Touched" is the freshest of ``last_activity_at`` and the latest message timestamp (else ``started_at``) — i.e. real recency, not creation time — so a session created long ago but active yesterday is spared, while an old abandoned one (even a still-open one) is swept. Unlike :meth:`archive_sessions`, this method can also archive unended sessions. Guards: * ``pinned = 0`` when ``exclude_pinned`` (the Desktop "keep" flag). * ``archived = 0`` so repeat runs are idempotent no-ops. * only lineage *tips* / standalone rows are candidates (``end_reason <> 'compression'``); a stale tip archives its whole chain via :meth:`set_session_archived`, so we never resurrect an active conversation by matching an old compressed-away root whose live continuation is recent. Returns the number of sessions archived. Never raises for an empty or non-positive ``idle_days`` — it simply archives nothing. Nrg@zAND s.pinned = 0r>z SELECT s.id FROM sessions s WHERE s.archived = 0 AND COALESCE(s.end_reason, '') <> 'compression' z AND rsz? < ? ORDER BY s.started_at ASC cbg|],}t|tjr|dn|d-S)rr)rWrr$rs r9rz4SessionDB.archive_stale_sessions..k s5OOOQ:a55?$1Q4OOOr;T) r rr9r#rrrr<r)rrrr pin_clauserrr{s r9archive_stale_sessionsz SessionDB.archive_stale_sessionsB sF2   A 1uY//'99+9A''r Z  :%%  044    hjj                PO$OOO 1 1C  % %c4 0 0 0 03xxsAB  BBZc 6 |d3|d|tj|dzz |d<jdd|i|\ g fd}|}D]}|||S)uDelete sessions matching the filters. Returns count deleted. By default, delete ended sessions inactive for ``older_than_days`` days, optionally restricted to ``source``. Activity is the latest message timestamp, falling back to ``started_at`` for sessions without messages. Additional keyword filters AND together — the full set is defined by :meth:`_prune_filter_where`: * ``last_active_before`` / ``last_active_after`` — epoch bounds on the latest message timestamp (falling back to ``started_at``). * ``started_before`` / ``started_after`` — epoch bounds on ``started_at``. An explicit ``started_before`` overrides the default ``older_than_days`` inactivity cutoff; pass ``older_than_days=None`` for no implicit upper age bound. * ``title_like`` / ``model_like`` / ``branch_like`` — case-insensitive substring matches. * ``end_reason`` / ``provider`` / ``user_id`` / ``chat_id`` / ``chat_type`` — exact matches (provider case-insensitive, against ``billing_provider``). * ``cwd_prefix`` — session cwd equals or is under this path. * ``min_messages`` / ``max_messages`` — bounds on message_count. * ``min_tokens`` / ``max_tokens`` — bounds on input+output tokens. * ``min_cost`` / ``max_cost`` — bounds on USD cost (actual, falling back to estimated). * ``min_tool_calls`` / ``max_tool_calls`` — bounds on tool_call_count. * ``archived`` — tri-state: None = both (default), True = only archived, False = only unarchived. Only prunes ended sessions (not active ones). Child sessions outside the prune window are orphaned (parent_session_id set to NULL) rather than cascade-deleted. When *sessions_dir* is provided, also removes on-disk transcript files (``.json`` / ``.jsonl`` / ``request_dump_*``) for every pruned session, outside the DB transaction. rNrrrc|d}d|D}|sdSddt|z}|d|dt ||D]E}|d|f|d |f|F|t|S) Nz"SELECT s.id FROM sessions s WHERE ch|] }|d Srrfrs r9r|z8SessionDB.prune_sessions.._do.. rr;rr}r~rrrrr) rrr6rr{rrr where_paramss r9rz%SessionDB.prune_sessions.._do s \\._do s0 LLHe      r;)rr)rrprUrrs `` r9set_metazSessionDB.set_meta sp   NNHe     F       C     r;workspaces_rootct|dsdSddkrdSfd}|S)uRetag legacy kanban worker rows from ``cli`` to ``kanban``. Workers used to spawn without ``HERMES_SESSION_SOURCE``, so their runs landed as untitled ``cli`` rows and the sidebar rendered one per attempt labeled with the worker's own prompt. New workers tag themselves; this reclaims the rows already on disk so they drop out of the session lists too. Identified by cwd under the board's workspaces root — a path only the dispatcher ever runs a session in. Gated per workspaces root (``state_meta``) so each board reclaims its own rows exactly once. Returns the number of rows retagged. rjrzkanban_worker_source_retagged:rc|ddddddddzf}|jpd }d | |S) NzaUPDATE sessions SET source = 'kanban' WHERE source = 'cli' AND (cwd = ? OR cwd LIKE ? ESCAPE '\')rrrrlrrrkrrr)rrrr)rrretaggedgaternrs r9rz3SessionDB.retag_kanban_worker_sessions.._do!s\\Of55==c5IIQQRUW\]]`ddeF+!H MM$FM 3 3 3Or;)rXrmrr)rrrrrns` @@r9retag_kanban_worker_sessionsz&SessionDB.retag_kanban_worker_sessions s_%%,,U33 1888 ==  # % %1       ""3'''r;c6d}||dS)uFCreate Telegram DM topic-mode tables on explicit /topic opt-in. This migration is deliberately not part of automatic SessionDB startup reconciliation. Operators must be able to upgrade Hermes, keep the old Telegram bot behavior running, and only mutate topic-mode state when the user executes /topic to opt into the feature. Schema versions: v1 — initial shape (no ON DELETE CASCADE on session_id FK) v2 — session_id FK gets ON DELETE CASCADE so session pruning automatically clears bindings. c|d|dd}|rCASCADENrfrs r9rzHSessionDB.apply_telegram_topic_migration.._do..Q!sQ$$Fj(Hc!fly-H$$$$$$r;a CREATE TABLE telegram_dm_topic_bindings_new ( chat_id TEXT NOT NULL, thread_id TEXT NOT NULL, user_id TEXT NOT NULL, session_key TEXT NOT NULL, session_id TEXT NOT NULL REFERENCES sessions(id) ON DELETE CASCADE, managed_mode TEXT NOT NULL DEFAULT 'auto', linked_at REAL NOT NULL, updated_at REAL NOT NULL, PRIMARY KEY (chat_id, thread_id) ); INSERT INTO telegram_dm_topic_bindings_new SELECT chat_id, thread_id, user_id, session_key, session_id, managed_mode, linked_at, updated_at FROM telegram_dm_topic_bindings; DROP TABLE telegram_dm_topic_bindings; ALTER TABLE telegram_dm_topic_bindings_new RENAME TO telegram_dm_topic_bindings; CREATE UNIQUE INDEX idx_telegram_dm_topic_bindings_session ON telegram_dm_topic_bindings(session_id); CREATE INDEX idx_telegram_dm_topic_bindings_user ON telegram_dm_topic_bindings(user_id, chat_id); rq)r2)rrrrrXisdigitrCrr)rrcurrent_versionfk_rows needs_rebuilds r9rz5SessionDB.apply_telegram_topic_migration.._do!!s+   ! ! ! Lll<5hjj 29]S__=T=T=V=V]c'!*ooo\]O"",,K(**!$$$&$$$!! !&&6 LLH9     r;Nr)rrs r9apply_telegram_topic_migrationz(SessionDB.apply_telegram_topic_migration!s1T T T j C     r;)has_topics_enabledallows_users_to_create_topicsrrc|tjdttdttfdfd}||dS)zEnable Telegram DM topic mode for one private chat/user. This method intentionally owns the explicit topic migration. Ordinary SessionDB startup must not create these side tables. rUr3c|dS|rdndS)Nr?rrfrYs r9_to_intz5SessionDB.enable_telegram_topic_mode.._to_int!s}t$111 $r;c |dttfdS)Na INSERT INTO telegram_dm_topic_mode ( chat_id, user_id, enabled, activated_at, updated_at, has_topics_enabled, allows_users_to_create_topics, capability_checked_at ) VALUES (?, ?, 1, ?, ?, ?, ?, ?) ON CONFLICT(chat_id) DO UPDATE SET user_id = excluded.user_id, enabled = 1, updated_at = excluded.updated_at, has_topics_enabled = excluded.has_topics_enabled, allows_users_to_create_topics = excluded.allows_users_to_create_topics, capability_checked_at = excluded.capability_checked_at )rrX)rrrrrrPrs r9rz1SessionDB.enable_telegram_topic_mode.._do!sf LL LLLLG.//G9::     r;N)rr rrvrCr)rrrrrrrrPs ```` @@r9enable_telegram_topic_modez$SessionDB.enable_telegram_topic_modex!s ++---ikk %8D> %hsm % % % %           4 C     r;)clear_bindingsrc@fd}||dS)aDisable Telegram DM topic mode for one private chat. When ``clear_bindings`` is True (default) the (chat_id, thread_id) bindings for this chat are also cleared so re-enabling later starts from a clean slate. Set to False if the operator wants to preserve bindings for a later re-enable. Never creates the topic-mode tables from scratch; if they don't exist there is nothing to disable and the call is a no-op. c |dtjtfr&|dtfdSdS#tj$rYdSwxYw)NOUPDATE telegram_dm_topic_mode SET enabled = 0, updated_at = ? WHERE chat_id = ?z8DELETE FROM telegram_dm_topic_bindings WHERE chat_id = ?)rr rXrr)rrrs r9rz2SessionDB.disable_telegram_topic_mode.._do!s  (Y[[#g,,/ "LLRW +    sAA##A65A6Nr)rrrrs `` r9disable_telegram_topic_modez%SessionDB.disable_telegram_topic_mode!s>        C     r;c|j5 |jdt|t|f}n!#t j$rYddddSwxYw dddn #1swxYwY|dSt|t jr|dn|d}t|S)zDReturn whether Telegram DM topic mode is enabled for this chat/user.z SELECT enabled FROM telegram_dm_topic_mode WHERE chat_id = ? AND user_id = ? NFrr) r9r#rrXrrrrWr$rv)rrrr[rs r9is_telegram_topic_mode_enabledz(SessionDB.is_telegram_topic_mode_enabled!s0 Z   j((\\3w<<0  (** +                            ;5$.sGK$@$@L#i..c!fG}}5BA ABA2#B1A22BB Bc@|j5 |jdt|t|f}n!#t j$rYddddSwxYw dddn #1swxYwY|rt|ndS)z?Return the session binding for a Telegram DM topic, if present.z SELECT * FROM telegram_dm_topic_bindings WHERE chat_id = ? AND thread_id = ? Nr9r#rrXrrrr)rrrr[s r9get_telegram_topic_bindingz$SessionDB.get_telegram_topic_binding!sZ   j((\\3y>>2  (** +                            )tCyyyT)rc|j5 |jdt|f}n##t j$rgcYcdddSwxYw dddn #1swxYwYd|DS)zAll Telegram DM topic bindings for one chat, newest first. Read-only; returns [] if the bindings table doesn't exist yet (does not trigger the topic-mode migration). zSSELECT * FROM telegram_dm_topic_bindings WHERE chat_id = ? ORDER BY updated_at DESCNc,g|]}t|Srfrrs r9rzCSessionDB.list_telegram_topic_bindings_for_chat.."s***cS ***r;)r9r#rrXrrr)rrrs r9%list_telegram_topic_bindings_for_chatz/SessionDB.list_telegram_topic_bindings_for_chat!sZ   z))A\\O(**  +                             +*T****s4A6;AA6A&A6%A&&A66A:=A:c$|j5 |jdt|f}n!#t j$rYddddSwxYw dddn #1swxYwY|rt|ndS)aReturn the Telegram DM topic binding for a given session_id, if present. Uses the UNIQUE INDEX on telegram_dm_topic_bindings(session_id) for an efficient reverse lookup. Returns None when the session has no binding or the table does not exist yet. z{ SELECT * FROM telegram_dm_topic_bindings WHERE session_id = ? Nrrrr[s r9%get_telegram_topic_binding_by_sessionz/SessionDB.get_telegram_topic_binding_by_session"sZ   j((__&  (** +                            )tCyyyT)4A4;AA4A$A4#A$$A44A8;A8cttddifd}||dS)uRemove the binding row for a single (chat, thread) pair. Called when the Telegram Bot API confirms a topic was deleted externally (``Thread not found`` after the same-thread retry already failed). Without this prune, the stale row keeps living in ``telegram_dm_topic_bindings`` and the recovery logic in ``gateway.run._recover_telegram_topic_thread_id`` cheerfully redirects future inbound messages to the deleted topic, causing tool progress, approvals, and replies to land in the wrong place. Issue #31501. When this prune removes the chat's *last* remaining binding, the chat's row in ``telegram_dm_topic_mode`` is also flipped to ``enabled = 0`` in the same transaction. Otherwise the chat would be left in topic mode with zero lanes — and ``gateway.run._recover_telegram_topic_thread_id`` keeps treating the chat as topic-enabled, lobby messages keep hunting for a binding that no longer exists, and a user who disabled topics in the Telegram client (rather than via ``/topic off``) stays stuck until the next send happens to fail. Clearing the flag makes recovery fully stand down once the dead topics are gone. Returns the number of binding rows deleted (0 when the binding was already absent or the topic-mode tables haven't been migrated yet — both are silent no-ops; we never raise from a cleanup hot path). r^rcx |df}|jpdd<n#tj$r dd<YdSwxYwdsdS |df}|+|dt jfdSdS#tj$rYdSwxYw)Nz DELETE FROM telegram_dm_topic_bindings WHERE chat_id = ? AND thread_id = ? rr^z SELECT 1 FROM telegram_dm_topic_bindings WHERE chat_id = ? LIMIT 1 r)rrrrrr )rrrrrrs r9rz4SessionDB.delete_telegram_topic_binding.._doB"s  i( $*?#7a  +   #$  7#   LLJ  (** $LLLg.%$ +    s!$(AAAB&&B98B9)rXr)rrrrrs `` @r9delete_telegram_topic_bindingz'SessionDB.delete_telegram_topic_binding"slBg,, NN A,# # # # # # # J C   wr;auto) managed_moderc8|tjtttttfd}||dS)a Bind one Telegram DM topic thread to one Hermes session. A Hermes session may only be linked to one Telegram topic in MVP. Rebinding the same topic to the same session is idempotent; trying to link the same session to a different topic raises ValueError. c |df}|t|tjr|dn|d}t|tjr|dn|d}t |kst | krt d|d fdS)Nz SELECT chat_id, thread_id FROM telegram_dm_topic_bindings WHERE session_id = ? rrrr?z3session is already linked to another Telegram topicaI INSERT INTO telegram_dm_topic_bindings ( chat_id, thread_id, user_id, session_key, session_id, managed_mode, linked_at, updated_at ) VALUES (?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT(chat_id, thread_id) DO UPDATE SET user_id = excluded.user_id, session_key = excluded.session_key, session_id = excluded.session_id, managed_mode = excluded.managed_mode, updated_at = excluded.updated_at )rrrWrr$rXrF) rexisting_session linked_chat linked_threadrrrPrrrrs r9rz*SessionDB.bind_telegram_topic.._do"s#||    hjj  +=GHXZaZe=f=f.y99l|}~l AKL\^e^iAjAj!D 0 = =qABCqD {##w..#m2D2D 2Q2Q$%Z[[[ LL        r;N)rr rXr) rrrrrrrrrPs `````` @r9bind_telegram_topiczSessionDB.bind_telegram_topicj"s ++---ikkg,, NN g,,+&& __ % % % % % % % % % % % L C     r;c|j5 |jdt|f}n!#t j$rYddddSwxYw dddn #1swxYwY|duS)aMReturn True if a Hermes session is already bound to any Telegram DM topic. Read-only: does NOT trigger the telegram-topic migration. If the topic-mode tables have not been created yet (i.e. nobody has run ``/topic`` in this profile), the session is by definition unbound and we return False. z SELECT 1 FROM telegram_dm_topic_bindings WHERE session_id = ? LIMIT 1 NF)r9r#rrXrrrrs r9#is_telegram_session_linked_to_topicz-SessionDB.is_telegram_session_linked_to_topic"sZ   j(( __& (**+                           $r  )rWc |j5 |jdtdt ddt |t |f}nw#tj $re|jdtdt ddt |t |f}YnwxYwdddn #1swxYwYg}|D]R}| |}t| dd|d <| |S|S) uGList previous Telegram sessions for this user that are not bound to a topic. Read-only: does NOT trigger the telegram-topic migration. If the topic-mode tables are absent, fall back to a simpler query that just returns this user's Telegram sessions — there can't be any bindings yet. z SELECT s.*, COALESCE(sp.prompt, s.system_prompt) AS _system_prompt_resolved, COALESCE( (SELECT a7 FROM messages m WHERE m.session_id = s.id AND m.role = 'user' AND m.content IS NOT NULL ORDER BY m.timestamp, m.id LIMIT 1), '' ) AS _preview_raw, rsa AS last_active FROM sessions s LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hash WHERE s.source = 'telegram' AND s.user_id = ? AND NOT EXISTS ( SELECT 1 FROM telegram_dm_topic_bindings b WHERE b.session_id = s.id ) ORDER BY last_active DESC, s.started_at DESC LIMIT ? ae AS last_active FROM sessions s LEFT JOIN system_prompts sp ON sp.hash = s.system_prompt_hash WHERE s.source = 'telegram' AND s.user_id = ? ORDER BY last_active DESC, s.started_at DESC LIMIT ? Nrtr>ru)r9r#rrrrXrCrrrrrrr)rrrrWrrr[rs r9(list_unlinked_telegram_sessions_for_userz2SessionDB.list_unlinked_telegram_sessions_for_user"sZ6 6 5 z)) &9 2#660\\3u::.34(**56+   z)) &9 2#66(\\3u::.+,(**- ;6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 p*, % %C,,S11G!/ NB0O0O!P!PGI  OOG $ $ $ $s6C0A"A-,C0-A1C!C0 C!!C00C47C4rc |j5|j ddddS|jdd}|jdd}dddn #1swxYwYt |t |zS#t $r&}t d|Yd}~dSd}~wwxYw)u3Database size in bytes as SQLite itself accounts for it. ``page_count * page_size`` — the size the main DB file will have once the WAL is checkpointed back into it. Prefer this over ``os.path.getsize(db_path)`` when reporting the effect of a VACUUM. In WAL mode a VACUUM's rewrite lands in the ``-wal`` file, and the checkpoint that folds it back is refused while any other connection (a live gateway) holds a read-mark. Until that happens the main file on disk still carries its pre-VACUUM size and keeps growing, so a stat()-based before/after delta understates the win and can go negative — the "reclaimed -3820.1 MB" report on a database that had actually shrunk 60%. Returns None if the pragmas cannot be read. NzPRAGMA page_countrzPRAGMA page_sizez"Could not read logical DB size: %s)r9r#rrrCrKrr)r page_count page_sizers r9logical_size_byteszSessionDB.logical_size_bytes#s["  Q Q:% Q Q Q Q Q Q Q Q"Z//0CDDMMOOPQR  J../ABBKKMMaP  Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q z??S^^3 3    LL=s C C C44444 sEB9 B B9A$B B9BB9B!B99 C)C$$C)cd} |}n2#t$r%}td|Yd}~nd}~wwxYw|j5 |jdn2#t$r%}td|Yd}~nd}~wwxYw|jddddn #1swxYwY|S)uRun VACUUM to reclaim disk space after large deletes. SQLite does not shrink the database file when rows are deleted — freed pages just get reused on the next insert. After a prune that removed hundreds of sessions, the file stays bloated unless we explicitly VACUUM. VACUUM rewrites the entire DB, so it's expensive (seconds per 100MB) and cannot run inside a transaction. It also acquires an exclusive lock, so callers must ensure no other writers are active. Safe to call at startup before the gateway/CLI starts serving traffic. FTS5 segments are merged first via :meth:`optimize_fts` so the subsequent VACUUM reclaims the pages freed by the merge. This is a layout-only optimization — search results are unchanged. Returns the number of FTS indexes that were optimized (0 if the merge step failed or no FTS tables exist). rz%FTS optimize before VACUUM failed: %sNrz2WAL checkpoint (TRUNCATE) before VACUUM failed: %sr) optimize_ftsrKrrr9r#rr)r optimizedrs r9vacuumzSessionDB.vacuum1#sN.  I))++II I I I NNBC H H H H H H H H IZ ) ) X ""#DEEEE X X X QSVWWWWWWWW X J  x ( ( (  ) ) ) ) ) ) ) ) ) ) ) ) ) ) )sP AAACA/.C/ B9BCBCC C retention_daysmin_interval_hoursr!min_vacuum_interval_dayscdddd} |d}tj}|r; t|} || z |dzkrd|d<|Sn#ttf$rYnwxYw|||} | |d <|d } d} | r4 |t| z |d zk} n#ttf$rd} YnwxYw|rx| dkrr| rp |d|d <|d t|n2#t$r%} t d | Yd} ~ nd} ~ wwxYw|dt|| dkr't d| ||d rdndnD#t$r7} t d| t| |d<Yd} ~ nd} ~ wwxYw|S)uDIdempotent auto-maintenance: prune inactive sessions + optional VACUUM. Records the last run timestamp in state_meta so subsequent calls within ``min_interval_hours`` no-op. VACUUM has its own, typically longer, throttle controlled by ``min_vacuum_interval_days`` so routine pruning does not repeatedly rewrite the database. Designed to be called once at startup from long-lived entrypoints (CLI, gateway, cron scheduler). When *sessions_dir* is provided, on-disk transcript files (``.json`` / ``.jsonl`` / ``request_dump_*``) for pruned sessions are removed as part of the same sweep (issue #3015). Never raises. On any failure, logs a warning and returns a dict with ``"error"`` set. Returns a dict with keys: - ``"skipped"`` (bool) — true if within min_interval_hours of last run - ``"pruned"`` (int) — number of sessions deleted - ``"vacuumed"`` (bool) — true if VACUUM ran - ``"error"`` (str, optional) — present only on failure Fr)skippedprunedvacuumedlast_auto_pruneTr()rrr) last_vacuumrr*zstate.db VACUUM failed: %sNzFstate.db auto-maintenance: pruned %d session(s) inactive for %d days%sz + VACUUMr>z$state.db auto-maintenance failed: %sr") rr rrErFrr!rrXrKrrr)rr$r%r!rr&rlast_rawrPlast_tsr)last_vacuum_raw vacuum_duers r9maybe_auto_prune_and_vacuumz%SessionDB.maybe_auto_prune_and_vacuumW#s}<.3aU!S!S7 '}}%677H)++C #HooGW}'9D'@@@,0y)% A":.D(( .))F &F8 #mmM::OJ &&"%o(>(>">C[^cCc!cJJ!:.&&&!%JJJ& F&1***FKKMMM)-F:&MM-S:::: FFFNN#?EEEEEEEEF MM+SXX 6 6 6zz \"#)*#5=KK2   ' ' ' NNA3 G G G!#hhF7OOOOOO '  s*F!!AF!A*'F!)A**8F!#B=<F!=CF!C F!!= %s daysz state.db auto-archive failed: %sr"N) rr rrErFrrrXrrrKr) rrr%rrr.rPr7rs r9maybe_auto_archivezSessionDB.maybe_auto_archive#sa(.3!B!B '}}%899H)++C U8__,/AD/HHH,0y)% I":.D22.3H"*F:  MM-s3xx 8 8 8!|| S  ' ' ' NN=s C C C!#hhF7OOOOOO ' s;*C AC A'$C &A''A$C D -DD c<fd}||S)zMark a session as pending handoff to the given platform. Returns True if the row was found and not already in flight; False if the session is already in a non-terminal handoff state. cJ|df}|jdkS)NzUPDATE sessions SET handoff_state = 'pending', handoff_platform = ?, handoff_error = NULL WHERE id = ? AND (handoff_state IS NULL OR handoff_state IN ('completed', 'failed'))rr')rrfrrs r9rz&SessionDB.request_handoff.._do#s5,,Q :&C.*$s'FFF!D**1--FFFr;)r#rrrK)rrfs` r9list_pending_handoffszSessionDB.list_pending_handoffs$sm *$$,CGFFFs||~~FFF F   III s9= A  A c8fd}||S)uCAtomically transition pending → running. Returns True if claimed.cH|df}|jdkS)NzXUPDATE sessions SET handoff_state = 'running' WHERE id = ? AND handoff_state = 'pending'rr')rrfrs r9rz$SessionDB.claim_handoff.._do0$s/,,= C ._do;$s- LL4      r;Nrr"s ` r9complete_handoffzSessionDB.complete_handoff9$s8      C     r;c@fd}||dS)z/Mark a handoff as failed and record the reason.cH|dddfdS)NzLUPDATE sessions SET handoff_state = 'failed', handoff_error = ? WHERE id = ?r r)rr"rs r9rz#SessionDB.fail_handoff.._doE$s8 LL1ttj)     r;Nr)rrr"rs `` r9 fail_handoffzSessionDB.fail_handoffC$s>       C     r;)r3Nr]r) NNNNNNNNNNNN)T)r$)NN)rb)r)r)rrNrrrNNNNNNNNrF)rN)NNNNrTrFrTFFFNNFFN)NNNNNNNNNNNNFNNNNNN)F)FNr)r)FFFF)NrTrN)NNNrFFFN)r?)rNN)rr"TNr#)rr"T)rrrrr/rr~rrrrr3r4rr_FTS_MERGE_MAX_PAGES_PER_INDEX_FTS_MERGE_COMMANDS_PER_PASS_IMPORT_MAX_SESSIONS _IMPORT_MAX_MESSAGES_PER_SESSION_IMPORT_MAX_TOTAL_MESSAGES_IMPORT_MAX_SESSION_BYTES_IMPORT_MAX_TOTAL_BYTES staticmethodrrXrrrr$rrrrrvr ConnectionrUrrWrrr]Cursorrarergr~rrrrrrrr5rrrr_FTS_REBUILD_CHUNK_ROWS_FTS_REBUILD_DUTY_FACTOR_FTS_REBUILD_MIN_PAUSErrrrrrrrrrrrrrr rrr#r(r.r4r:r@rDrJrMrCrRrZr]rargrnrqrsrrrrrrrrrrrrrrrrrrrrrrrrrrrr rrrrr#r+r0r2r5r<r@rBrFrNrR frozenset_SESSION_COMPACT_EXCLUDEDrS__annotations__rr classmethodrrrrrrrrrrrrrrrrrrr rrrr"r)r7r4r5rGrLrOr3r;r`rfrmrrrtrwrxrzr}rrrrrrrrrrrrrrrrrrrrrrrrrrr rrr _FTS_TABLESrr!r2r5r8r>rArDrGrJrfr;r9rrsg.H#' "% # ###!#!%%("#$ '-$!' /.(3-HSM\   \ w{tCH~\tttttttp3); <3333j^$ (@ T   \   7+C      \  &'.&T&&&\&2  W-E  $       '*B  t     t,t,t,t,l7>d\   <'+@@ g()1, -@UO@ @@@@D+0 8 2G,A 2d 2 2 2\ 2/G,A/d////bGGGG<+"+"+"V"" )&d"'+!%)!% !eOeOeOeO eO 38n eO  eOeOc]eOeOeOeOeOeOeOeO !eOeOeOeONc .3R!R!R!R! R!  R!  R!R!R!R!R!R!(,R! R!R!R!R!h!!s!t!t!!!!(BD!!!!,/!;>! !!!!:8:!!!CH~!14! !!!!.<>AAASA$sCx.AAAA8: ! ! ! I !14 !  ! ! ! !$#' '9'9'93-'9 '9 d38n  '9'9'9'9\$(!% 0"0"0"0" 0" C= 0" # 0" #0"0"0"0"l"&%)!%#'#'G<G<G<G<# G< c] G< # G<C=G<C=G< $sCx. !G<G<G<G !C ! !PT ! ! ! !N# ,,,, ,  ,,,,d# VVVV V  VVVVp34IcIhsmIIII(#-M &*37 -M-M-M-M UO-M c] -M /0 -M -M-M-M-M^.M.M.M.M.M.M`  s  xS#X7O     & $ !!!!} !  !!!!, ! !.6sm !  ! ! ! !#!s#!3#!4#!#!#!#!R $"&26&*6!6!6!6!} 6! 3- 6!  S#X/ 6!sm6!6! 6!6!6!6!p&!3&!&!$&!&!&!&!P*8DcN+C****\*.'+ !!!!!!!! !!  !! sm !! !!!!!!!!h I! *;S*;t*;*;*;*;X11%1$1111f8888*U3S#X3F-G(HT04%T#s(^ 3454 eCc3h'( )4444<4848u4848484848l!""# !.2+/%)%))-*.*.&*%g!g!g!g! g!  g!  g! g!g!%UOg!"%g!c]g!c]g!"#g!#3-g!#3-g! sm!g!"#g!$%g!& 'g!g!g!g!x'd d d d } d #3- d #3-d smd d d d  d d %UOd "%d c]!d "c]#d $%d &'d ( )d d d d R           $*.*.!""# !.29!9!9!9!9! } 9! #3- 9!#3-9!9!9!9! 9!9!%UO9! 9!9!9!9!v  7G SV    >%-%-%-%-%-N2J < < < <chsm:!(C!(C!(!(!(!(F:c:hsm::::D!* ./!!04x}333!%)!&!")-%*!&# "$%ooococ o  o  oooo#'o#ooooo !o"#o$%o& d38n 'oooor ' 6c 6c 6 6 6[ 6D c c   [ 38C=\8< #<>Fsm< <<<2+Q ,"*#-Q . /Q Q Q Q n26$( G G G tCH~&G "*# G SM G G G G G V6: . . . .(+ .69 .IL ."4S>2 .  . . . .J) =(=(=(=(=(} =(  =( $tCH~& '=(=(=(=(~dd/2d d38n dddd,179.9.9.9.*-9. d38n 9.9.9.9.x/5A\` ' ' ' '(+ '>A 'UY ' # ' ' ' 'DCSCXc]CCCC'3'' ''''"_*S_*DcSVhDX_*]bcfhkck]l_*_*_*_*J" 8!8!8!tCH~&8! 8!  8!8!8!8!t 1 1 1 1 1 12(2(37S#X3G2( 2(2(2(2(h(((+(:=( ((((F"'# 2222} 2  2 d38n  2222p N N N N  N c3h N N N N `W>R' ' ' ' R @ @ @ @ @ @5s5tCy5555, d4S>6J QUVY[^V^Q_ dh   \ "U U 25U c3hU U U U n(#((((((<! +N+N+N+N +N  +N d38n  +N+N+N+Nf!!"!&#!&%);(;(;(c;( ;(  ;(  ;(;(;(c;( ;(;(;(;(z#d&"'#!& +L+L+L+L +L  +L c3h +L+L+L+LZ ( ( (s ( ( ( (11471 1111,ODcNOtOOOOJtCH~J$JJJJ+B#+B$s)+B+B+B+BZ ! ! ! ! ! !HTNPT\43S3T#Y3333((,37 999tn9&d3i0 9  9999|(,***tn*  ****^(,RR#YRtnR RRRRh(c((((6(,BBtnB BBBBH/3-1*.)- $$($($(&*&*#'$("&!%!%#'%)$($($($((,(,1I-I-I-$UOI-$E?I-! I-   I-  I-SMI-SMI-SMI-smI-smI-4.I-SMI-3-I-#I- #!I-"C=#I-$c]%I-&SM'I-(SM)I-*5/+I-,5/-I-.! /I-0! 1I-2 sDy 3I-I-I-\I-Z,0'<'<!%'<'< d38n  '<'<'<'!>!>!>! >!  >!  >!>!>! >!>!>!>!@6 KKKK K  K d38n  KKKKfOKHSM8$$$$$P!"$'+(* XXX X X tn X #& X c3hXXXXx"$# 333 3 3 c3h 3333B(#((((((&CHT#s(^4L.tDcN';$ ( ( ( ( ( (!3!4!!!!!s!3!4!!!!!!r;rc&eZdZdZd dZdefdZdS) AsyncSessionDBuAsync door onto SessionDB: offloads each call via asyncio.to_thread so a blocking SQLite call never freezes the event loop. Generic forwarder — the audit confirms no method returns a live cursor/generator.dbrr3Nc||_dSr)_db)rr_s r9rzAsyncSessionDB.__init__Q$s r;rLc^t|j|tsSfd}|S)Nc>Ktjg|Ri|d{VSr)asyncio to_thread)argsrattrs r9 _offloadedz.AsyncSessionDB.__getattr__.._offloadedY$s: *4A$AAA&AAAAAAAA Ar;)r>racallable)rrLrhrgs @r9 __getattr__zAsyncSessionDB.__getattr__T$sMtx&&~~ K B B B B Br;)r_rr3N)rrrrrrXrjrfr;r9r^r^N$sOZZr;r^)rc)rr)rrdrr;r5rloggingrGr1rrrr7r  collectionsr contextlibrpathlibragent.memory_managerrr}ragent.message_sanitizationr agent.skill_commandsr r r hermes_constantsr hermes_cli.sqlite_runtimerrtypingrrrrrrrhermes_state_commonrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.hermes_state_portabilityr/hermes_state_schemar0hermes_state_searchr1rIrO getLoggerrrcompilerArXr:rvrTrZrbrgrorurrrrrrrrrZr8rrrr!rrrrrrrrrSrrrrrrrrrrrrKrr0r6r1rErMr8r@rBrFrYryrrrsrtrrrrrrrrrCrr rr^rfr;r9r{sU         %%%%%%111111555555;;;;;; -,,,,,GFFFFFFFFFFFFFFFFF6=<<<<<222222222222MMMM FFF  8 $ $","*-G"H"HEsEsEEEE-S-T----`LSLSLLLLtCH~(3-"HHSHcHHHHb3b5d3i+@bbbb suS$s)^'<$6$s)6S 6666@ S d3i     GCLL!/##j0**$****`#'(3-&&&& (( (+suuCH,,,*IN,,),SX---+Y^--hsm"Xc] EtCH~E$EEEE&4S>" $sCx.6''#'UX''''.C 2Cx}CCCC*  '*<       F '*<     >%) 1 15/ 1  1 1 1 1 #    8c88884     '2   " ccc  cc c  ccccT! ,,,  ,, ,  ,,,,^ :SyT@;;;  ;; ;;;;h%(CEES)))%y~'' S} S S S S SCMC/$6C4CCCC" 4 D    )T)htn))))^W W W W W  W W W W t{t{ {{{{|=AeeeDeTeT#s(^eeeeT )V8$8888'"42     L   IIIII,III     (C       H'*$+$+$+ $+ #$+26$+ $+$+$+$+NoTohtnoooodLu!Lu!Lu!Lu!Lu!"$68OLu!Lu!Lu!^js CCC