§ Pٍmj-Bمَؤ—UdZddlmZddlZddlZddlZddlZddlZddlm Z ddl mZeje ¦«ZdZdZej¦«ZiZded <Gd „de¦«Zd;d„Zdd„ZGd„d¦«ZGd„deej¦«ZiZded<d?d"„Z ddd#œd@d&„Z!dAd(„Z"dBd*„Z#dCd,„Z$d-d.d/œdDd4„Z%Gd5„d6e¦«Z&ej'd7d8œdEd:„¦«Z(dS)FuٹLock-safe inspection of SQLite database files. Why this module exists ---------------------- POSIX advisory locks are cancelled **process-wide** by ``close()`` on *any* file descriptor for that file:: the close() system call will cancel all POSIX advisory locks on the same file for all threads and all file descriptors in the process -- https://sqlite.org/howtocorrupt.html#_posix_advisory_locks_canceled_by_a_separate_thread_doing_close_ So a bare ``open(db_path, "rb") ... close()`` on a **live** database silently drops every lock SQLite holds on it from this process -- including the EXCLUSIVE lock a ``VACUUM`` is holding while it rewrites the whole file, and the RESERVED lock an in-flight ``BEGIN IMMEDIATE`` is holding. Other processes are then free to write into a file that a writer still believes it owns, which is the documented route to "database disk image is malformed". Hermes is exactly the topology this hits: gateway, dispatcher, dashboard, TUI, CLI, cron and kanban workers all open the same ``state.db`` / ``kanban.db``, and several code paths used to byte-probe those files while connections were live. The rules --------- 1. **Never** ``open()`` a database file that may have live connections in this process. Ask SQLite instead -- :func:`page_count_bytes` reads the same header field via ``PRAGMA``, over the existing connection, taking no new descriptor. 2. Byte-level probes are only safe **before any connection exists** for that path (first-open validation). Route those through :func:`read_header_bytes_preopen`, which refuses once a connection has been registered for the path. Concurrency contract -------------------- The registry is not advisory bookkeeping -- it is the guard, so the check and the byte read must be **atomic with respect to connection lifecycle**. ``_live_lock`` is therefore held across three critical sections, each of which spans the syscall *and* the registry mutation: * open + register (:func:`connect_tracked`) * close + unregister (:meth:`TrackedConnection.close`) * check + ``open``/``read``/``close`` (:func:`read_header_bytes_preopen`) Without that, a thread could pass the "no live connection" check, a second thread could open a connection and take a write lock, and the first thread's ``close()`` would then cancel it -- reintroducing the exact bug this module exists to prevent. The lock is never held while a caller *uses* a connection, only across these transitions, so it does not serialise database work. Path identity ------------- Connections are keyed by the **canonical database path**, resolved from ``PRAGMA database_list`` on the opened connection. The caller's spelling is not trustworthy: ``SessionDB``'s read-only path opens ``file:/â€¦/state.db?mode=ro`` with ``uri=True``, and treating that string as a filesystem path yields a key like ``/file:/â€¦/state.db?mode=ro`` which no later probe of the real ``Path`` can ever match. é)عannotationsN)عPath)عOptionalsSQLite format 3ézdict[str, int]ع_live_connectionscَ—eZdZdZdS)عUntrackableConnectionErrorzعA connection to a probe-able database could not be tracked. Raised rather than silently returning an untracked connection: on these paths tracking is part of the correctness contract, not an optimisation. N©ع__name__ع __module__ع__qualname__ع__doc__©َْA/home/thesage/.hermes/hermes-agent/hermes_cli/sqlite_safe_read.pyr r Vs€€€€€ًًًًrr عpathْ Path | strعreturnعstrcَ‍— tt|¦« ¦«¦«S#t$rt|¦«cYSwxYw)z;Canonicalise a *filesystem* path for use as a registry key.)rrعresolveعOSError©rs rع_keyr^sT€ًف•4ک‘:”:×%ز%ر'ش'ر(ش(ذ(ّفًًًفگ4‰yŒyذذذًّّّs‚-0°AءAعconnْsqlite3.Connectionْ Optional[str]cَن— | d¦« ¦«}n#tj$rYdSwxYw|rt |¦«dkrdS|d}|sdSt|¦«S)a The on-disk path of ``main``, as SQLite itself reports it. Immune to the caller's spelling (``file:`` URIs, relative paths, symlinks). Returns ``None`` for in-memory or unnamed databases, which cannot be byte-probed and therefore need no tracking. zPRAGMA database_listNéé)عexecuteعfetchoneعsqlite3عErrorعlenr)rعrowعpath_strs rع_canonical_db_pathr(fsˆ€ًطڈlٹlذ1ر2ش2×;ز;ر=ش=ˆˆّفŒ=ًًًطˆtˆtًّّّàً•#گc‘(”(کQ’,گ,طˆtطگ1Œv€Hطًطˆtفگ‰>Œ>ذs‚'*ھ=¼=عNonecَ°—t|¦«}t5t |d¦«dzt|<ddd¦«dS#1swxYwYdS)z¹Record that this process now holds a connection to *path*. Prefer :func:`connect_tracked`; this exists for callers that manage their own connection objects, and for tests. réN)rع _live_lockrعget)rعkeys rعtrack_connectionr/ys€ُˆt‰*Œ*€Cف ًCًCف!2×!6ز!6°s¸Aر!>ش!>ہر!Bصک#رًCًCًCٌCôCًCًCًCًCًCًCًCًّّّCًCًCًCًCًCs—'AءAءAcَّ—t|¦«}t5t |d¦«dz }|dkr|t|<nt |d¦«ddd¦«dS#1swxYwYdS)z5Record that one connection to *path* has been closed.rr+N)rr,rr-عpop)rr.ع remainings rعuntrack_connectionr3„s¾€ه ˆt‰*Œ*€Cف ً-ً-ف%×)ز)¨#¨qر1ش1°Aر5ˆ طگqٹ=ˆ=ط%.صکcر"ذ"ه×!ز! # tر,ش,ذ,ً-ً-ً-ٌ-ô-ً-ً-ً-ً-ً-ً-ً-ًّّّ-ً-ً-ً-ً-ً-s—AA/ء/A3ء6A3عboolcَp—t5t|¦«tvcddd¦«S#1swxYwYdS)z>Whether this process currently holds any connection to *path*.N)r,rrrs rعhas_live_connectionr6ڈs{€ه ً/ً/فگD‰zŒzص.ذ.ً/ً/ً/ً/ٌ/ô/ً/ً/ً/ً/ً/ً/ًّّّ/ً/ً/ً/ً/ً/sˆ+«/²/cَ4‡—eZdZUdZdZded<dˆfd„ZˆxZS) ع_TrackingMixinzAUntrack-on-close behaviour, mixable into any Connection subclass.Nz str | Noneع_hermes_tracked_pathrr)cَع•—t5t|dd¦«}t¦« ¦«|پd|_t|¦«ddd¦«dS#1swxYwYdS)Nr9)r,عgetattrعsuperعcloser9r3)عselfrع __class__s €rr=z_TrackingMixin.closeڑs®ّ€ف ً )ً )فک4ذ!7¸ر>ش>ˆDُ ‰GŒGڈMٹM‰OŒOˆOطذط,0گش)ف" 4ر(ش(ذ(ً )ً )ً )ٌ )ô )ً )ً )ً )ً )ً )ً )ً )ًّّّ )ً )ً )ً )ً )ً )s‰A A ء A$ء'A$)rr))rrr rr9ع__annotations__r=ع __classcell__)r?s@rr8r8•sWّ€€€€€€طKذKà'+ذذ+ذ+ذ+ر+ً )ً )ً )ً )ً )ً )ً )ً )ً )ً )rr8cَ—eZdZdZdS)عTrackedConnectionuڈA ``sqlite3.Connection`` that untracks its path exactly once on close. Counting opens is easy; counting closes reliably is not, because callers close connections in many places (and some hand them to ``contextlib.closing``). Putting the decrement on ``close()`` â€” the one method every close path must go through â€” keeps the registry from drifting upward and permanently disabling byte-probes. The real ``close()`` and the unregister happen together under ``_live_lock`` so a concurrent probe can never observe "no live connection" while this descriptor is still open. Unregister runs only after ``close()`` succeeds; a raising close leaves the connection tracked so the byte-probe guard keeps refusing. Note ``with conn:`` does NOT close a sqlite3 connection (it only commits or rolls back), so this hook is not fired spuriously by transaction scopes. Nr rrrrCrC§s€€€€€ًًًًrrCzdict[type, type]ع_tracked_factory_cacheعfactoryعtypecَê—|tjurtSt|t¦«r|St |¦«}|€*td|j›‌t|fi¦«}|t |<|S)u¥Return *factory* augmented with untrack-on-close. Callers legitimately supply their own ``Connection`` subclasses (the test suite uses them to simulate FTS5-less or pragma-failing runtimes). Rather than refusing those â€” or silently leaving them untracked, which would quietly unguard the database â€” we mix the tracking ``close()`` into the caller's class so tracking is preserved either way. NعTracked) r#ع ConnectionrCع issubclassr8rDr-rFr)rEعcacheds rع_tracking_factoryrL¾sy€ً•'ش$ذ$ذ$ف ذ فگ'‌>ر*ش*ًطˆف #× 'ز '¨ر 0ش 0€Fط €~فذ2 ش 0ذ2ذ2µ^ہWذ4MبrرRشRˆط*0صکwر'ط€Mr)ع tracking_pathع connect_fnrMْPath | str | Nonecَz—|پ|ntj}t| dtj¦«¦«|d<t 5|t |¦«fi|¤ژ} |پt|¦«nt|¦«}|€|cddd¦«St|t¦«st||¦«}||_t |d¦«dzt|<|cddd¦«S#t$r3 tj |¦«n#t$rYnwxYw‚wxYw#1swxYwYdS)u``sqlite3.connect`` that registers the connection for the lifetime of the fd. Use for any connection to a database whose file might otherwise be byte-probed (``state.db``, ``kanban.db``). The registration is released automatically on ``close()``. The open and the registration happen together under ``_live_lock``, so a concurrent :func:`read_header_bytes_preopen` cannot slip between them and cancel this connection's locks. The registry key is the canonical path reported by ``PRAGMA database_list`` -- not *path*, which may be a ``file:`` URI. Pass ``tracking_path`` to override when the caller already knows the real path. ``connect_fn`` lets a caller supply its own opener (defaults to :func:`sqlite3.connect`), so a module that owns the connection â€” and any test that patches that module's ``sqlite3.connect`` â€” keeps control of how the connection is created while this helper owns tracking. A caller-supplied ``factory`` is honoured but is transparently augmented with untrack-on-close, so tracking is never silently skipped. If a file-backed connection still cannot be tracked, :class:`UntrackableConnectionError` is raised rather than handing back a connection whose database has quietly lost byte-probe protection. NrErr+)r#عconnectrLr-rIr,rrr(ع isinstancer8ع_retrofit_trackingr9rع Exceptionr=)rrMrNعkwargsعopenerrعresolveds rعconnect_trackedrXزsذ€ً@&ذ1ˆZˆZµw´€Fف)¨&¯*ھ*°Yإش@Rر*Sش*SرTشT€Fˆ9ره ًًطˆv•cک$‘i”iذ*ذ* 6ذ*ذ*ˆً ً!ذ,ُگ]ر#ش#ذ#ه'¨ر-ش-ً ً ذàًًًًٌôًًُکd¥Nر3ش3ً :ُ*¨$°ر9ش9گط(0ˆDش%ف*;×*?ز*?ہب!ر*Lش*Lبqر*Pصکhر'طً)ًًًٌôًًُّ*ً ً ً ً ُش"×(ز(¨ر.ش.ذ.ذ.ّفً ً ً طگً ّّّàً ًّّّ+ًًًًًًًًًّّّsOءD0ء #C0آAC0أ0 D-أ;DؤD-ؤ D(ؤ%D-ؤ'D(ؤ(D-ؤ-D0ؤ0D4ؤ7D4rWcَط—t|¦«}t|t¦«r|S t|¦«|_|S#t $r!}t d|›d|j›d‌¦«|‚d}~wwxYw)a!Give an already-open connection untrack-on-close semantics. ``sqlite3.Connection`` subclasses are ordinary Python classes, so the instance's ``__class__`` can be swapped for one that mixes in the tracking ``close()``. Used when an opener ignored the factory we asked for. zconnection to z uses factory zn, which cannot release its tracking entry on close; byte-probe safety for this database would be silently lostN)rFrJr8rLr?ع TypeErrorr r)rrWعclsعexcs rrSrSs™€ُˆt‰*Œ*€Cفگ#•~ر&ش&ًطˆًف*¨3ر/ش/ˆŒطˆّفًًًف(ً 7کXً 7ً 7°S´\ً 7ً 7ً 7ٌ ô ًً ًّّّّّّّs¨>¾ A)ءA$ء$A)ْ Optional[int]cَ´— | d¦« ¦«d}| d¦« ¦«d}nE#tjtt f$r&}t d|¦«Yd}~dSd}~wwxYw t|¦«t|¦«zS#ttf$rYdSwxYw)aNLogical database size in bytes, read through *conn*. ``page_count * page_size`` is the same quantity the 4-byte header field at offset 28 carries, but reading it via ``PRAGMA`` opens no new file descriptor and therefore cannot cancel this process's POSIX locks. Returns ``None`` when the pragmas cannot be read. zPRAGMA page_countrzPRAGMA page_sizez$page_count/page_size unavailable: %sN) r!r"r#r$rZع IndexErrorعloggerعdebugعintع ValueError)rع page_countع page_sizer\s rعpage_count_bytesrf)sـ€ًط—\’\ذ"5ر6ش6×?ز?رAشAہ!شDˆ ط—L’Lذ!3ر4ش4×=ز=ر?ش?ہشBˆ ˆ ّفŒM‌9¥jذ1ًًًفڈٹذ;¸SرAشAذAطˆtˆtˆtˆtˆtًًّّّّّّّفگ:‰Œ¥ Y،¤ر/ذ/ّف•zذ"ًًًطˆtˆtًّّّs*‚AAءBء9BآBآ#CأCأCْOptional[bool]cَ¾—t|¦«}|€dSt|¦«}|sdS tj |¦«}n#t $rYdSwxYw||kS)aWhether the file on disk is at least as long as the header claims. Detects the "torn extend" shape (file shorter than its own page count) without ever opening the database file: the header side comes from ``PRAGMA page_count`` over *conn*, and the on-disk side from ``stat()``, which takes no descriptor and cannot break locks. Returns ``None`` when the check is not applicable (in-memory database, unreadable pragmas, or a stat failure). Note: in WAL mode a freshly committed page may still live in the ``-wal`` file, so the main file legitimately lags. Callers must treat this as advisory unless the database is in a rollback journal mode. N)r(rfعosrعgetsizer)rr'عlogicalعactuals rعfile_length_matches_headerrm>s}€ُ" $ر'ش'€Hطذطˆtهکtر$ش$€Gطًطˆtًف”—’ ر*ش*ˆˆّفًًًطˆtˆtًّّّàگWزذs¨Aء AءAédF)عlengthعforcerorbrpْOptional[bytes]cَک—t5|s?t|¦«tvr)t d|¦« ddd¦«dS t|d¦«5}| |¦«cddd¦«cddd¦«S#1swxYwYn#t$rYddd¦«dSwxYw ddd¦«dS#1swxYwYdS)aVRead the first *length* bytes of *path* -- only when no connection is live. This is the ONLY sanctioned byte-level read of a database file, and it is restricted to first-open validation (is this file a real SQLite database, is it zeroed, has it been overwritten by something else). Once any connection to *path* exists in this process, the read is refused and ``None`` is returned, because the ``close()`` would cancel that connection's POSIX locks. The registry check and the ``open``/``read``/``close`` are performed together under ``_live_lock``, so a connection cannot be opened in the window between deciding "nothing is live" and closing this descriptor. Set ``force=True`` only for genuinely offline files (quarantined copies, snapshot artifacts, archives) that no live connection can reference. zqrefusing byte-level read of %s: a live connection exists in this process and close() would cancel its POSIX locksNعrb)r,rrr`raعopenعreadr)rrorpعhandles rعread_header_bytes_preopenrw[sء€ُ, ًًطً ‌کd™œص'8ذ8ذ8فڈLٹLًHàٌ ô ً ً ًًًٌôًًًً فگdکDر!ش!ً + Vط—{’{ 6ر*ش*ً +ً +ً +ً +ٌ +ô +ً +ًًًًٌôًًً +ً +ً +ً +ًّّّ +ً +ً +ً +ً +ّهً ً ً طًًًٌôًًًً ًّّّ +ًًًٌôًًًًًًًًًًًًًّّّsYˆ5B?ءBءB ء0Bآ B آ BآB آBآB?آ B.آB?آ-B.آ.B?آ?CأCcَ—eZdZdZdS)عLiveConnectionErrorzGA raw file operation was attempted on a database with live connections.Nr rrrryry€s€€€€€طQذQذQذQrryru)عwhatrzc#َھK—t5t|¦«tvrtd|›d|›d‌¦«‚dV—ddd¦«dS#1swxYwYdS)u§Hold the connection-lifecycle lock across a raw read of a database file. Checking :func:`has_live_connection` and *then* doing the raw I/O is a check/use race: a connection can be opened in the window between the two, and the raw ``close()`` will cancel its POSIX advisory locks â€” the exact failure class the registry exists to prevent. Any multi-step raw access (copying a database plus its ``-wal``/``-shm``/``-journal`` sidecars, hashing a file, moving a bundle aside) must therefore run *inside* this context manager rather than after a bare check. While held, :func:`connect_tracked` blocks, so no new connection can appear mid-copy. Raises :class:`LiveConnectionError` if a connection is already live when the guard is entered. The lock is only held for the duration of the raw I/O; it never spans caller work on an open connection, so it does not serialise database use. zRefusing to ع z؟: a connection to it is still open in this process, and raw file access would cancel that connection's POSIX advisory locks. Close all database handles (stop the gateway/dashboard) and retry.N)r,rrry)rrzs rعoffline_file_accessr}„sةèè€ُ& ًًفگ‰:Œ:ص*ذ*ذ*ف%ًBکtًBًB dًBًBًBٌôً ً ˆˆˆًًًٌôًًًًًًًًًًًًًّّّsٹ1AءAءA)rrrr)rrrr)rrrr))rrrr4)rErFrrF)rrrMrOrr)rrrWrrr)rrrr])rrrrg)rrrorbrpr4rrq)rrrzr))rع __future__rع contextlibعloggingrir#ع threadingعpathlibrعtypingrع getLoggerrr`عSQLITE_HEADER_MAGICع_HEADER_PAGE_COUNT_OFFSETعRLockr,rr@عRuntimeErrorr rr(r/r3r6r8rIrCrDrLrXrSrfrmrwryعcontextmanagerr}rrrْrٹsجًً;ً;ً;ًz#ذ"ذ"ذ"ذ"ذ"àذذذط€€€ط € € € ط€€€طذذذطذذذذذطذذذذذà ˆش ک8ر $ش $€à,ذًذًˆYŒ_ر ش € à$&ذذ&ذ&ذ&ر&ًًًًً ٌôًًًًًًًًًً&CًCًCًCً-ً-ً-ً-ً/ً/ً/ً/ً)ً)ً)ً)ً)ٌ)ô)ً)ً$ًًًًک¨ش(:ٌôًً(,.ذذ-ذ-ذ-ر-ًًًًً.(,طً ?ً?ً?ً?ً?ً?ًDًًًً*ًًًً*ًًًً@طً "ً"ً"ً"ً"ً"ًJRًRًRًRًRک,ٌRôRًRًشط9?ًًًًًٌشًًًr