epjdZddlZddlmZmZmZmZddlmZidddddd d d d d d ddddddddddddddddddddddddddddddZ de de fdZ d e d!e d"e de fd#Z dWd e d%eeeefd&ede fd'Z dXd e d!e d"e d)e dee eee ee ff d*Zd e d%eeeefd!e d"e dee f d+Zd,e de fd-Zde dee fd.Zd/e d!e d"e de fd0Zd"e d e d%eeeefde fd1Zd e d%eeeefd!e d"e de f d2Z dYd e d%eeeefd"e d!ee de f d3Zd e d4e deeeeffd5Zd e d4e deeeeffd6Zd e d4e deeeeffd7Zd e d4e deeeeffd8Zd e d4e deeeeffd9Zd e d4e deeeeffd:Zd;e deefd<Zd=eed>eeeefdeeeeffd?Zd e d4e deeeeffd@Z d e d4e deeeeffdAZ!d e d4e deeeeffdBZ"dCee dDedEedFedeeeff dGZ#d e dCee dHee d4e dIe deeeeff dJZ$d;e dKe dLeeeefdeeeeffdMZ%d,e de fdNZ&dZd!e d e dQedRede f dSZ'dTee dUed!e d e de f dVZ(dS)[aP Fuzzy Matching Module for File Operations Implements a multi-strategy matching chain to robustly find and replace text, accommodating variations in whitespace, indentation, and escaping common in LLM-generated code. The 9-strategy chain (inspired by OpenCode), tried in order: 1. Exact match - Direct string comparison 2. Line-trimmed - Strip leading/trailing whitespace per line 3. Whitespace normalized - Collapse multiple spaces/tabs to single space 4. Indentation flexible - Ignore indentation differences entirely 5. Escape normalized - Convert \n literals to actual newlines 6. Trimmed boundary - Trim first/last line whitespace only 7. Block anchor - Match first+last lines, use similarity for middle 8. Context-aware - 50% line similarity threshold Multi-occurrence matching is handled via the replace_all flag. Usage: from tools.fuzzy_match import fuzzy_find_and_replace new_content, match_count, strategy, error = fuzzy_find_and_replace( content="def foo():\n pass", old_string="def foo():", new_string="def bar():", replace_all=False ) N)TupleOptionalListCallable)SequenceMatcheru“"u”u‘'u’u—z--u–-u…...  u−u u u u u u u u )u u u u u u　textreturncptD]\}}|||}|S)zBNormalizes Unicode characters to their standard ASCII equivalents.) UNICODE_MAPitemsreplace)rcharrepls 7/home/thesage/.hermes/hermes-agent/tools/fuzzy_match.py_unicode_normalizer<s;!''))(( d||D$'' Kcontent old_string new_stringcx|r%t|dkrdS||vrdS||krdS||vS)uReturn True when the requested edit is already present in the file. Production trajectory mining shows the most common patch failure is a re-send of an edit that already landed (old_string == new_string, or old_string gone while new_string is present) — the model's intent is "make the file contain this text", and it already does. Callers use this to convert those errors into an explicit success-shaped no-op so the model moves on instead of re-reading and re-patching. Deliberately conservative: - new_string must be non-trivial (>= 8 chars stripped) — a tiny target matching by coincidence must not mask a genuine typo'd edit; - new_string must appear EXACTLY in the content (no fuzzy matching — approximate presence is not proof the edit landed); - when old_string differs from new_string, old_string must be GONE (still-present old text means the edit is at best half-applied). FT)lenstrip)rrrs ris_already_appliedr CsZ$ Z--//00144u  uZt W $$rmatchescapcg}|d|D]\}}|dd|dz}|dd|dz}|d|}|dkrt|}|||} t| dkr | dddz} |d |d | t||z } | dkr|d | d d|S) aRender up to ``cap`` match positions as 'L: ' rows. Gives the model the information it needs to disambiguate an ambiguous old_string in ONE follow-up (add neighboring context, or choose replace_all) instead of re-reading the file to find the occurrences. N rPMr z Lz: z ... and z more)countrfindfindrrappendjoin) rr"r#rowsstart_endline_no line_startline_endsnippetextras r_format_match_locationsr7^s* Dtt} 0 0 t--a//!3]]4E22Q6 <<j11 r>>7||H*X-.4466 w<<"  crclU*G .'..W..//// LL3 E qyy ----... 99T??rF replace_allc |s|dddfS|s|dddfS||kr|dddfSdtfdtfdtfd tfd t fd t fd tfd tfdtfg }d dh}|D]\}}|||}|rt|dkr-|s+t||} |dddt|d| fcS|r3t|dkr ||vr|dddt|d|dfcS|dkrt||||} | r|dd| fcSt|||} |d krt|||| } t||| |dkr|nd} | t||dfcS|dddfS)a) Find and replace text using a chain of increasingly fuzzy matching strategies. Args: content: The file content to search in old_string: The text to find new_string: The replacement text replace_all: If True, replace all occurrences; if False, require uniqueness Returns: Tuple of (new_content, match_count, strategy_name, error_message) - If successful: (modified_content, number_of_replacements, strategy_used, None) - If failed: (original_content, 0, None, error_description) rNzold_string cannot be emptyuAold_string is only whitespace — provide non-blank text to matchz'old_string and new_string are identicalexact line_trimmedwhitespace_normalizedindentation_flexibleescape_normalizedtrimmed_boundaryunicode_normalized block_anchor context_awarer&zFound zc matches for old_string. Provide more context to make it unique, or use replace_all=True. Matches: z approximate matches via the 'z' strategy; replace_all only applies to exact matches. Provide the precise text (whitespace included) so an exact/line-trimmed match can be made.)rz1Could not find a match for old_string in the file)r_strategy_exact_strategy_line_trimmed_strategy_whitespace_normalized_strategy_indentation_flexible_strategy_escape_normalized_strategy_trimmed_boundary_strategy_unicode_normalized_strategy_block_anchor_strategy_context_awarerr7_detect_escape_drift_maybe_unescape_new_string _preserve_unicode_in_replacement_apply_replacements) rrrr8 strategies_SIMILARITY_STRATEGIES strategy_name strategy_fnr" locations drift_err effective_new new_contents rfuzzy_find_and_replacerXws >4!===     e 4!dddZ4!JJJ /" /0 "AB !?@ 9: 78 ;< /0 12 .J"-o>&0RBRB" {+gz22 O B7||a 3GWEE 4-S\\--!*-- s7||a//ME[4[4[4=S\\==%==="''0':zZZ 7"AtY666627GWM 444 @Wj-!! .-)6')A)A::tKG mTA A A A_O Bd AtP PPrcd|vrd|vrdSdfd|D}dD]!}||vr||vr||vr|d}d|d |d cS"dS) u'Detect tool-call escape-drift artifacts in new_string. Looks for ``\'`` or ``\"`` sequences that are present in both old_string and new_string (i.e. the model copy-pasted them as "context" it intended to preserve) but don't exist in the matched region of the file. That pattern indicates the transport layer inserted spurious shell-style escapes around apostrophes or quotes — writing new_string verbatim would literally insert ``\'`` into source code. Returns an error string if drift is detected, None otherwise. \'\"Nc32K|]\}}||VdSN.0r0endrs r z'_detect_escape_drift..0KKZUCgeCi0KKKKKKr)rZr[r&zNEscape-drift detected: old_string and new_string contain the literal sequence a but the matched region of the file does not. This is almost always a tool-call serialization artifact where an apostrophe or quote got prefixed with a spurious backslash. Re-read the file with read_file and pass old_string/new_string without backslash-escaping z characters.)r.)rr"rrmatched_regionssuspectplains` rrLrLs  J5 #:#:t ggKKKK7KKKKKO!   j W %:%:wo?]?]AJE<(/<< ', <<<    4rlinecd}|t|kr,||dvr"|dz }|t|kr ||dv"|d|S)z=Return the leading whitespace prefix of a line (spaces/tabs).rr  r&Nr)rhis r_leading_whitespacern(sY A c$ii--DG{22 Q c$ii--DG{22 8Orcf|dD]}|r|cSdS)zReturn the first line of ``text`` that has any non-whitespace content. Returns ``None`` if no such line exists (text is empty or all whitespace). r%N)splitr)rrhs r_first_meaningful_linerq0sB  4   ::<< KKK  4r file_regioncN|s|St|}t|}|||St|}t|}||kr|Sg}|dD]}|s||,t|} | |r0|t |d} ||| z|||dzd|S)a[Adjust ``new_string`` so its indentation matches ``file_region``. Used after a non-exact fuzzy match: the LLM may have sent old_string and new_string with a different indent than the file actually has (e.g. 2-space indent in tool args vs 4-space indent on disk). The fuzzy strategy successfully matched anyway, but writing ``new_string`` verbatim would corrupt the file's indentation. Approach: 1. For each non-blank line in ``new_string``, compute its indent *relative* to the shallowest non-blank line of ``old_string`` (the LLM's base indent). 2. Anchor that relative indent onto the file's actual base indent (the leading whitespace of the file_region's first non-blank line). 3. Re-emit each non-blank line as ``file_base + (line_indent - llm_base)``. Blank lines and lines less-indented than the LLM's base are anchored directly to the file's base indent. No-op cases (returns ``new_string`` unchanged): - file_region or old_string has no meaningful line - LLM base indent equals file base indent - new_string is empty Nr% ) rqrnrprr- startswithrlstripr.) rrrr old_first file_first old_indent file_indent out_linesrh line_indent remainders r_reindent_replacementr~;sF4 &z22I' 44JJ.$Y//J%j11K[  I  &&??zz||    T " " " )$//  ! !* - - ?S__--.I   [94 5 5 5 5   [4;;u+=+== > > > > 99Y  rcd|vrd|vr|Sdfd|D}|}d|vrd|vr|dd}d|vrd|vr|dd}|S)u Conditionally unescape ``\t``/``\r`` in new_string. LLMs frequently send the two-character sequences ``\t`` (backslash + t) and ``\r`` (backslash + r) inside JSON tool-call arguments where they meant a real tab or carriage-return byte. Writing the string verbatim corrupts tab-indented files with literal backslash-letter pairs. The unescape is only applied per-sequence when the *matched region of the file* actually contains the corresponding control character — that is, we only convert ``\t`` -> tab when the file region we're replacing contains a real tab byte. Files that legitimately contain the literal two-character string ``"\t"`` (e.g. a Python source line that defines ``sep = "\t"``) get a backslash+t in the matched region instead of a tab, so we leave new_string alone. ``\n`` is intentionally excluded: newlines serialize correctly through JSON and rewriting backslash-n would corrupt escape sequences in string literals far more often than it would help. \t\rr\c32K|]\}}||VdSr^r_r`s rrcz-_maybe_unescape_new_string..rdrrk )r.r)rrr"reouts ` rrMrM|s0 J5 #:#:ggKKKK7KKKKKO C ||//kk%&& ||//kk%&& Jrc(dfd|D}t|}t|}||kr|St|}i}t|ddD]\} } | |vr| || <t d||} | } g} | D]\}}}}}|dkry||d}|}|t|kr0|||kr$|dz }|t|kr |||k$| ||||dkr| ||||d kr|d kr| |||d| S) uPreserve Unicode characters from the file in the replacement string. When strategy 7 (unicode_normalized) matched, the file has Unicode characters (em-dashes, smart quotes, ellipsis, non-breaking spaces) but old_string/new_string from the LLM are ASCII equivalents. Writing new_string verbatim would silently corrupt the file's Unicode — em-dashes become two hyphens, smart quotes become straight quotes. This function aligns the replacement with the file's actual Unicode by diffing old_string→new_string and applying only the actual edits to the file's original text, preserving Unicode for unchanged portions. r\c32K|]\}}||VdSr^r_r`s rrcz3_preserve_unicode_in_replacement..s0GG'%),GGGGGGrNr'equalrr&rdeleteinsert) r.r_build_orig_to_norm_map enumerater get_opcodesgetrr-)rr"rrrrnorm_old norm_filefile_orig_to_normfile_norm_to_origorig_posnpsmopcodes result_partstagi1i2j1j2 orig_startorig_ends` rrNrNs$''GGGGwGGGGGK"*--H";//I90 <<(*!"3CRC"899-- " & & &$, b ! x 4 4BnnG!L&33RR '>>*..r155J!H3{++++%h/"44A 3{++++%h/"44    Jx,? @ A A A A I      2b5 1 2 2 2 2 H__  H__    2b5 1 2 2 2 77<  rct|dd}|}|D]=\}}||||}t|||} n|} |d|| z||dz}>|S)a Apply replacements at the given positions. Args: content: Original content matches: List of (start, end) positions to replace new_string: Replacement text old_string: When non-None, signals that the match came from a non-exact fuzzy strategy; ``new_string`` is re-indented to match the file's actual indentation before substitution. Returns: Content with replacements applied c|dSNrr_xs rz%_apply_replacements..s 1Q4rT)keyreverseN)sortedr~) rr"rrsorted_matchesresultr0rbrradjusteds rrOrOs$GFFFN F$:: s  !!%),K,[*jQQHH!H(*VCDD\9 Mrpatterncg}d} |||}|dkrn:|||t|zf|t|z}W|S)zStrategy 1: Exact string match.rTr')r,r-r)rrr"r0poss rrCrCspG E #ll7E** "99 S3w<</0111 c'll" # Nrcd|dD}d|}|d}d|D}t|||||S)z Strategy 2: Match with line-by-line whitespace trimming. Strips leading/trailing whitespace from each line before matching. c6g|]}|Sr_rrarhs r z*_strategy_line_trimmed..!s BBBdTZZ\\BBBrr%c6g|]}|Sr_rrs rrz*_strategy_line_trimmed..%s GGG GGGr)rpr._find_normalized_matches)rr pattern_linespattern_normalized content_linescontent_normalized_liness rrDrDs}CBgmmD.A.ABBBM=11MM$''MGGGGG $ 8#  rc~d}||}||}t||}|sgSt|||S)zC Strategy 3: Collapse multiple whitespace to single space. c.tjdd|S)Nz[ \t]+r )resubss r normalizez2_strategy_whitespace_normalized..normalize2svia(((r)rC_map_normalized_positions)rrrrcontent_normalizedmatches_in_normalizeds rrErE.sj)))#7++"7++,,>@RSS   %W.@BW X XXrc |d}d|D}d|dD}t||||d|S)z Strategy 4: Ignore indentation differences entirely. Strips all leading whitespace from lines before matching. r%c6g|]}|Sr_rvrs rrz2_strategy_indentation_flexible..Js FFFdkkmmFFFrc6g|]}|Sr_rrs rrz2_strategy_indentation_flexible..Ks CCCtT[[]]CCCr)rprr.)rrrcontent_stripped_linesrs rrFrFCss MM$''MFF FFFCCw}}T/B/BCCCM # 6=))  rcNd}||}||krgSt||S)zt Strategy 5: Convert escape sequences to actual characters. Handles \n -> newline, \t -> tab, etc. c~|ddddddS)Nz\nr%rrkrr)rrs runescapez-_strategy_escape_normalized..unescapeYs6yy%%--eT::BB5$OOOr)rC)rrrpattern_unescapeds rrGrGSsG PPP!))G## 7$5 6 66rc |d}|sgS|d|d<t|dkr|d|d<d|}|d}g}t|}t t||z dzD]}||||z}|} | d| d<t| dkr| d| d<d| |kr ? ?11#Aa*<&<$<= "&&(( $Q--// A {  a  )"o3355KO 99[ ! !%5 5 5!:q!&8"8#g,,"" Iw NNIw/ 0 0 0 Nroriginalcg}d}|D]G}||t|}||t|ndz }H|||S)uBuild a list mapping each original character index to its normalized index. Because UNICODE_MAP replacements may expand characters (e.g. em-dash → '--', ellipsis → '...'), the normalised string can be longer than the original. This map lets us convert positions in the normalised string back to the corresponding positions in the original string. Returns a list of length ``len(original) + 1``; entry ``i`` is the normalised index that character ``i`` maps to. rNr&)r-rrr)rrnorm_posrrs rrrsrFH99 ht$$!1CIIIq8 MM( Mr orig_to_norm norm_matchesc,i}t|ddD]\}}||vr|||<g}t|dz }|D]T\}}||vr ||} | } | |kr#|| |kr| dz } | |kr || |k|| | fU|S)zNConvert (start, end) positions in the normalised string to original positions.Nr'r&)rrr-) rrnorm_to_orig_startrrresultsorig_len norm_startnorm_endrrs r_map_positions_norm_to_origrs *,' SbS(9::44( - - -+3 x (%'G<  1$H , / / H / / / ' 3 !!l8&>L J-lLII  *733L &|\ B BBrc t|}t|}|d}t|dkrgS|d}|d}|d}|d}t|} g} t t|| z dzD]Y} || |kr9|| | zdz |kr| | Zg} t| } | dkrdnd}| D]} | dkrd}nfd|| dz| | zdz }d|dd}td ||}||krRST>T:T'UVVN!YY}QrT':;;N(~~NNTTVVJ  " "!:"Aq+='=s7||"" Iw NNIw/ 0 0 0 Nrc z|d}|d}|sgSt|}|t|krgS|d}|d}d}dtdtdtfd}g} t t||z d zD]} || | |z} ||| d|kr8||| d|krad } t || D]B\} }| }|s|||dkrd } nC| r=0.80) to the aligned content line. The anchor filter keeps this from being an O(file x pattern) scan on every miss, and the all-lines requirement stops a single coincidental line-match from silently replacing an unrelated block (the old 50%-of-lines threshold accepted half-garbage patterns and destroyed the non-matching lines). r%rr'g?abrcX||krdStd||S)Nr)rr)rrs r_simz%_strategy_context_aware.._sim7s. 663tQ**00222rr&TF) rprrstrfloatrziprr-)rrrrr first_patlast_patANCHOR_THRESHOLDrr"rmr all_matchp_linec_line p_strippedrrs rrKrKsMM$''MMM$''M  ]++C .... a &&((IR &&((H3333333 G 3}%%(::Q> ? ?11#Aa*<&<$<=  4 ;q>//11 2 25E E E  4+b///11 2 25E E E  !-==  NFFJ tJ //$66! 7  1!:q!&8"8#g,,"" Iw NNIw/ 0 0 0 Nrr start_lineend_linecontent_lengthctd|d|D}td|d|Ddz }t||}||fS)aCalculate start and end character positions from line indices. Args: content_lines: List of lines (without newlines) start_line: Starting line index (0-based) end_line: Ending line index (exclusive, 0-based) content_length: Total length of the original content string Returns: Tuple of (start_pos, end_pos) in the original content c3:K|]}t|dzVdSr&Nrlrs rrcz,_calculate_line_positions..ls,IIdCIIMIIIIIIrNc3:K|]}t|dzVdSrrlrs rrcz,_calculate_line_positions..ms,EED#d))a-EEEEEErr&)summin)rrrrrrs rrr_smIImKZK.HIIIIIIEEM)8),DEEEEEIG.'**G g rrrc `|d}t|}g}tt||z dzD]d}d||||z} | |kr.s+VV41aa:ooQooooVVr)rr-rr )rrrrorig_idxnorm_idxrnorm_to_orig_endrroriginal_matchesrrrrs @rrrs-  LHH S]] " "x#j//'A'A H H!5 5 5    ) ) ) MH MHH h 5 ( (Z-AS-H-H    ) ) ) MH#h--''HX,>e,K,KA  h 5 ( (    ) ) ) MHH    ) ) ) MH' S]] " "x#j//'A'A, S]] " "C OO,,,A  S]] " " ' 55..( - - -+3 x (%-"" 2LL H + + ++J7JJVVVV9\+B+BVVVVVJ a<+ + +'1 59HH!X %:;H c*oo % %*X\*Bc*I*IS]]**x/AU/J/JA S]]**x/AU/J/J S3x==-I-I JKKKK rcd}g}|t|krO||dvrE|||dkrdnd|dz }|t|kr ||dvEd|||dzS) uRender leading whitespace visibly (→ = tab, · = space). Only the leading run is visualized — interior spacing is rarely the culprit and full visualization makes lines unreadable. rrjrku→·r&r\N)rr-r.)rhrmprefixs r_visualize_whitespacers A F c$ii--DG{22 tAw$eeD999 Q c$ii--DG{22 776??T!""X %%rr context_lines max_resultscz|r|sdS|}||rsdS|d}|sd|D}|sdS|d}g}tD]\\}} | } | std|| } | dkr|| |f]|sdS|d|d|} g} t}| D]\}}td||z tt|t|z|z}|f}||vrQ| |d fd t|z D}| || sdSd | }| dd }||kr8||dkr,|d t|d t|ddz }|S)zFind lines in content most similar to old_string for "did you mean?" feedback. Returns a formatted string showing the closest matching lines with context, or empty string if no useful match is found. r\rc^g|]*}||+Sr_r)rals rrz&find_closest_lines..s-@@@Aaggii@aggii@@@rNg333333?c|d Srr_rs rrz$find_closest_lines..s qter)rr%c3FK|]}|zdzdd|zVdS)r&4dz| Nr_)rajrr0s rrcz%find_closest_lines..&sY  qy1} = = ==#; = =      rz --- r&uF Whitespace difference detected (→ = tab, · = space): file has: z you sent: z. Use the exact whitespace shown in 'file has'.) splitlinesrrrrr-sortsetmaxr raddr.rr)rrrr old_linesanchor candidatesscoredrmrhstrippedrtopparts seen_ranges_line_idxrbrr5r best_linerr0s @@rfind_closest_linesr5s Wr%%''I&&((M Mrq\   ! !F @@@@@  2AF]++&&4::<<  fh77==?? 3;; MM5!* % % % r KKOOK$$$ + C E%%K   8Ax-/00#m$$hY&?-&OPPcl +   ))     3;''       W r ^^E " "Fc!fQi(IF""yIaL'@'@ <0;; < <01>> < < <  Mrerror match_countcv|dkrdS|r|dsdSt||}|sdSd|zS)uReturn a '\n\nDid you mean...' snippet for plain no-match errors. Gated so the hint only fires for actual "old_string not found" failures. Ambiguous-match ("Found N matches"), escape-drift, and identical-strings errors all have ``match_count == 0`` but a "did you mean?" snippet would be misleading — those failed for unrelated reasons. Returns an empty string when there's nothing useful to append. rr\zCould not findz& Did you mean one of these sections? )rur5)r6r7rrhints rformat_no_match_hintr:Bs^ar (()9::r j' 2 2D r 6 ==r)r!)Fr^)rr))__doc__rtypingrrrrdifflibrrrrboolr intr7rXrLrnrqr~rMrNrOrCrDrErFrGrHrrrIrJrKrrrrr5r:r_rrr@s< 222222222222###### cS cS dc e s  c cS cS  c! S! #+C!" c#"!S+ 0SS%%%#%$%%%%8()S4c3h3H!$-0416FQFQCFQSFQcFQ)-FQ:?S(SV-YabeYf@f:gFQFQFQFQR%#%U38_0E%%(%69%>Fsm%%%%Pcc#> s> > > QT> > > > B!3!(+!(,U38_(=!BE!!!!HA! A!c3h0A!A!!$A! A!A!A!A!JFJtE#s(O/D$'5=c]NQHS34c3h3H$C#$uS#X:O(YSY3Y4cSVhCXYYYY* C # $uSRUXBW     77s7tE#s(O?T7777&''c'd5c?>S''''Tcd3i*s)uS#X' %S/:C#CCU3PS8_@UCCCC>5C5#5$uS#X:O5555p?S?3?4c3h;P????LT#YC(+=@EJ3PS8_& c $s) 8 >C>%(>36>;>>>>>>>r