U -e)G @sddlmZddlmZmZmZddlmZmZddl m Z ddl m Z ddlmZmZmZdddd d d d Zdddd d d dZddd dddZdddddddddddd dddZdddddddddddd dddZddddddddddddddd Zdddddddddddddd!d"Zddd dd#d$Zddd%ddddd&d'd(d)Zddd%ddddd*d'd+d,ZdS)-) annotations)CallableHashableSequence) common_affixconv_sequences)is_none)Indel_py)EditopEditopsOpcodeszSequence[Hashable]ztuple[int, int, int]int)s1s2weightsreturnc Cslt|}t|}|\}}}||||}||krNt||||||}nt||||||}|S)N)lenmin) rrrlen1Zlen2insertdeletereplaceZmax_distrb/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/rapidfuzz/distance/Levenshtein_py.py_levenshtein_maximums rc Cst|}|\}}}ttd|d||}|D]v}|d} |d|7<t|D]P} | } || |krt|| ||| d|| |} || d} | || d<qRq.|dS)Nr)rlistranger) rrrrrrrcachech2tempixrrr_uniform_generics   $ r$)rrrcCs|s t|Sdt|>d}d}t|}dt|d>}i}|j}d}|D]} || d|B|| <|dK}qJ|D]} || d} | } | |@||A| B|B} || |BB}| |@}|||@dk7}|||@dk8}|d>dB}|d>}|| |BB}|| @}qn|SNrr)rget)rrVPVNcurrDistmaskblock block_getr#ch1r PM_jXD0HPHNrrr_uniform_distance/s2    r3rrrN)r processor score_cutoff score_hintztuple[int, int, int] | Nonez(Callable[..., Sequence[Hashable]] | Nonez int | None)rrrr5r6r7rcCs|}|dk r||}||}t||\}}|dks:|dkrFt||}n"|dkr\t||}n t|||}|dksx||kr||S|dS)a Calculates the minimum number of insertions, deletions, and substitutions required to change one sequence into the other according to Levenshtein with custom costs for insertion, deletion and substitution Parameters ---------- s1 : Sequence[Hashable] First string to compare. s2 : Sequence[Hashable] Second string to compare. weights : Tuple[int, int, int] or None, optional The weights for the three operations in the form (insertion, deletion, substitution). Default is (1, 1, 1), which gives all three operations a weight of 1. processor : callable, optional Optional callable that is used to preprocess the strings before comparing them. Default is None, which deactivates this behaviour. score_cutoff : int, optional Maximum distance between s1 and s2, that is considered as a result. If the distance is bigger than score_cutoff, score_cutoff + 1 is returned instead. Default is None, which deactivates this behaviour. score_hint : int, optional Expected distance between s1 and s2. This is used to select a faster implementation. Default is None, which deactivates this behaviour. Returns ------- distance : int distance between s1 and s2 Raises ------ ValueError If unsupported weights are provided a ValueError is thrown Examples -------- Find the Levenshtein distance between two strings: >>> from rapidfuzz.distance import Levenshtein >>> Levenshtein.distance("lewenstein", "levenshtein") 2 Setting a maximum distance allows the implementation to select a more efficient implementation: >>> Levenshtein.distance("lewenstein", "levenshtein", score_cutoff=1) 2 It is possible to select different weights by passing a `weight` tuple. >>> Levenshtein.distance("lewenstein", "levenshtein", weights=(1,1,2)) 3 Nr4)rrr)rr3Indeldistancer$)rrrr5r6r7_distrrrr:SsB  r:c Csl|}|dk r||}||}t||\}}|p0d}t|||}t|||d}||} |dksd| |krh| SdS)a Calculates the levenshtein similarity in the range [max, 0] using custom costs for insertion, deletion and substitution. This is calculated as ``max - distance``, where max is the maximal possible Levenshtein distance given the lengths of the sequences s1/s2 and the weights. Parameters ---------- s1 : Sequence[Hashable] First string to compare. s2 : Sequence[Hashable] Second string to compare. weights : Tuple[int, int, int] or None, optional The weights for the three operations in the form (insertion, deletion, substitution). Default is (1, 1, 1), which gives all three operations a weight of 1. processor : callable, optional Optional callable that is used to preprocess the strings before comparing them. Default is None, which deactivates this behaviour. score_cutoff : int, optional Maximum distance between s1 and s2, that is considered as a result. If the similarity is smaller than score_cutoff, 0 is returned instead. Default is None, which deactivates this behaviour. score_hint : int, optional Expected similarity between s1 and s2. This is used to select a faster implementation. Default is None, which deactivates this behaviour. Returns ------- similarity : int similarity between s1 and s2 Raises ------ ValueError If unsupported weights are provided a ValueError is thrown Nr4rr)rrr:) rrrr5r6r7r;maximumr<simrrr similaritys0 r@z float | Nonefloatc Cs|}t|st|rdS|dk r0||}||}t||\}}|pDd}t|||}t|||d}|rl||nd} |dks| |kr| SdS)a Calculates a normalized levenshtein distance in the range [1, 0] using custom costs for insertion, deletion and substitution. This is calculated as ``distance / max``, where max is the maximal possible Levenshtein distance given the lengths of the sequences s1/s2 and the weights. Parameters ---------- s1 : Sequence[Hashable] First string to compare. s2 : Sequence[Hashable] Second string to compare. weights : Tuple[int, int, int] or None, optional The weights for the three operations in the form (insertion, deletion, substitution). Default is (1, 1, 1), which gives all three operations a weight of 1. processor : callable, optional Optional callable that is used to preprocess the strings before comparing them. Default is None, which deactivates this behaviour. score_cutoff : float, optional Optional argument for a score threshold as a float between 0 and 1.0. For norm_dist > score_cutoff 1.0 is returned instead. Default is None, which deactivates this behaviour. score_hint : float, optional Expected normalized distance between s1 and s2. This is used to select a faster implementation. Default is None, which deactivates this behaviour. Returns ------- norm_dist : float normalized distance between s1 and s2 as a float between 1.0 and 0.0 Raises ------ ValueError If unsupported weights are provided a ValueError is thrown ?Nr4r=rr)rrrr:) rrrr5r6r7r;r>r< norm_distrrrnormalized_distances/ rDc Cst|}t|st|rdS|dk r0||}||}t||\}}|pDd}t|||d}d|}|dksl||krp|SdS)a Calculates a normalized levenshtein similarity in the range [0, 1] using custom costs for insertion, deletion and substitution. This is calculated as ``1 - normalized_distance`` Parameters ---------- s1 : Sequence[Hashable] First string to compare. s2 : Sequence[Hashable] Second string to compare. weights : Tuple[int, int, int] or None, optional The weights for the three operations in the form (insertion, deletion, substitution). Default is (1, 1, 1), which gives all three operations a weight of 1. processor : callable, optional Optional callable that is used to preprocess the strings before comparing them. Default is None, which deactivates this behaviour. score_cutoff : float, optional Optional argument for a score threshold as a float between 0 and 1.0. For norm_sim < score_cutoff 0 is returned instead. Default is None, which deactivates this behaviour. score_hint : int, optional Expected normalized similarity between s1 and s2. This is used to select a faster implementation. Default is None, which deactivates this behaviour. Returns ------- norm_sim : float normalized similarity between s1 and s2 as a float between 0 and 1.0 Raises ------ ValueError If unsupported weights are provided a ValueError is thrown Examples -------- Find the normalized Levenshtein similarity between two strings: >>> from rapidfuzz.distance import Levenshtein >>> Levenshtein.normalized_similarity("lewenstein", "levenshtein") 0.81818181818181 Setting a score_cutoff allows the implementation to select a more efficient implementation: >>> Levenshtein.normalized_similarity("lewenstein", "levenshtein", score_cutoff=0.85) 0.0 It is possible to select different weights by passing a `weight` tuple. >>> Levenshtein.normalized_similarity("lewenstein", "levenshtein", weights=(1,1,2)) 0.85714285714285 When a different processor is used s1 and s2 do not have to be strings >>> Levenshtein.normalized_similarity(["lewenstein"], ["levenshtein"], processor=lambda s: s[0]) 0.81818181818181 gNr4r=rBr)rrrD) rrrr5r6r7r;rCZnorm_simrrrnormalized_similarity!sGrEcCs&|st|ggfSdt|>d}d}t|}dt|d>}i}|j}d}|D]} || d|B|| <|dK}qPg} g} |D]} || d} | }||@||A|B|B}|||BB}||@}|||@dk7}|||@dk8}|d>dB}|d>}|||BB}||@}| || |q||| | fSr%)rr&append)rrr'r(r)r*r+r,r#r-Z matrix_VPZ matrix_VNr r.r/r0r1r2rrr_matrixws:     rGr5r7r )rrr5r7rcCs|}|dk r||}||}t||\}}t||\}}||t||}||t||}t||\}}} tgdd} t|||| _t|||| _|dkr| Sdg|} t|} t|} | dkr| dkr|| dd| d>@r|d8}| d8} td| || || |<q| d8} | rh| | dd| d>@rh|d8}td| || || |<q| d8} || || kr|d8}td| || || |<q| dkr|d8}| d8} td| || || |<q| dkr|d8}| d8} td| || || |<q| | _| S)u  Return Editops describing how to turn s1 into s2. Parameters ---------- s1 : Sequence[Hashable] First string to compare. s2 : Sequence[Hashable] Second string to compare. processor : callable, optional Optional callable that is used to preprocess the strings before comparing them. Default is None, which deactivates this behaviour. score_hint : int, optional Expected distance between s1 and s2. This is used to select a faster implementation. Default is None, which deactivates this behaviour. Returns ------- editops : Editops edit operations required to turn s1 into s2 Notes ----- The alignment is calculated using an algorithm of Heikki Hyyrö, which is described [8]_. It has a time complexity and memory usage of ``O([N/64] * M)``. References ---------- .. [8] Hyyrö, Heikki. "A Note on Bit-Parallel Alignment Computation." Stringology (2004). Examples -------- >>> from rapidfuzz.distance import Levenshtein >>> for tag, src_pos, dest_pos in Levenshtein.editops("qabxcd", "abycdf"): ... print(("%7s s1[%d] s2[%d]" % (tag, src_pos, dest_pos))) delete s1[1] s2[0] replace s1[3] s2[2] insert s1[6] s2[5] Nrrrrr) rrrrGr Z_src_lenZ _dest_lenr Z_editops)rrr5r7r; prefix_lenZ suffix_lenr<r'r(editopsZ editop_listcolrowrrrrJsP/     rJr cCst||||dS)u Return Opcodes describing how to turn s1 into s2. Parameters ---------- s1 : Sequence[Hashable] First string to compare. s2 : Sequence[Hashable] Second string to compare. processor : callable, optional Optional callable that is used to preprocess the strings before comparing them. Default is None, which deactivates this behaviour. score_hint : int, optional Expected distance between s1 and s2. This is used to select a faster implementation. Default is None, which deactivates this behaviour. Returns ------- opcodes : Opcodes edit operations required to turn s1 into s2 Notes ----- The alignment is calculated using an algorithm of Heikki Hyyrö, which is described [9]_. It has a time complexity and memory usage of ``O([N/64] * M)``. References ---------- .. [9] Hyyrö, Heikki. "A Note on Bit-Parallel Alignment Computation." Stringology (2004). Examples -------- >>> from rapidfuzz.distance import Levenshtein >>> a = "qabxcd" >>> b = "abycdf" >>> for tag, i1, i2, j1, j2 in Levenshtein.opcodes("qabxcd", "abycdf"): ... print(("%7s a[%d:%d] (%s) b[%d:%d] (%s)" % ... (tag, i1, i2, a[i1:i2], j1, j2, b[j1:j2]))) delete a[0:1] (q) b[0:0] () equal a[1:3] (ab) b[0:2] (ab) replace a[3:4] (x) b[2:3] (y) equal a[4:6] (cd) b[3:5] (cd) insert a[6:6] () b[5:6] (f) rH)rJZ as_opcodes)rrr5r7rrropcodess5rM) __future__rtypingrrrZrapidfuzz._common_pyrrZrapidfuzz._utilsrZrapidfuzz.distancer r9Z!rapidfuzz.distance._initialize_pyr r r rr$r3r:r@rDrErGrJrMrrrrsF   (VACV-k