U 祡cq@sdZddlmZmZmZmZmZmZmZm Z m Z ddl Z ddl Z ddl Z ddlmZddlmZmZmZddlmZe rddlmZmZmZd d d gZeed d dZdeeeddddZeedddddZ eeedddZ!Gdd d e"Z#Gdd d e#Z$Gdd d e#Z%dS)zL Provide saving, loading and presenting gathered `ClassTracker` statistics. ) AnyDictIOIterableListOptionalTuple TYPE_CHECKINGUnionN)deepcopy)truncpp pp_timestamp)Asized) TrackedObject ClassTrackerSnapshotStats ConsoleStats HtmlStats)refreturncCs|jddS)N:r)namesplit)rr>/tmp/pip-unpacked-wheel-8ad_c8mj/pympler/classtracker_stats.py_ref2keysr)baseotherlevelrcCs|j|j7_|j|j7_|dkr2t||_t|j|_i}|jD]}||t|<qH|jD]H}t|}||krt||||ddq`|jt|||jd_q`dS)zD Merge **Asized** instances `base` and `other` into `base`. rrr!N) sizeZflatrrlistrefs _merge_asizedappendr )rr r!r&rkeyrrrr's    r'r)trefmergedobjrcCs2d}|jD]\}}||kr |}q |r.t||dS)z Merge the snapshot size information of multiple tracked objects. The tracked object `obj` is scanned for size information at time `tref`. The sizes are merged into **Asized** instance `merged`. N) snapshotsr')r*r+r,r$ timestamptsizerrr_merge_objects3s r0)tracercCsZg}|D]F\}}}}}|r:|D]}|d|dq|d|||fqd|S)z Convert the (stripped) stack-trace to a nice readable format. The stack trace `trace` is a list of frame records as returned by **inspect.stack** but without the frame objects. Returns a string.   z %s:%4d in %s )r(stripjoin)r1linesfnamelinenofuncsrc_linerrr _format_traceAsr>c@seZdZdZddeeeedddZeeee fdddd Z deeee fe dd d d Z ddddZ eddddZddddZddddZdeeeeeffdddZeeedddZdS) rzd Presents the memory statistics gathered by a `ClassTracker` based on user preferences. NzOptional[ClassTracker])trackerfilenamestreamcCsZ|r ||_ntj|_||_i|_g|_|rB|j|_|j|_|j|_g|_|rV||dS)ap Initialize the data log structures either from a `ClassTracker` instance (argument `tracker`) or a previously dumped file (argument `filename`). :param tracker: ClassTracker instance :param filename: filename of previously dumped statistics :param stream: where to print statistics, defaults to ``sys.stdout`` N) rAsysstdoutr?indexr-historysorted load_stats)selfr?r@rArrr__init__Ws zStats.__init__)fdumprcCs6t|trt|d}t||_t||_g|_dS)z Load the data from a dump file. The argument `fdump` can be either a filename or an open file object that requires read access. rbN) isinstancestropenpickleloadrDr-rF)rHrJrrrrGrs     zStats.load_statsT)rJclosercCs\|jr|jt|tr$t|d}tj|j|tjdtj|j |tjd|rX| dS)a Dump the logged data to a file. The argument `file` can be either a filename or an open file object that requires write access. `close` controls if the file is closed before leaving this method (the default behaviour). wb)protocolN) r?stop_periodic_snapshotsrLrMrNrOdumprDHIGHEST_PROTOCOLr-rQ)rHrJrQrrr dump_stats~s   zStats.dump_statsrcCs|js|d}d}|jD]}|j|kr|j}qt|jD]B}|j|D] }||_||_ | ||_ qF|j |j|q8dS)z Prepare the data to be sorted. If not yet sorted, import all tracked objects from the tracked index. Extend the tracking information by implicit information to make sorting easier (DSU pattern). Nr) rFr- tracked_totalr.r%rDkeys classname get_max_sizer$get_size_at_timer/extend)rHZtmaxmaxsizesnapshotr)tobjrrr _init_sorts   zStats._init_sort)argsrcs\d}t|tstds&|dttdfdfdd }||jj|d|S) a- Sort the tracked objects according to the supplied criteria. The argument is a string identifying the basis of a sort (example: 'size' or 'classname'). When more than one key is provided, then additional keys are used as secondary criteria when there is equality in all keys selected before them. For example, ``sort_stats('name', 'size')`` will sort all the entries according to their class name, and resolve all ties (identical class names) by sorting by size. The criteria are fields in the tracked object instances. Results are stored in the ``self.sorted`` list which is used by ``Stats.print_stats()`` and other methods. The fields available for sorting are: 'classname' the name with which the class was registered 'name' the classname 'birth' creation timestamp 'death' destruction timestamp 'size' the maximum measured size of the object 'tsize' the measured size during the largest snapshot 'repr' string representation of the object Note that sorts on size are in descending order (placing most memory consuming items first), whereas name, repr, and creation time searches are in ascending order (alphabetical). The function returns self to allow calling functions on the result:: stats.sort_stats('size').reverse_order().print_stats() )r[r/birthdeathrreprr$zInvalid sort criteriar.)r,rcs>g}D],}t||d}|dkr*t| }||qt|S)Nr4)r/r$)getattrintr(tuple)r,rZattr attributercrr args_to_tuples   z'Stats.sort_stats..args_to_tupler))set issuperset ValueErrorrrMrbrFsort)rHrcZcriteriarmrrlr sort_statss% zStats.sort_statscCs||j|S)zP Reverse the order of the tracked instance index `self.sorted`. )rbrFreverserHrrr reverse_orders zStats.reverse_ordercCs|jD]}||qdS)zD Annotate all snapshots with class-based summaries. N)r-annotate_snapshot)rHr`rrrannotates zStats.annotater)r`rc Cs|jdk r|jSi|_t|jD]}d}d}tdd}|j|D]L}t|j|||||j7}|j|jkrD|j dks|j |jkrD|d7}qDz|d|j }Wnt k rd}YnXz ||}Wnt k rd}YnXt ||||d|j|<||j|d<q$|jS)z@ Store additional statistical data in snapshot. NrrY@)sumavgpctactiver+) classesr%rDrZrr0r.r]rdretotalZeroDivisionErrordict) rHr`r[rr}r+rar|r{rrrrws>         zStats.annotate_snapshotcCstt|jS)z?Return a list of all tracked classes occurring in any snapshot.)rFr%rDrZrurrrtracked_classesszStats.tracked_classes)NNN)T)__name__ __module__ __qualname____doc__rrMrrIr bytesrGboolrWrbrsrvrxrrrwpropertyrrrrrrrQs*   < 'c @sfeZdZdZdeeeeeeedddd Z d dd d d Z de eeddddZ ddddZ dS)rzK Presentation layer for `Stats` to be used in text-based consoles. r2rr皙?N)r&rprefixr!minsizeminpctrc Cst|}|jddd||D]x}|j|kr$|jd||kr$|jdt|t|jdt |jt |jd||f|j |j ||d|dd q$d S) 9 Print individual referents recursively. cSs|jSNr$xrrr)z*ConsoleStats._print_refs..rnryz%-50s %-14s %3d%% [%d] 2z r)rr!N) r%rrrtr$rAwriter rMrr rh _print_refsr&) rHr&rrr!rrlrefsrrrrr"szConsoleStats._print_refsr)rarcCs|jr0|jdt|jdddt|jdfn,|jdt|jddd|jt|jdf|jrt|jt|j|j D]6\}}|jdt |t |j f| |j|j qz|jdk r|jd t |jfdS) zc Print the gathered information of object `tobj` in human-readable format. z%-32s ( free ) %-35s T)left#z%-32s 0x%08x %-35s z %-30s %s Nz %-30s finalize )rerArr rrfidr1r>r-rr r$rr&)rHrar.r$rrr print_object6s,   zConsoleStats.print_object?)clsnamelimitrcs|jr|j|js||j}r:fdd|D}|dkrZtdtt|j|}|dt|}|D]}||qndS)a Write tracked objects to stdout. The output can be filtered and pruned. Only objects are printed whose classname contain the substring supplied by the `clsname` argument. The output can be pruned by passing a `limit` value. :param clsname: Only print objects whose classname contain the given substring. :param limit: If `limit` is a float smaller than one, only the supplied percentage of the total tracked data is printed. If `limit` is bigger than one, this number of tracked objects are printed. Tracked objects are first filtered, and then pruned (if specified). csg|]}|jkr|qSr)r[).0torrr is z,ConsoleStats.print_stats..rrN)r?rTrFrsmaxrhlenr)rHrrZ_sortedrarrr print_statsQs  zConsoleStats.print_statsrXc Cs|j}|j}|d|jD]|}||}|dt|jddt|jddf|D]B}||}|dt|d|dt|d t|d |dfqTq|d d S) z< Print per-class summary for each snapshot. zP---- SUMMARY ------------------------------------------------------------------ z%-35s %11s %12s %12s %5s rr}Zaverager|z %-33s %11d %12s %12s %4d%% !rzr{zP------------------------------------------------------------------------------- N) rrArr-rwr descr asizeof_total)rH classlistfobjr`r~r[inforrr print_summaryvs,       zConsoleStats.print_summary)r2rrr)Nr)rrrrrrrhrMfloatrrrrrrrrrrs"    %c @seZdZdZdZdZdZdZdZdZ dZ d Z d.e e eeeeed dddZdZdZeed dddZdZdZdZd/eeeedddZd0eed dddZd1eeed d!d"Zd2eed#d$d%Zd3d&eed'd(d)Zd4eed d+d,d-Zd S)5rzH Output the `ClassTracker` statistics as HTML pages and graphs. aL z
Could not generate %s chart! Install Matplotlib to generate charts.
z z-%s%s z
z z %(name)s %(size)s %(pct)3.1f%%rrrN)rr&rr!rrrc Cst|}|jddd||dkr2|d|D]t}|j|kr6|jd||kr6t|tt|jdt |j|jd|d} ||j | |j ||j ||dd q6|dkr|d d S) rcSs|jSrrrrrrrrz'HtmlStats._print_refs..rnrz ry)r!rr$r|r"rN) r%rrrtrr$rr rMrr refrowrr&) rHrr&rr!rrrrdatarrrrs   zHtmlStats._print_refsz

%(cnt)d instances of %(cls)s were registered. The average size is %(avg)s, the minimal size is %(min)s, the maximum size is %(max)s.

zV

Snapshot: %(name)s, %(total)s occupied by instances of class %(cls)s

)r8r[rcCsnt|d}||j||jf|d|dd|j|D}d}|D] }||7}qJt|j||d}t|t||d<tt||d<tt||d <||j |||j ||d |j D]h}|j r||j kr|j |d } ||j |j|t| jd | jr.||| j| jq|d q|d|j|D]} |d|dd| j| jf| jr|dd| j|dt| jt| jf| jrdt| j} |d| | j D]j\} } |dt| | js|dt| jn0|dt| j||| j| j|dq|dqN||j|dS)z Print detailed statistics and instances for the class `classname`. All data will be written to the file `fname`. w

%s

cSsg|] }|qSr)r\)rrarrrrsz1HtmlStats.print_class_details..r)cntclsr{rminz*

Coalesced Referents per Snapshot

r+)rrrz'

No per-referent sizes recorded.

z

Instances

z*
z+z# zz z+ z 
%s
z+ zz z rN)rNrheaderstylerDrr rr class_summarychartsr-r~class_snapshotrr$r&rrrrfrrdrer1r>footerrQ)rHr8r[rsizesrsrr`r+rar1r.r$rrrprint_class_detailssn          zHtmlStats.print_class_detailsz z a4

Total virtual memory assigned to the program at that time was %(sys)s, which includes %(overhead)s profiling overhead. The ClassTracker tracked %(tracked)s in total. The measurable objects including code objects but excluding overhead have a total size of %(asizeof)s.

)filepathbasepathrcCsR|dkr|j}|s|S||r0|t|d}|rN|dtjkrN|dd}|S)z} Convert the filepath path to a relative path against basepath. By default basepath is self.basedir. Nrr)basedir startswithrossep)rHrrrrr relative_path0s  zHtmlStats.relative_pathr4)r@titlerc Cst|d}||j||jf|d||d||jd|d|dt|j}||j D]F}|d|d|d |j pd t |j fi}t |jj|d <t |j|d <t |j|d <t t|dd|d<||j||jr||j|D]l}|jr|j|}||j|} d| |f|d<t |d|d<t |d|d<||j|q|d|d|jr||j||dqx|d||j|dS)z( Output the title page. rrz'

Memory distribution over time

r-z

Snapshots statistics

z
Instance%s at 0x%08x
Representation%s 
Lifetime%s - %s
Instantiation%s
%s%s
%s
Class Instance # Total Average size Share
%(cls)s %(active)d %(sum)s %(avg)s %(pct)3.2f%%
z
z z

%s snapshot at %s

ZUntitledrBZtrackedZasizeofZoverheadrz%srrzr{z
z 		rrN)rNrrrrr%rDrZrrr-rrr.r Z system_totalZvszrYrrgsnapshot_summarysnapshot_cls_headerr~copyrlinks snapshot_clsrrQ) rHr@rrrr`rr[rpathrrrcreate_title_page@sP             zHtmlStats.create_title_page)r[r@rcCs"z$ddlm}m}m}m}m}m}Wn"tk rFtj |dYSXg} |j |D]*} | | j dg| j rV| | j dgqV| tdt| D]$} | | d| | dd7<qdd| D} dd| D} ||d |d |d ||| | d |||jtj|S) z Create chart that depicts the lifetime of the instance registered with `classname`. The output is written to `filename`. r)figurerxlabelylabelplotsavefigz lifetimerr#cSsg|] \}}|qSrrrtcrrrrsz3HtmlStats.create_lifetime_chart..cSsg|] \}}|qSrrrrrrrszExecution time [s]z Instance #z %s instanceso)pylabrrrrrr ImportErrorr nopylab_msgrDr(rdrerrranger chart_tagrrbasename)rHr[r@rrrrrrrrairyrrrcreate_lifetime_chartus*$"  zHtmlStats.create_lifetime_chart)r@rcsz8ddlm}m}m}m}m}m}m}m} ddl m } Wnt k rV|j dYSX|j } dd|jD} dgt|j} g}g}| D]fdd|jD}|rt|d krfd d|jD}d dt| |D}| | | |\}}|||fd if||} q||d |d|ddd|jD}|| |ddddd|jD}|| |ddd|D]\}}|||qt|dd| ||j||S)zw Create chart that depicts the memory allocation over time apportioned to the tracked classes. r)rrrrrfilllegendrNzmemory allocationcSsg|] }|jqSr)r.rr`rrrrsz3HtmlStats.create_snapshot_chart..gcs$g|]}|jdk r|jdqS)Nr|)r~rcnrrrs @cs,g|]$}|jdk rt|jddqS)Nrz)r~rrfprrrrs cSsg|]\}}||qSrr)rZsxZsyrrrrslabelzSnapshot MemoryzExecution Time [s]zVirtual Memory [MiB]cSsg|]}t|jdqSr)rrrrrrrszr--ZTotal)rcSsg|]}t|jdqSr)rrYrrrrrszb--z Tracked total)loc)rrrrrrrrrZmatplotlib.mlabmlabrrrr-rrzipZ poly_betweenr(rr)rHr@rrrrrrrrrrtimesrZ poly_labelsZpolysr|szZxpZyprrckwdsrrrcreate_snapshot_chartsL(    zHtmlStats.create_snapshot_chartr)r`r@rc Csz,ddlm}m}m}m}m}ddlm}Wntk rJ|jdYSX|j r\|j dkr`dSg} g} t |j D],\} } | ddkrv| | | | d qv| d|j|| | dd |d |j|d d |ddddg|| | d||dd|j||S)z Create a pie chart that depicts the distribution of the allocated memory for a given `snapshot`. The chart is saved to `filename`. r)rrpieaxesr)rzZ pie_chartNr4r|rrzZOtherz!Snapshot (%s) Memory Distribution)r)Zfigsizerg?)labelsr)Zdpi)rrrrrrrzrrrYr~r%itemsr(insertrrrr) rHr`r@rrrrrZ pylab_sumrZsizelistkvrrrcreate_pie_charts*      zHtmlStats.create_pie_chartClassTracker Statistics)r8rrcCsptjtj||_tj|dd|_tj|jsFt|jtj|j|_i|_ | i|_ tj |jd}| ||j d<t|jttt|jD].\}}tj |jd|}||||j |<qt|jD]2}tj |j|ddd}||||j |<qt|jD]8}tj |j|ddd }||j |<|||q$|j||d d S) zl Create HTML page `fname` and additional files in a directory derived from `fname`. r_filesz timespace.pngr-zfp%d.png.r<z-lt.pngz.html)rN)rrdirnameabspathrsplitextZfilesdirisdirmkdirrrxrr6rrr-r%rrrrDrZreplacerrr)rHr8rfnridxrrrr create_htmls* " zHtmlStats.create_html)rrr)N)r4)r4)r4)r4)r) rrrrrrrrZ tableheaderZ tablefooterrrrrrrhrrrrrMrrrrrrrrrrr rrrrrsD  @  52 )r)&rtypingrrrrrrrr r rrOrBrr Zpympler.util.stringutilsr r rZpympler.asizeofrZ classtrackerrrr__all__rMrrhr'rr0r>objectrrrrrrrs$,   Mx