U 祡cSW@sNdZddlmZmZmZmZmZmZmZddl m Z ddl m Z ddl mZmZddlmZmZddlmZmZddlmZdd lmZdd lmZdd lmZdd lZd gZ eZ!Gd dde"Z#e$dddZ%Gddde"Z&eee$e'fd dddZ(eee$e'fd dddZ)GdddeZ*Gddde"Z+Gdd d e"Z,d S)a The `ClassTracker` is a facility delivering insight into the memory distribution of a Python program. It can introspect memory consumption of certain classes and objects. Facilities are provided to track and size individual objects or all instances of certain classes. Tracked objects are sized recursively to provide an overview of memory distribution between the different tracked objects. )AnyCallableDictIOListOptionalTuple) defaultdict)partial)stackisclass)ThreadLock)sleeptimeref) ConsoleStats) safe_reprN ClassTrackerc@s@eZdZdZdZeeeeedddZ eeeeddddZ dS) _ClassObserverzy Stores options for tracked classes. The observer also keeps the original constructor of the observed class. initnamedetailkeeptracecCs"||_||_||_||_||_dSNr)selfrrrrrr8/tmp/pip-unpacked-wheel-8ad_c8mj/pympler/classtracker.py__init__'s z_ClassObserver.__init__N)rrrrreturncCs||_||_||_||_dSr)rrrr)rrrrrrrr modify/sz_ClassObserver.modify) __name__ __module__ __qualname____doc__ __slots__rstrintboolr!r#rrrr r s  rr"cCs ttS)z= Get a timestamp relative to the program start time. )r _local_startrrrr _get_time6sr.c@seZdZdZdZdeeeee e dddZ e d d d Z e dd d dZdd ddZeejddddZed ddZeedddZeddddZeddddZdS) TrackedObjectz Stores size and lifetime information of a tracked object. A weak reference is attached to monitor the object without preventing its deletion. ) ridreprrbirthdeathr snapshots_resolution_level__dict__rFN)instancerresolution_levelr on_deletecCs|t||j|_t||_d|_||_t|_d|_||_ d|_ |rJ| t |pVd}t ||}|j|fg|_||_dS)a Create a weak reference for 'instance' to observe an object but which won't prevent its deletion (which is monitored by the finalize callback). The size of the object is recorded in 'snapshots' as (timestamp, size) tuples. Nr) weakref_reffinalizerr0r1rr.r2r3r5r _save_traceasizeofZ basicsizeZAsizedr4r9)rr7rr8rr9 initial_sizesizerrr r!Es  zTrackedObject.__init__r,cCs:i}ttddD]$}t||r|dkrt||||<q|S)a7 Make the object serializable for dump_stats. Read the available slots and store the values in a dictionary. Derived values (stored in the dict) are not pickled as those can be reconstructed based on the other data. References cannot be serialized, ignore 'ref' as well. r(r)rr6)getattrr/hasattr)rstaterrrr __getstate__^s zTrackedObject.__getstate__)rCr"cCs&t|D]\}}t|||q dS)zf Restore the state from pickled data. Needed because a slotted class is used. N)listitemssetattr)rrCkeyvaluerrr __setstate__kszTrackedObject.__setstate__cCsBt}z2g|_|ddD]}|jd|ddqW5~XdS)z? Save current stack trace as formatted string. Nr)r rinsert)rZ stack_tracefrmrrr r=ss zTrackedObject._save_trace)tssizerr"cCs>|}|j||j||jdf|dk r:t|dd|_dS)z Store timestamp and current size for later evaluation. The 'sizer' is a stateful sizing facility that excludes other tracked objects. )rN)Zclip)rr4appendZasizedr5rr1)rrOrPobjrrr track_sizes zTrackedObject.track_sizecCstdd|jDS)z7 Get the maximum of all sampled sizes. cSsg|]\}}|jqSr)r@).0_srrr sz.TrackedObject.get_max_size..)maxr4rrrr get_max_sizeszTrackedObject.get_max_size) timestampr"cCs&d}|jD]\}}||kr |j}q |S)z Get the size of the object at a specific time (snapshot). If the object was not alive/sized at that instant, return 0. r)r4r@)rr\r@trWrrr get_size_at_times zTrackedObject.get_size_at_time)r8r"cCs ||_dS)z Set resolution level to a new value. The next size estimation will respect the new value. This is useful to set different levels for different instances of tracked classes. N)r5)rr8rrr set_resolution_levelsz"TrackedObject.set_resolution_level)rr"cCs4zt|_|jr|Wntk r.YnXdS)a7 Mark the reference as dead and remember the timestamp. It would be great if we could measure the pre-destruction size. Unfortunately, the object is gone by the time the weakref callback is called. However, weakref callbacks are useful to be informed when tracked objects died without the need of destructors. If the object is destroyed at the end of the program execution, it's not possible to import modules anymore. Hence, the finalize callback just does nothing (self.death stays None). N)r.r3r9 Exception)rrrrr r<s  zTrackedObject.finalize)rFN)r$r%r&r'r(rr)r*r+rrr!rrDrJr=floatr>AsizerrTr[r^r_r;r<rrrr r/=s"     r/) time_seriesr"cCs,|r|ddnd}|t|dfdS)NrLrrRr.rcZ num_instancesrrr track_object_creationsrgcCs$|dd}|t|dfdS)NrdrLrerfrrr track_object_deletions rhcs:eZdZdZdeeedfdd Zdddd ZZS) PeriodicThreadz7 Thread object to take snapshots periodically. r)trackerintervalargskwargscs(||_||_d|_tt|j||dS)zb Create thread with given interval and associated with the given tracker. FN)rkrjstopsuperrir!)rrjrkrlrm __class__rr r!szPeriodicThread.__init__Nr,cCs&d|_|js"|jt|jqdS)z2 Loop until a stop signal is set. FN)rnrjcreate_snapshotrrkrZrrr runs zPeriodicThread.run) r$r%r&r'rarr!rs __classcell__rrrpr ris  ric@sHeZdZdZd eeddddZeeddd Z eedd d Z dS) Snapshotz6Sample sizes of objects and the process at an instant.r:N)r\ descriptionr"cCs4d|_d|_d|_||_tj|_||_d|_ dS)z)Initialize process-wide size information.rN) tracked_total asizeof_totaloverheadr\pymplerprocessZProcessMemoryInfo system_totaldescclasses)rr\rvrrr r!s zSnapshot.__init__r,cCs&|jjr|jjS|jr|jS|jSdS)z Return the total (virtual) size of the process in bytes. If process information is not available, get the best number available, even if it is a poor approximation of reality. N)r| availableZvszrxrwrZrrr totals zSnapshot.totalcCs |jsd|jSd|j|jfS)z?Return timestamped label for this snapshot, or a raw timestamp.z%.3fsz %s (%.3fs))r}r\rZrrr labels zSnapshot.label)r:) r$r%r&r'rar)r!propertyr*rrrrrr rus   ruc@sleZdZd5eedddZeedddZe e e e ddd d Z e e eeeedd d d Ze edddZe eeeeddddZe ddddZd6e eddddZd7e eeeeeddddZd8e eeeeedddd Ze ddd!d"Zddd#d$Zddd%d&Zddd'd(Zddd)d*Zd9edd,d-d.Zddd/d0Ze Z!d:eedd2d3d4Z"dS);rN)streamcCs<tt|_i|_g|_tt|_g|_i|_d|_||_ dS)z Creates a new `ClassTracker` object. :param stream: Output stream to use when printing statistics via ``stats``. N) r rEindexobjectsr4history _keepalive _observers_periodic_thread_stream)rrrrr r!s  zClassTracker.__init__r,cCst||jdS)zu Return a ``ConsoleStats`` instance initialized with the current state of the class tracker. )rjr)rrrZrrr stats'szClassTracker.stats) _observer__self_rlkwdsr"cOs2|j||j|j|j|jd|j|f||dS)a Injected constructor for tracked classes. Call the actual constructor of the object and track the object. Attach to the object before calling the constructor to track the object with the parameters of the most specialized class. )rr8rrN) track_objectrrrrr)rrrrlrrrr _tracker/szClassTracker._tracker)clsfuncrr8rrr"c spz |j}Wn(tk r2tttdddd}YnXt||||||j|<ttddfdd }||_dS)z Modifying Methods in Place - after the recipe 15.7 in the Python Cookbook by Ken Seehof. The original constructors may be restored later. N)r_args_kwargsr"c_sdSrr)rrrrrr constructorIsz5ClassTracker._inject_constructor..constructor)rlrmr"csf||Srr)rlrmrZobserverrr new_constructorWsz9ClassTracker._inject_constructor..new_constructor)r!AttributeErrorrrr) rrrrr8rrrrrrr _inject_constructor>s  z ClassTracker._inject_constructor)rr"cCs ||jkS)z4 Determine if the class is tracked. )rrrrrr _is_tracked\szClassTracker._is_tracked)rrrrrr"cCs|j|||||dS)z4 Modify settings of a tracked class N)rr#)rrrrrrrrr _track_modifybszClassTracker._track_modifycCs|j|j|_|j|=dS)zH Restore the original constructor, lose track of class. N)rrr!rrrr _restore_constructorisz!ClassTracker._restore_constructorr)r7r8r"cCs|jt|}||dS)z Change tracking options for the already tracked object 'instance'. If instance is not tracked, a KeyError will be raised. N)rr0r_)rr7r8tobjrrr track_changepszClassTracker.track_changeF)r7rr8rrr"cCst||jkr(|jt|dk r(dS|r0|n|jj}t|j|tt|j|}t |||||d}|j | |||jt|<|r|j |dS)a Track object 'instance' and sample size and lifetime information. Not all objects can be tracked; trackable objects are class instances and other objects that can be weakly referenced. When an object cannot be tracked, a `TypeError` is raised. :param resolution_level: The recursion depth up to which referents are sized individually. Resolution level 0 (default) treats the object as an opaque entity, 1 sizes all direct referents individually, 2 also sizes the referents of the referents and so forth. :param keep: Prevent the object's deletion by keeping a (strong) reference to the object. N)r8rr9) r0rrrqr$rgrr rhr/rrRr)rr7rr8rrr9rrrr rxs"zClassTracker.track_object)rrr8rrr"cCs`t|std|dkr(|jd|j}||rF||||||n|||j||||dS)au Track all objects of the class `cls`. Objects of that type that already exist are *not* tracked. If `track_class` is called for a class already tracked, the tracking parameters are modified. Instantiation traces can be generated by setting `trace` to True. A constructor is injected to begin instance tracking on creation of the object. The constructor calls `track_object` internally. :param cls: class to be tracked, may be an old-style or a new-style class :param name: reference the class by a name, default is the concatenation of module and class name :param resolution_level: The recursion depth up to which referents are sized individually. Resolution level 0 (default) treats the object as an opaque entity, 1 sizes all direct referents individually, 2 also sizes the referents of the referents and so forth. :param keep: Prevent the object's deletion by keeping a (strong) reference to the object. :param trace: Save instantiation stack trace for each instance z!only class objects can be trackedN.)r TypeErrorr%r$rrrr)rrrr8rrrrr track_classs  zClassTracker.track_classcCs||dS)z Stop tracking class 'cls'. Any new objects of that type are not tracked anymore. Existing objects are still tracked. N)rrrrr detach_classszClassTracker.detach_classcCs&t|j}|D]}||qdS)z2 Detach from all tracked classes. N)rErkeysr)rr~rrrr detach_all_classesszClassTracker.detach_all_classescCs.||j|jg|jdd<dS)z Detach from all tracked classes and objects. Restore the original constructors and cleanse the tracking lists. N)rrclearrrrZrrr detach_alls  zClassTracker.detach_allcCs|g|jdd<dS)zV Clear all gathered data and detach from all tracked objects/classes. N)rr4rZrrr rszClassTracker.clearcCs |dS)am Detach from tracked classes by removing injected constructors. Makes it possible to use ClassTracker in `contextlib.closing` to safely remove profiling hooks when the tracker goes out of scope:: import contextlib with contextlib.closing(ClassTracker()) as tracker: tracker.track_class(Foo) N)rrZrrr closes zClassTracker.close?)rkr"cCs:|js.t||dd|_|jd|jn||j_dS)a? Start a thread which takes snapshots periodically. The `interval` specifies the time in seconds the thread waits between taking snapshots. The thread is started as a daemon allowing the program to exit. If periodic snapshots are already active, the interval is updated. ZBackgroundMonitor)rTN)rri setDaemonstartrk)rrkrrr start_periodic_snapshotss  z%ClassTracker.start_periodic_snapshotscCs,|jr(|jr(d|j_|jd|_dS)z Post a stop signal to the thread that takes the periodic snapshots. The function waits for the thread to terminate which can take some time depending on the configured interval. TN)ris_alivernjoinrZrrr stop_periodic_snapshotss z$ClassTracker.stop_periodic_snapshotsr:)rv compute_totalr"c Csz|jt}t}ddt|jD}|j |t|j}|j ddd|D]}| ||q^t |t |}|j|_|rtjddd|_d|_|jr|||_|jr|j|j8_|j|W5|jXd S) am Collect current per instance statistics and saves total amount of memory associated with the Python process. If `compute_total` is `True`, the total consumption of all objects known to *asizeof* is computed. The latter might be very slow if many objects are mapped into memory at the time the snapshot is taken. Therefore, `compute_total` is set to `False` by default. The overhead of the `ClassTracker` structure is also computed. Snapshots can be taken asynchronously. The function is protected with a lock to prevent race conditions. cSsg|] }|qSrr)rUrrrr rX2sz0ClassTracker.create_snapshot..cSs|jSr)r2)xrrr :z.ClassTracker.create_snapshot..)rHT)allcoderN) snapshot_lockreleaseacquirer.r>rbrErvaluesZ exclude_refssortrTrur)rrwrxryr4rR) rrvrr\rPobjsZtracked_objectsrZsnapshotrrr rrs*   zClassTracker.create_snapshot)N)r)NrFF)NrFF)r)r:F)#r$r%r&rrr!rrrrrrtyperr)r*r+rrrrrrrrrrrrrarrrrrrrrrr rsl%     ,  ! )-r'typingrrrrrrr collectionsr functoolsr inspectr r threadingr rrrweakrefrr;Zpympler.classtracker_statsrZpympler.util.stringutilsrZpympler.asizeofr>Zpympler.processrz__all__r-objectrrar.r/r*rgrhrirurrrrr s* $      |#