U *-e@sddlZddlZddlZzddlZdZWnek r@dZYnXddlmZdddgZ Gdddej j j Z dd Z d dd d dZd dZdS)NT)Anyautocast custom_fwd custom_bwdcsdeZdZdZdejdfeejedfdd ZfddZ e e e dfd d Z fd d Z Z S) rz See :class:`torch.autocast`. ``torch.cuda.amp.autocast(args...)`` is equivalent to ``torch.autocast("cuda", args...)`` Tenableddtype cache_enabledcs8tjr ||_d|_||_dStjd|||ddS)Ncudar)torch _jit_internal is_scriptingZ_enabledZdeviceZ fast_dtypesuper__init__)selfrrr  __class__]/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/torch/cuda/amp/autocast_mode.pyrs zautocast.__init__cstjr|StSN)r r r r __enter__)rrrrr&s zautocast.__enter__)exc_typeexc_valexc_tbcstjrdSt|||Sr)r r r r__exit__)rrrrrrrr,s zautocast.__exit__cstjr|St|Sr)r r r r__call__)rfuncrrrr1s zautocast.__call__)__name__ __module__ __qualname____doc__r Zfloat16boolrrrrrr __classcell__rrrrrs cst|tjr8|o$|jo$|jtjk }|r4|S|St|tt frJ|St r^t|t j r^|St|t jjrfdd|DSt|t jjr‡fdd|D}t|ttfrt||S|Sn|SdS)Ncs"i|]\}}t|t|qSr_cast).0kvrrr Fsz_cast..c3s|]}t|VqdSrr#)r%r'r(rr Hsz_cast..) isinstancer ZTensorZis_floating_pointZis_cudarZfloat64tostrbytes HAS_NUMPYnpZndarray collectionsabcMappingitemsIterablelisttupletype)valuerZ is_eligibleiterablerr(rr$9s&   r$ cast_inputscs2dkrtjtdStfdd}|S)a> Helper decorator for ``forward`` methods of custom autograd functions (subclasses of :class:`torch.autograd.Function`). See the :ref:`example page` for more detail. Args: cast_inputs (:class:`torch.dtype` or None, optional, default=None): If not ``None``, when ``forward`` runs in an autocast-enabled region, casts incoming floating-point CUDA Tensors to the target dtype (non-floating-point Tensors are not affected), then executes ``forward`` with autocast disabled. If ``None``, ``forward``'s internal ops execute with the current autocast state. .. note:: If the decorated ``forward`` is called outside an autocast-enabled region, :func:`custom_fwd` is a no-op and ``cast_inputs`` has no effect. Nr;c st|d_dkr.t|d_||St}d|d_|rztdd$t|t|W5QRSQRXn ||SdS)NrF)r)r Zget_autocast_gpu_dtype_dtypeZis_autocast_enabled_fwd_used_autocastrr$)argskwargsZautocast_contextr<fwdrr decorate_fwdls   *z custom_fwd..decorate_fwd) functoolspartialrwraps)rBr<rCrrArrYs cstfdd}|S)a Helper decorator for backward methods of custom autograd functions (subclasses of :class:`torch.autograd.Function`). Ensures that ``backward`` executes with the same autocast state as ``forward``. See the :ref:`example page` for more detail. c s:t|dj|djd||W5QRSQRXdS)Nr)rr)rr>r=)r?r@bwdrr decorate_bwdsz custom_bwd..decorate_bwd)rDrF)rHrIrrGrrs)N)r1rDr numpyr0r/ModuleNotFoundErrortypingr__all__ampZ autocast_moderr$rrrrrrs   ( (