U d@sddlZddlZddlZzddlZdZWnek r@dZYnXddlmZddl m Z Gdddej j j Z ddZd d d Zd d ZdS)NT)string_classes)AnycsdeZdZdZdejdfeejedfdd ZfddZ e e e dfd d Z fd d Z Z S) autocastz See :class:`torch.autocast`. ``torch.cuda.amp.autocast(args...)`` is equivalent to ``torch.autocast("cuda", args...)`` Tenableddtype cache_enabledcs8tjr ||_d|_||_dStjd|||ddS)NZcudar)torch _jit_internal is_scriptingZ_enabledZdeviceZ fast_dtypesuper__init__)selfrrr __class__@/tmp/pip-unpacked-wheel-ua33x9lu/torch/cuda/amp/autocast_mode.pyr s  zautocast.__init__cstjr|StSN)r r r r __enter__)rrrrrs zautocast.__enter__)exc_typeexc_valexc_tbcstjrdSt|||Sr)r r r r __exit__)rrrrrrrr!s zautocast.__exit__cstjr|St|Sr)r r r r __call__)rfuncrrrr&s zautocast.__call__)__name__ __module__ __qualname____doc__r Zfloat16boolrr rrrr __classcell__rrrrr s " rcst|tjr8|o$|jo$|jtjk }|r4|S|St|trF|St rZt|t j rZ|St|t j jr~fdd|DSt|t j jrtfdd|}t|tst|trt||S|Sn|SdS)Ncs"i|]\}}t|t|qSr_cast).0kvrrr 7sz_cast..cs t|Srr!)r%r&rr9z_cast..) isinstancer ZTensorZis_floating_pointZis_cudarZfloat64tor HAS_NUMPYnpZndarray collectionsabcMappingitemsIterablemaplisttupletype)valuerZ is_eligibleiterablerr&rr".s   r"c sdkr@t|dkrdnt|dks*t|dtjtdSt|dkrRdnt|dksbt|dtfdd}|S)a> Helper decorator for ``forward`` methods of custom autograd functions (subclasses of :class:`torch.autograd.Function`). See the :ref:`example page` for more detail. Args: cast_inputs (:class:`torch.dtype` or None, optional, default=None): If not ``None``, when ``forward`` runs in an autocast-enabled region, casts incoming floating-point CUDA Tensors to the target dtype (non-floating-point Tensors are not affected), then executes ``forward`` with autocast disabled. If ``None``, ``forward``'s internal ops execute with the current autocast state. .. note:: If the decorated ``forward`` is called outside an autocast-enabled region, :func:`custom_fwd` is a no-op and ``cast_inputs`` has no effect. Nr cast_inputs)r:c szdkr t|d_||St}d|d_|rltdd$t|t|W5QRSQRXn ||SdS)NrF)r)r Zis_autocast_enabled_fwd_used_autocastrr")argskwargsZautocast_contextr:fwdrr decorate_fwdjs   *z custom_fwd..decorate_fwd)lenAssertionError functoolspartial custom_fwdwraps)r?r=r@rr>rrELs   rEcstfdd}|S)a Helper decorator for backward methods of custom autograd functions (subclasses of :class:`torch.autograd.Function`). Ensures that ``backward`` executes with the same autocast state as ``forward``. See the :ref:`example page` for more detail. c s0t|dj||W5QRSQRXdS)Nr)rr;)r<r=bwdrr decorate_bwdsz custom_bwd..decorate_bwd)rCrF)rHrIrrGr custom_bwd}srJ)N)r rCr.Znumpyr-r,ModuleNotFoundErrorZ torch._sixrtypingrampZ autocast_moderr"rErJrrrrs   ! 1