U d+i@sUdZddlZddlZddlZddlmZddlZddlZddlZddl m Z m Z m Z m Z mZddlmZmZddlmZdd lmZmZmZmZmZdd lmZmZmZdd lmZddl Zzdd l m!Z!Wne"k rdZ!YnXd a#e$Z%e&Z'ga(e)ej*dddZ+e ee,e-dfZ.GdddZ/e/a0e1ej*drNej*j2Z2nedZ2d Z3e4e5d<d Z6e4e5d<dZ7e ej*j8e5d<e4dddZ9ddZ:ddZ;dd Zd%d&Z?e?e<e?e=Gd'd(d(e@ZAd)d*ZBd+d,ZCd-d.ZDGd/d0d0eEZFGd1d2d2eGZHe-dd3d4d5ZIGd6d7d7eEZGd8d9d9eZJe.dd:d;d<ZKde e.e,d:d=d>ZLde e.e e-e-fd:d?d@ZMe.e2d:dAdBZNe.e.e4dCdDdEZOGdFdGdGeEZPe dHePdIdJdKZQedLdMdNZRe-ddOdPZSe e,ddQdRZTe,ddSdTZUe-ddUdVZVde.dd:dWdXZWdYdZZXde e.ed:d[d\ZYde e.ed:d]d^ZZd_d`Z[e e-e,fddadbdcZ\e-ddddeZ]de e ee-fe-d:dfdgZ^de e ee-fe-d:dhdiZ_ddjl`TddjlaTebdkdlZcGdmdndneEZdddolemfZfGdpdqdqefZgGdrdsdsegZhGdtduduegZiGdvdwdwegZjGdxdydyegZkGdzd{d{egZlGd|d}d}egZmGd~ddegZnGdddegZoGdddegZpGdddegZqGdddegZrGdddegZs[f[gejtueiejtuejejtuelejtuemejtuenejtueoejtuehejtuekejtuepejtueqejtuerejtuesddlmvZvddlmwZwddlmxZxddlmyZyddlmzZzdS)aM This package adds support for CUDA tensor types, that implement the same function as CPU tensors, but they utilize GPUs for computation. It is lazily initialized, so you can always import it, and use :func:`is_available()` to determine if your system supports CUDA. :ref:`cuda-semantics` has more details about working with CUDA. N)Device)ListOptionalTupleUnionAny)_get_device_index _dummy_type) classproperty) CUDAGraphgraph_pool_handlegraphmake_graphed_callablesis_current_stream_capturing)ExternalStreamStreamEventdevice)_cudartFZ_cuda_isInBadForkcCsdSNFrrr7/tmp/pip-unpacked-wheel-ua33x9lu/torch/cuda/__init__.py$rc@s2eZdZddZddZddZeddd Zd S) _LazySeedTrackercCsd|_d|_g|_dSNmanual_seed_all_cbmanual_seed_cb call_orderselfrrr__init__-sz_LazySeedTracker.__init__cCs||f|_|j|jg|_dSrrr$cb tracebackrrrqueue_seed_all2s z_LazySeedTracker.queue_seed_allcCs||f|_|j|jg|_dSr)r!r r"r&rrr queue_seed7s z_LazySeedTracker.queue_seedreturncCs|jSr)r"r#rrr get_calls<sz_LazySeedTracker.get_callsN)__name__ __module__ __qualname__r%r)r*rr-rrrrr(sr_CudaDeviceProperties has_magmahas_halfrdefault_generatorsr+cCsttjdsdStjdkS)z9Returns a bool indicating if CUDA is currently available._cuda_getDeviceCountFr)hasattrtorch_Cr5rrrr is_availableMs r9cCsHtjj}|dk r(t|dddk}nd}tjtjjdkoF|S)zLReturns a bool indicating if the current CUDA device supports dtype bfloat16N.r F)r7versioncudaintsplitget_device_propertiescurrent_devicemajor)Zcu_versZcuda_maj_deciderrris_bf16_supportedUs rDcCstj|dSr)r7r8Z _cuda_sleep)Zcyclesrrr_sleep_srEc Csd}d}tjjdk rtj}ttD]}t|}|d}|d}t|}|d|}t ddtj Ddd } || krt |||||| d| dfq(|d kr(|d kr(|d kr(t |||d |fq(dS)Nz Found GPU%d %s which requires CUDA_VERSION >= %d to work properly, but your PyTorch was compiled with CUDA_VERSION %d. Please install the correct PyTorch binary using instructions from https://pytorch.org z Found GPU%d %s which is of cuda capability %d.%d. PyTorch no longer supports this GPU because it is too old. The minimum cuda capability supported by this library is %d.%d. rr css |]}t|ddVqdS)_rNr?r@.0archrrr ysz$_check_capability..#)defaulti(#i') r7r=r>r8Z_cuda_getCompiledVersionrange device_countget_device_capabilityget_device_namemin get_arch_listwarningswarn) Zincorrect_binary_warnZ old_gpu_warnZ CUDA_VERSIONd capabilityrCminornameZ current_archZmin_archrrr_check_capabilitycs   $r]c sd}tjjdkrdSt}t|dkr*dSdd|D}ttD]Z}t|\}tfdd|D}|sBt |}d|}t | ||d ||qBdS)Na  {} with CUDA capability sm_{} is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities {}. If you want to use the {} GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/ rcSs&g|]}d|krt|ddqS)Zsm_rGrrHrIrrr sz!_check_cubins..csg|]}|dkqS)rFr)rJsmZ cap_majorrrr^srF )r7r=r>rVlenrQrRrSanyrTrWrXformatjoin)Zincompatible_device_warn arch_listZ supported_smidxZ cap_minor supportedZ device_namerZrr`r _check_cubinss    ricCs to t S)z:Returns whether PyTorch's CUDA state has been initialized.) _initialized_is_in_bad_forkrrrris_initializedsrlcKs`tr|nN|ddr,t|tn0|ddrJt|tnt|tfdS)NZseed_allFseed) rlget_lazy_seed_trackerr)r( format_stackr* _queued_callsappend)callablekwargsrrr _lazy_calls  ruc@s eZdZdS)DeferredCudaCallErrorN)r.r/r0rrrrrvsrvcCs tdS)aInitialize PyTorch's CUDA state. You may need to call this explicitly if you are interacting with PyTorch via its C API, as Python bindings for CUDA functionality will not be available until this initialization takes place. Ordinary users should not need this, as all of PyTorch's CUDA methods automatically initialize CUDA state on-demand. Does nothing if the CUDA state is already initialized. N) _lazy_initrrrrinits rxc Cs tsttdrdSttr.W5QRdStrrBr set_devicejit is_scriptingrwr#rrr __enter__s    zdevice.__enter__typevaluer(cCs|j|jkrtj|jdSr)rrgr7r>r)r$rrr(rrr__exit__s zdevice.__exit__N)r.r/r0__doc__rr%rrrrrrrs rcs eZdZdZfddZZS) device_ofaContext-manager that changes the current device to that of given object. You can use both tensors and storages as arguments. If a given object is not allocated on a GPU, this is a no-op. Args: obj (Tensor or Storage): object allocated on the selected device. cs&|jr|nd}tt||dSr)is_cuda get_devicerrr%)r$objrgrrrr%)szdevice_of.__init__)r.r/r0rr%rrrrrrs r)rr,cCs t|}|dkrtj|dS)a>Sets the current device. Usage of this function is discouraged in favor of :any:`device`. In most cases it's better to use ``CUDA_VISIBLE_DEVICES`` environmental variable. Args: device (torch.device or int): selected device. This function is a no-op if this argument is negative. rN)r r7r8Z_cuda_setDevicerrrrr.s rcCs t|jS)aGets the name of a device. Args: device (torch.device or int, optional): device for which to return the name. This function is a no-op if this argument is a negative integer. It uses the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). Returns: str: the name of the device )rAr\rrrrrT=s rTcCst|}|j|jfS)aGets the cuda capability of a device. Args: device (torch.device or int, optional): device for which to return the device capability. This function is a no-op if this argument is a negative integer. It uses the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). Returns: tuple(int, int): the major and minor cuda capability of the device )rArCr[)rproprrrrSLs rScCs4tt|dd}|dks$|tkr,tdt|S)zGets the properties of a device. Args: device (torch.device or int or str): device for which to return the properties of the device. Returns: _CudaDeviceProperties: the properties of the device TrrInvalid device id)rwr rRr}Z_get_device_propertiesrrrrrA]s  rA)r peer_devicer,cCs\tt|dd}t|}|dks,|tkr4td|dksF|tkrNtdtj||S)z;Checks if peer access between two devices is possible. TrrrzInvalid peer device id)rwr rRr}r7r8Z_cuda_canDeviceAccessPeer)rrrrrcan_device_access_peerms rc@sJeZdZUdZeded<eddddZddZeeed d d Z d S) StreamContextaContext-manager that selects a given stream. All CUDA kernels queued within its context will be enqueued on a selected stream. Args: Stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: Streams are per-device. torch.cuda.Stream cur_streamstreamcCsh||_tdd|_tjs,|jdkr,d|_tjs:dn tjd|_tjsVdn tjd|_ dS)NTr) rr rgr7rrr>default_streamsrc_prev_streamdst_prev_stream)r$rrrrr%s   zStreamContext.__init__c Csn|j}|dks|jdkrdStjd|_|jj|jkr^t|jtj|j|_W5QRXtj|dSr) rrgr7r>current_streamrrr set_stream)r$rrrrrs zStreamContext.__enter__rcCsJ|j}|dks|jdkrdS|jj|jkr8tj|jtj|jdSr)rrgrrr7r>rr)r$rrr(rrrrrs zStreamContext.__exit__N) r.r/r0rrrr%rrrrrrrrzs   rr)rr,cCst|S)aTWrapper around the Context-manager StreamContext that selects a given stream. Arguments: stream (Stream): selected stream. This manager is a no-op if it's ``None``. ..Note:: In eager mode stream is of type Stream class while in JIT it is an object of the custom class ``torch.classes.cuda.Stream``. )rrrrrrs rrcCs|dkr dStj|jdS)aSets the current stream.This is a wrapper API to set the stream. Usage of this function is discouraged in favor of the ``stream`` context manager. Args: stream (Stream): selected stream. This function is a no-op if this argument is ``None``. N)r7r8Z_cuda_setStream_cdatarrrrrs rcCstrtjSdSdS)z%Returns the number of GPUs available.rN)r9r7r8r5rrrrrRs rRcCs(ts gStj}|dkr gS|S)z>Returns list CUDA architectures this library was compiled for.N)r9r7r8Z_cuda_getArchFlagsr@)Z arch_flagsrrrrVs  rVcCs8t}t|dkrdSdd|D}ddd|DS)z:Returns NVCC gencode flags this library was compiled with.rcSsg|]}|dqS)rG)r@rIrrrr^sz%get_gencode_flags..racSs&g|]\}}d|d|d|qS)z-gencode compute=compute_z,code=rGr)rJkindrKrrrr^s)rVrbre)rfZ arch_list_rrrget_gencode_flagss  rcCsttjS)z1Returns the index of a currently selected device.)rwr7r8Z_cuda_getDevicerrrrrBsrBc Cs4ttj|tjW5QRSQRXdS)a-Waits for all kernels in all streams on a CUDA device to complete. Args: device (torch.device or int, optional): device for which to synchronize. It uses the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). N)rwr7r>rr8Z_cuda_synchronizerrrr synchronizesrcCsttjS)axForce collects GPU memory after it has been released by CUDA IPC. .. note:: Checks if any sent CUDA tensors could be cleaned from the memory. Force closes shared memory file used for reference counting if there is no active counters. Useful when the producer process stopped actively sending tensors and want to release unused memory. )rwr7r8Z_cuda_ipc_collectrrrr ipc_collects rcCs tttjt|dddS)aTReturns the currently selected :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the currently selected :class:`Stream` for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). Trr)rwrr7r8Z_cuda_getCurrentStreamr rrrrrs  rcCs tttjt|dddS)a>Returns the default :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the default :class:`Stream` for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). Trr)rwrr7r8Z_cuda_getDefaultStreamr rrrrrs  rcCsttjS)z7Returns cublasHandle_t pointer to current cuBLAS handle)rwr7r8Z_cuda_getCurrentBlasHandlerrrrcurrent_blas_handlesr) debug_moder,cCsRtt|trB|dkrd}n$|dkr,d}n|dkr:d}ntdtj|dS) aSets the debug mode for cuda synchronizing operations. Args: debug_mode(str or int): if "default" or 0, don't error or warn on synchronizing operations, if "warn" or 1, warn on synchronizing operations, if "error" or 2, error out synchronizing operations. Warning: This is an experimental feature, and not all synchronizing operations will trigger warning or error. In particular, operations in torch.distributed and torch.sparse namespaces are not covered yet. rNrrXrerrorr zGinvalid value of debug_mode, expected one of `default`, `warn`, `error`N)rw isinstancerr|r7r8Z_cuda_set_sync_debug_mode)rrrrset_sync_debug_mode!s  rcCsttjS)zFReturns current value of debug mode for cuda synchronizing operations.)rwr7r8Z_cuda_get_sync_debug_moderrrrget_sync_debug_mode:srcCsz ddl}Wntk r(tdYnXddlm}z |Wn|k r^tdYnXt|dd}||}||jS)aReturns the percent of time over the past sample period during which global (device) memory was being read or written. as given by `nvidia-smi`. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). Warning: Each sample period may be between 1 second and 1/6 second, depending on the product being queried. rN.pynvml module not found, please install pynvmlNVMLError_DriverNotLoaded-cuda driver can't be loaded, is cuda enabled?Tr) pynvmlModuleNotFoundErrorrnvmlInitr|r nvmlDeviceGetHandleByIndexnvmlDeviceGetUtilizationRatesmemoryrrrhandlerrr memory_usageAs      rcCsz ddl}Wntk r(tdYnXddlm}z |Wn|k r^tdYnXt|dd}||}||jS)aReturns the percent of time over the past sample period during which one or more kernels was executing on the GPU as given by `nvidia-smi`. Args: device (torch.device or int, optional): selected device. Returns statistic for the current device, given by :func:`~torch.cuda.current_device`, if :attr:`device` is ``None`` (default). Warning: Each sample period may be between 1 second and 1/6 second, depending on the product being queried. rNrrrTr) rrrrr|r rrZgpurrrr utilization[s      r)*cOsttt|j|f||Sr)rwr _CudaBase__new__clsargsrtrrr _lazy_new~srcs(eZdZdZdZfddZeZZS)rTFc s6t| tt|j||W5QRSQRXdSr)rrrrr)r$rrtrrrrsz_CudaBase.type) r.r/r0rZ is_sparserrrrrrrrrs r)_LegacyStoragec@s8eZdZeddZeddZedddddZdS) _CudaLegacyStoragecOs tddS)Nz+from_buffer: Not available for CUDA storager|rrrr from_buffersz_CudaLegacyStorage.from_buffercOs tddS)Nz2_new_with_weak_ptr: Not available for CUDA storagerrrrr_new_with_weak_ptrsz%_CudaLegacyStorage._new_with_weak_ptrN)rdtypecCs tddS)Nz4_new_shared_filename: Not available for CUDA storager)rmanagerrsizerrrrr_new_shared_filenamesz'_CudaLegacyStorage._new_shared_filename)r.r/r0 classmethodrrrrrrrrs   rc@seZdZeddZdS) ByteStoragecCstjSr)r7Zuint8r#rrrrszByteStorage.dtypeNr.r/r0r rrrrrrsrc@seZdZeddZdS) DoubleStoragecCstjSr)r7doubler#rrrrszDoubleStorage.dtypeNrrrrrrsrc@seZdZeddZdS) FloatStoragecCstjSr)r7floatr#rrrrszFloatStorage.dtypeNrrrrrrsrc@seZdZeddZdS) HalfStoragecCstjSr)r7Zhalfr#rrrrszHalfStorage.dtypeNrrrrrrsrc@seZdZeddZdS) LongStoragecCstjSr)r7longr#rrrrszLongStorage.dtypeNrrrrrrsrc@seZdZeddZdS) IntStoragecCstjSr)r7r?r#rrrrszIntStorage.dtypeNrrrrrrsrc@seZdZeddZdS) ShortStoragecCstjSr)r7shortr#rrrrszShortStorage.dtypeNrrrrrrsrc@seZdZeddZdS) CharStoragecCstjSr)r7Zint8r#rrrrszCharStorage.dtypeNrrrrrrsrc@seZdZeddZdS) BoolStoragecCstjSr)r7boolr#rrrrszBoolStorage.dtypeNrrrrrrsrc@seZdZeddZdS)BFloat16StoragecCstjSr)r7Zbfloat16r#rrrrszBFloat16Storage.dtypeNrrrrrrsrc@seZdZeddZdS)ComplexDoubleStoragecCstjSr)r7Zcdoubler#rrrrszComplexDoubleStorage.dtypeNrrrrrrsrc@seZdZeddZdS)ComplexFloatStoragecCstjSr)r7Zcfloatr#rrrrszComplexFloatStorage.dtypeNrrrrrrsr)sparse)profiler)nvtx)amp) jiterator)N)N)N)N)N)N)N){r contextlibosr7Z torch.typesrr(rW threadingtypingrrrrr_utilsr r r Zgraphsr rrrrZstreamsrrrrrZ_deviceZtorch._Cr ImportErrorrjlocalrzLockr{rqgetattrr8rkrr?Z _device_trror6r1r2rrr3r4 Generatorr9rDrEr]rirlrurrvrxrwrobjectrr|rrrrrTrSrArrrrrRrVrrBrrrrrrrrrrrandom staticmethodrrZ torch.storagerrrrrrrrrrrrrrZ_storage_classesaddrrrrrrrrrs          1 3