U 9%e>@s*UdZddlZddlZddlZddlZddlZddlZddlZddZddl m Z m Z ddl m Z mZmZmZerzdZn dd lmZdd lmZmZmZmZmZmZmZmZmZmZddlZd d d ddddddddddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?d@dAdBdCdDdEdFdGdHg>ZejdIkre dJdKZ!ej"#ej$dLdMZ%ej"#ej"&e'dNZ(ej$ej)krej"#ej)dLdMZ*ndOZ*e+e,ej"j-e(e%e*gZ.e/dPdQe.Drej"#e dRej"#e!dSdTdMdUZ0ndOZ0ddVl1m2Z3ddl4Z4e3rje/dWdQe.Drje35dXdYZ6dZe6Z7ej"#e!d[d\d]e3Z8ej"#e e7e8dMZ9ndOZ9e.:e,ej"j-e0e9gej;d^d_d`ZerejAered>Z_Gdwd?d?Z`Gdxd@d@ZadydAZbdzdCZcd{dBZdd|dDZed}dEZfzdd~l]mgZgWnDehk rRddl]m^Zieij'dkrLehejdkdYnXele^D]hZmemddYkr\emnds\eoemepe^emZqereqeseseqr\eqjtdkr\emdkr\deq_tq\esele^D]DZuepe^euZvewevewe^krdeuejxkrevejxdeu<qdd Zydd Zzdd Z{da|ddZ}ddZ~ddZdgdejejdddd5Zejdpdd6Zejdpdd7Zeejefdddd8Zejdpdd9Zejdpdd;Zedddd:Zejdddd<Zejdpdd=ZeejeafegefdddZdddZd ddZd!ddZd"ddZd#ddZd$ddZd%ddZddlmZmZmZmZe:ddddgddlmZddlmZmZmZmZmZGdd'd'eZGdd!d!eZGdd"d"eZGdddeZGdd#d#eZGdd$d$eZGdd%d%eZGdd&d&eZGdd(d(eZGdddeZGdddeZGdddeZGdd„deZGddĄdeZGddƄdeZGddȄdeZGddʄdeZeeeeeeeeeeeeeeeeeeehZeZeeed<ddlmZmZmZmZmZddlmZmZddlmZddЄZddlmZeZeZe^ge[e rddulTeZ[dZele^jʃD]nZmemdӡ semek r qepe^jemZqdeq_temdk reqẽem<dYemZmeqẽem<emdY seoem qddlmZddulT[[ddׄZddVlm2Z2ddlmZddlmZddlmZddlmZmZmZmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlZddlmZddlmZddlmZddlZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlZddlmZddlmZddlmZddlmZddlZddlZddlZddlZe^e+ejddlmZmZmZ[[[ejdpddZddlmZddlmZddlѐmZddlѐmZeZddlmZeeѐj[ddlmZejj j Z ejj j Z ddl m Z mZddlmZddlmZmZmZmZddlmZGdddZGdddZd&dgdddddgdeeejeejeeefeedfeeeeeejejffejedddFZddlѐmZddZdd lmZdd lmZe sdd lmZd ejTk rddlm2m Z!e!"ddl#Zdd lѐm$Z$ddl%m&Z&ddZ'ejjӐ(esrddlѐm)Z)GdddZ*ejjӐj+ejj2j+ejj,j-ejj.j-dZ/erddlѐm0Z0ddlѐm1Z1ddlѐm2Z2ddddhZ3ddZ4ddlm5Z5e56dS('a The torch package contains data structures for multi-dimensional tensors and defines mathematical operations over these tensors. Additionally, it provides many utilities for efficient serialization of Tensors and arbitrary types, and other useful utilities. It has a CUDA counterpart, that enables you to run your tensor computations on an NVIDIA GPU with compute capability >= 3.0. NcCstjddtkS)Nztorch._meta_registrations)sysmodulesgetobjectrrM/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/torch/__init__.py_running_with_deploysr)_import_dotted_name classproperty) get_file_path#prepare_multiprocessing_environmentUSE_RTLD_GLOBAL_WITH_LIBTORCHUSE_GLOBAL_DEPSztorch-deploy-1.8) __version__) AnyCallableDictOptionalSetTupleType TYPE_CHECKINGUnionListtypename is_tensor is_storageset_default_tensor_typeset_default_device set_rng_state get_rng_state manual_seed initial_seedseedsaveloadset_printoptionschunksplitstackmatmulno_grad enable_gradZrandZrandninference_mode DoubleStorage FloatStorage LongStorage IntStorage ShortStorage CharStorage ByteStorage BoolStorage TypedStorageUntypedStorageZ DoubleTensorZ FloatTensorZ LongTensorZ IntTensorZ ShortTensorZ CharTensorZ ByteTensorZ BoolTensorTensorlobpcguse_deterministic_algorithms$are_deterministic_algorithms_enabled-is_deterministic_algorithms_warn_only_enabledset_deterministic_debug_modeget_deterministic_debug_modeset_float32_matmul_precisionget_float32_matmul_precisionset_warn_alwaysis_warn_always_enabledSymIntSymFloatSymBoolsym_notsym_int sym_floatsym_maxsym_mincompilevmapexportwin32Z ProgramFileszC:\Program FilesLibrarybinlibccs&|]}tjtj|d VqdS)znvToolsExt64_1.dllN)ospathexistsjoin.0prrr Qsr[ZNVTOOLSEXT_PATHzNVIDIA CorporationZ NvToolsExtx64)cudaccs$|]}ttj|d VqdS)z cudart64*.dllN)globrTrUrWrXrrrr[Ys._Z CUDA_PATH_VzNVIDIA GPU Computing ToolkitCUDAvz kernel32.dllT)Zuse_last_errorZAddDllDirectoryzvcruntime140.dllz msvcp140.dllzvcruntime140_1.dllzMicrosoft Visual C++ Redistributable is not installed, this may lead to the DLL load failure. It can be downloaded at https://aka.ms/vs/16/release/vc_redist.x64.exez*.dllFi~z Error loading "z" or one of its dependencies.;PATHc Cstdkstdddl}d}tjD]P}tj|d}tj|sFq&|tj||d|}|rn|sn|d}|r&qxq&|st |dtjt |dS)z8Preloads cuda deps if they could not be found otherwise.LinuxzShould only be called on LinuxrNZnvidiarRz not found in the system path ) platformsystemAssertionErrorr^rrUrTrWrV ValueErrorctypesCDLL) lib_folderlib_namer^lib_pathrUZ nvidia_pathZcandidate_lib_pathsrrr_preload_cuda_depss  rpreturncststdkrdSdtdkr(dnd}tjt}tjtj|d|}zt j |t j dWnt k rzjdd d d d d dddddd }fdd| D}|s|D]\}}t||qt j |t j dW5dXYnXdS)NWindowsZlibtorch_global_depsDarwinz.dylibz.sorRmodezlibcublas.so.*[0-9]zlibcudnn.so.*[0-9]zlibnvrtc.so.*[0-9]zlibcudart.so.*[0-9]zlibcupti.so.*[0-9]zlibcufft.so.*[0-9]zlibcurand.so.*[0-9]zlibcusolver.so.*[0-9]zlibcusparse.so.*[0-9]zlibnccl.so.*[0-9]zlibnvToolsExt.so.*[0-9]) ZcublascudnnZ cuda_nvrtcZ cuda_runtimeZ cuda_cuptiZcufftZcurandZcusolverZcusparseZncclZnvtxcs(g|] }|ddjdkr|qS)r_r)r)args)rYrRerrrr sz%_load_global_deps..)rrgrhrTrUabspath__file__rWdirnamerkrl RTLD_GLOBALOSErrorvaluesitemsrp)rnhereroZ cuda_libsZis_cuda_lib_errrmrryr_load_global_depss4   rZTORCH_USE_RTLD_GLOBALrs)*c@seZdZdZddZddZddZdd Zee j d d d Z e j d ddZ e j d ddZ e j d ddZe j d ddZddZddZddZddZdS)rDz Like an int (including magic methods), but redirects all operations on the wrapped node. This is used in particular to symbolically record operations in the symbolic shape workflow. cCs ||_dSNnodeselfrrrr__init__szSymInt.__init__cCst|dkS)Nr)builtinsboolrrrr__bool__szSymInt.__bool__cCs |jSrrint_rrrr__int__szSymInt.__int__cCs |jSrrrrrr __index__szSymInt.__index__otherrrcCs tddSNztype stub not overriddenrirrrrr__eq__ sz SymInt.__eq__rqcCs tddSrrrrrr__lt__ sz SymInt.__lt__cCs tddSrrrrrr__gt__sz SymInt.__gt__cCs tddSrrrrrr__le__sz SymInt.__le__cCs tddSrrrrrr__ge__sz SymInt.__ge__cCs tddSrrrrrr __sym_max__szSymInt.__sym_max__cCs tddSrrrrrr __sym_min__szSymInt.__sym_min__cCs tddSrrrrrr __sym_float__szSymInt.__sym_float__cCs t|jSr)strrrrrr__repr__!szSymInt.__repr__N)__name__ __module__ __qualname____doc__rrrrrrrrrrrrrrrrrrrrrDsc@seZdZdZddZddZeejdddZ ejd d d Z ejd d d Z ejd ddZ ejd ddZ ddZddZddZddZdS)rEz Like an float (including magic methods), but redirects all operations on the wrapped node. This is used in particular to symbolically record operations in the symbolic shape workflow. cCs ||_dSrrrrrrr+szSymFloat.__init__cCs |jSrrbool_rrrrr0szSymFloat.__bool__rcCs tddSrrrrrrr5szSymFloat.__eq__rqcCs tddSrrrrrrr8szSymFloat.__lt__cCs tddSrrrrrrr;szSymFloat.__gt__cCs tddSrrrrrrr>szSymFloat.__le__cCs tddSrrrrrrrAszSymFloat.__ge__cCs tddSrrrrrrrDszSymFloat.__sym_max__cCs tddSrrrrrrrGszSymFloat.__sym_min__cCs tddSrrrrrr __sym_int__JszSymFloat.__sym_int__cCs |jSrrrrrrrrMszSymFloat.__repr__N)rrrrrrrrrrrrrrrrrrrrrrrE$sc@sZeZdZdZddZddZddZddd d Zddd d Zddd dZ ddZ dS)rFan Like an bool (including magic methods), but redirects all operations on the wrapped node. This is used in particular to symbolically record operations in the symbolic shape workflow. Unlike regular bools, regular boolean operators will force extra guards instead of symbolically evaluate. Use the bitwise operators instead to handle this. cCs ||_dSrrrrrrrZszSymBool.__init__cCs |jSrrrrrrr_szSymBool.__bool__cCst|jSr)rintrrrrrrrbszSymBool.__int__rqcCs tddSrrrrrr__and__fszSymBool.__and__cCs tddSrrrrrr__or__iszSymBool.__or__cCs tddSrrrrrr __sym_not__}szSymBool.__sym_not__cCs |jSrrrrrrrszSymBool.__repr__N) rrrrrrrrrrrrrrrrFPs cCst|dr|S| S)zi SymInt-aware utility for logical negation. Args: a (SymBool or bool): Object to negate r)hasattrrarrrrGs cCs(t|tr|St|dr |St|S)zp SymInt-aware utility for float casting. Args: a (SymInt, SymFloat, or object): Object to cast r) isinstancerErrpy_floatrrrrrIs   cCs<t|tr|St|tr4|dkr*t|St|St|S)zn SymInt-aware utility for int casting. Args: a (SymInt, SymFloat, or object): Object to cast r)rrDrEmathfloorceilpy_intrrrrrHs   cCs<t|ttfr||St|ttfr0||St||Sz SymInt-aware utility for max().)rrDrErrmaxrbrrrrJs   cCs<t|ttfr||St|ttfr0||St||Sr)rrDrErrminrrrrrKs   )_initExtensiona Failed to load PyTorch C extensions: It appears that PyTorch has loaded the `torch/_C` folder of the PyTorch repository rather than the C extensions which are expected in the `torch._C` namespace. This can occur when using the `install` workflow. e.g. $ python setup.py install && python -c "import torch" This error can generally be solved using the `develop` workflow $ python setup.py develop && python -c "import torch" # This should succeed or by running Python from a different directory. ZBasetorch)ZDisableTorchFunctionSubclassZDisableTorchFunction Generatorz torch._C.cCst|tjr|Sd}d}t|drN|jdkrN|jdkrN|jdk rN|jd}t|dr`|j}nt|drr|j}n|jj}||S)NrSrr __builtin__r_rr) rrr9typerrrr __class__)omodule class_namerrrrs     cCs t|tjS)aReturns True if `obj` is a PyTorch tensor. Note that this function is simply doing ``isinstance(obj, Tensor)``. Using that ``isinstance`` check is better for typechecking with mypy, and more explicit - so it's recommended to use that instead of ``is_tensor``. Args: obj (Object): Object to test Example:: >>> x = torch.tensor([1, 2, 3]) >>> torch.is_tensor(x) True )rrr9objrrrrscCs t|tkS)zgReturns True if `obj` is a PyTorch storage object. Args: obj (Object): Object to test )r_storage_classesrrrrrscCsFtdk rtddd|dkr&dadSddlm}||atdS)aSets the default ``torch.Tensor`` to be allocated on ``device``. This does not affect factory function calls which are called with an explicit ``device`` argument. Factory calls will be performed as if they were passed ``device`` as an argument. To only temporarily change the default device instead of setting it globally, use ``with torch.device(device):`` instead. The default device is initially ``cpu``. If you set the default tensor device to another device (e.g., ``cuda``) without a device index, tensors will be allocated on whatever the current device for the device type, even after :func:`torch.cuda.set_device` is called. .. warning:: This function imposes a slight performance cost on every Python call to the torch API (not just factory functions). If this is causing problems for you, please comment on https://github.com/pytorch/pytorch/issues/92701 Args: device (device or string): the device to set as default Example:: >>> # xdoctest: +SKIP("requires cuda, changes global state") >>> torch.tensor([1.2, 3]).device device(type='cpu') >>> torch.set_default_device('cuda') # current device is 0 >>> torch.tensor([1.2, 3]).device device(type='cuda', index=0) >>> torch.set_default_device('cuda:1') >>> torch.tensor([1.2, 3]).device device(type='cuda', index=1) Nr) DeviceContext)_GLOBAL_DEVICE_CONTEXT__exit__Ztorch.utils._devicer __enter__)devicerrrrr s& cCs t|trt|}t|dS)aSets the default ``torch.Tensor`` type to floating point tensor type ``t``. This type will also be used as default floating point type for type inference in :func:`torch.tensor`. The default floating point tensor type is initially ``torch.FloatTensor``. Args: t (type or string): the floating point tensor type or its name Example:: >>> # xdoctest: +SKIP("Other tests may have changed the default type. Can we reset it?") >>> torch.tensor([1.2, 3]).dtype # initial default for floating point is torch.float32 torch.float32 >>> torch.set_default_tensor_type(torch.DoubleTensor) >>> torch.tensor([1.2, 3]).dtype # a new floating point tensor torch.float64 N)rrr _CZ_set_default_tensor_type)trrrrPs cCst|dS)a Sets the default floating point dtype to :attr:`d`. Supports torch.float32 and torch.float64 as inputs. Other dtypes may be accepted without complaint but are not supported and are unlikely to work as expected. When PyTorch is initialized its default floating point dtype is torch.float32, and the intent of set_default_dtype(torch.float64) is to facilitate NumPy-like type inference. The default floating point dtype is used to: 1. Implicitly determine the default complex dtype. When the default floating point type is float32 the default complex dtype is complex64, and when the default floating point type is float64 the default complex type is complex128. 2. Infer the dtype for tensors constructed using Python floats or complex Python numbers. See examples below. 3. Determine the result of type promotion between bool and integer tensors and Python floats and complex Python numbers. Args: d (:class:`torch.dtype`): the floating point dtype to make the default. Either torch.float32 or torch.float64. Example: >>> # xdoctest: +SKIP("Other tests may have changed the default type. Can we reset it?") >>> # initial default for floating point is torch.float32 >>> # Python floats are interpreted as float32 >>> torch.tensor([1.2, 3]).dtype torch.float32 >>> # initial default for floating point is torch.complex64 >>> # Complex Python numbers are interpreted as complex64 >>> torch.tensor([1.2, 3j]).dtype torch.complex64 >>> torch.set_default_dtype(torch.float64) >>> # Python floats are now interpreted as float64 >>> torch.tensor([1.2, 3]).dtype # a new floating point tensor torch.float64 >>> # Complex Python numbers are now interpreted as complex128 >>> torch.tensor([1.2, 3j]).dtype # a new complex tensor torch.complex128 N)rZ_set_default_dtype)drrrset_default_dtypeis,r warn_only)rvrrrcCstj||ddS)aJ Sets whether PyTorch operations must use "deterministic" algorithms. That is, algorithms which, given the same input, and when run on the same software and hardware, always produce the same output. When enabled, operations will use deterministic algorithms when available, and if only nondeterministic algorithms are available they will throw a :class:`RuntimeError` when called. .. note:: This setting alone is not always enough to make an application reproducible. Refer to :ref:`reproducibility` for more information. .. note:: :func:`torch.set_deterministic_debug_mode` offers an alternative interface for this feature. The following normally-nondeterministic operations will act deterministically when ``mode=True``: * :class:`torch.nn.Conv1d` when called on CUDA tensor * :class:`torch.nn.Conv2d` when called on CUDA tensor * :class:`torch.nn.Conv3d` when called on CUDA tensor * :class:`torch.nn.ConvTranspose1d` when called on CUDA tensor * :class:`torch.nn.ConvTranspose2d` when called on CUDA tensor * :class:`torch.nn.ConvTranspose3d` when called on CUDA tensor * :func:`torch.bmm` when called on sparse-dense CUDA tensors * :func:`torch.Tensor.__getitem__` when attempting to differentiate a CPU tensor and the index is a list of tensors * :func:`torch.Tensor.index_put` with ``accumulate=False`` * :func:`torch.Tensor.index_put` with ``accumulate=True`` when called on a CPU tensor * :func:`torch.Tensor.put_` with ``accumulate=True`` when called on a CPU tensor * :func:`torch.Tensor.scatter_add_` when called on a CUDA tensor * :func:`torch.gather` when called on a CUDA tensor that requires grad * :func:`torch.index_add` when called on CUDA tensor * :func:`torch.index_select` when attempting to differentiate a CUDA tensor * :func:`torch.repeat_interleave` when attempting to differentiate a CUDA tensor * :func:`torch.Tensor.index_copy` when called on a CPU or CUDA tensor * :func:`torch.Tensor.scatter` when `src` type is Tensor and called on CUDA tensor * :func:`torch.Tensor.scatter_reduce` when ``reduce='sum'`` or ``reduce='mean'`` and called on CUDA tensor * :func:`torch.Tensor.resize_`, when called with a tensor that is not quantized, sets new elements to a known value. Floating point or complex values are set to NaN. Integer values are set to the maximum value. * :func:`torch.empty`, :func:`torch.empty_like`, :func:`torch.empty_strided`, and :func:`torch.empty_permuted` will fill the output tensor with a known value. Floating point or complex dtype tensors are filled with NaN. Integer dtype tensors are filled with the maximum value. The following normally-nondeterministic operations will throw a :class:`RuntimeError` when ``mode=True``: * :class:`torch.nn.AvgPool3d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.AdaptiveAvgPool2d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.AdaptiveAvgPool3d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.MaxPool3d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.AdaptiveMaxPool2d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.FractionalMaxPool2d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.FractionalMaxPool3d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.MaxUnpool1d` * :class:`torch.nn.MaxUnpool2d` * :class:`torch.nn.MaxUnpool3d` * :func:`torch.nn.functional.interpolate` when attempting to differentiate a CUDA tensor and one of the following modes is used: - ``linear`` - ``bilinear`` - ``bicubic`` - ``trilinear`` * :class:`torch.nn.ReflectionPad1d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.ReflectionPad2d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.ReflectionPad3d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.ReplicationPad1d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.ReplicationPad2d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.ReplicationPad3d` when attempting to differentiate a CUDA tensor * :class:`torch.nn.NLLLoss` when called on a CUDA tensor * :class:`torch.nn.CTCLoss` when attempting to differentiate a CUDA tensor * :class:`torch.nn.EmbeddingBag` when attempting to differentiate a CUDA tensor when ``mode='max'`` * :func:`torch.Tensor.put_` when ``accumulate=False`` * :func:`torch.Tensor.put_` when ``accumulate=True`` and called on a CUDA tensor * :func:`torch.histc` when called on a CUDA tensor * :func:`torch.bincount` when called on a CUDA tensor and ``weights`` tensor is given * :func:`torch.kthvalue` with called on a CUDA tensor * :func:`torch.median` with indices output when called on a CUDA tensor * :func:`torch.nn.functional.grid_sample` when attempting to differentiate a CUDA tensor * :func:`torch.cumsum` when called on a CUDA tensor when dtype is floating point or complex * :func:`torch.Tensor.scatter_reduce` when ``reduce='prod'`` and called on CUDA tensor * :func:`torch.Tensor.resize_` when called with a quantized tensor A handful of CUDA operations are nondeterministic if the CUDA version is 10.2 or greater, unless the environment variable ``CUBLAS_WORKSPACE_CONFIG=:4096:8`` or ``CUBLAS_WORKSPACE_CONFIG=:16:8`` is set. See the CUDA documentation for more details: ``_ If one of these environment variable configurations is not set, a :class:`RuntimeError` will be raised from these operations when called with CUDA tensors: * :func:`torch.mm` * :func:`torch.mv` * :func:`torch.bmm` Note that deterministic operations tend to have worse performance than nondeterministic operations. .. note:: This flag does not detect or prevent nondeterministic behavior caused by calling an inplace operation on a tensor with an internal memory overlap or by giving such a tensor as the :attr:`out` argument for an operation. In these cases, multiple writes of different data may target a single memory location, and the order of writes is not guaranteed. Args: mode (:class:`bool`): If True, makes potentially nondeterministic operations switch to a deterministic algorithm or throw a runtime error. If False, allows nondeterministic operations. Keyword args: warn_only (:class:`bool`, optional): If True, operations that do not have a deterministic implementation will throw a warning instead of an error. Default: ``False`` Example:: >>> # xdoctest: +SKIP >>> torch.use_deterministic_algorithms(True) # Forward mode nondeterministic error >>> torch.randn(10, device='cuda').kthvalue(0) ... RuntimeError: kthvalue CUDA does not have a deterministic implementation... # Backward mode nondeterministic error >>> torch.nn.AvgPool3d(1)(torch.randn(3, 4, 5, 6, requires_grad=True).cuda()).sum().backward() ... RuntimeError: avg_pool3d_backward_cuda does not have a deterministic implementation... rN)r_set_deterministic_algorithms)rvrrrrr;s cCstS)zReturns True if the global deterministic flag is turned on. Refer to :func:`torch.use_deterministic_algorithms` documentation for more details. )r_get_deterministic_algorithmsrrrrr<#scCstS)zReturns True if the global deterministic flag is set to warn only. Refer to :func:`torch.use_deterministic_algorithms` documentation for more details. )r'_get_deterministic_algorithms_warn_onlyrrrrr=)s) debug_moderrcCst|tjtfs"tdt|t|trd|dkr:d}n*|dkrHd}n|dkrVd}ntd||dkrxtd n:|dkrtjd d d n"|dkrtd ntd |d S)aSets the debug mode for deterministic operations. .. note:: This is an alternative interface for :func:`torch.use_deterministic_algorithms`. Refer to that function's documentation for details about affected operations. Args: debug_mode(str or int): If "default" or 0, don't error or warn on nondeterministic operations. If "warn" or 1, warn on nondeterministic operations. If "error" or 2, error on nondeterministic operations. z'debug_mode must be str or int, but got defaultrwarnr errorzQinvalid value of debug_mode, expected one of `default`, `warn`, `error`, but got FTrz:invalid value of debug_mode, expected 0, 1, or 2, but got N) rrrr TypeErrorr RuntimeErrorrr)rrrrr>0s*   cCs"trtrdSdSndSdS)zReturns the current value of the debug mode for deterministic operations. Refer to :func:`torch.set_deterministic_debug_mode` documentation for more details. r rrN)rrrrrrrr?Zs cCstS)zReturns the current value of float32 matrix multiplication precision. Refer to :func:`torch.set_float32_matmul_precision` documentation for more details. )rZ_get_float32_matmul_precisionrrrrrAhs) precisionrrcCst|dS)a Sets the internal precision of float32 matrix multiplications. Running float32 matrix multiplications in lower precision may significantly increase performance, and in some programs the loss of precision has a negligible impact. Supports three settings: * "highest", float32 matrix multiplications use the float32 datatype (24 mantissa bits) for internal computations. * "high", float32 matrix multiplications either use the TensorFloat32 datatype (10 mantissa bits) or treat each float32 number as the sum of two bfloat16 numbers (approximately 16 mantissa bits), if the appropriate fast matrix multiplication algorithms are available. Otherwise float32 matrix multiplications are computed as if the precision is "highest". See below for more information on the bfloat16 approach. * "medium", float32 matrix multiplications use the bfloat16 datatype (8 mantissa bits) for internal computations, if a fast matrix multiplication algorithm using that datatype internally is available. Otherwise float32 matrix multiplications are computed as if the precision is "high". When using "high" precision, float32 multiplications may use a bfloat16-based algorithm that is more complicated than simply truncating to some smaller number mantissa bits (e.g. 10 for TensorFloat32, 8 for bfloat16). Refer to [Henry2019]_ for a complete description of this algorithm. To briefly explain here, the first step is to realize that we can perfectly encode a single float32 number as the sum of three bfloat16 numbers (because float32 has 24 mantissa bits while bfloat16 has 8, and both have the same number of exponent bits). This means that the product of two float32 numbers can be exactly given by the sum of nine products of bfloat16 numbers. We can then trade accuracy for speed by dropping some of these products. The "high" precision algorithm specifically keeps only the three most significant products, which conveniently excludes all of the products involving the last 8 mantissa bits of either input. This means that we can represent our inputs as the sum of two bfloat16 numbers rather than three. Because bfloat16 fused-multiply-add (FMA) instructions are typically >10x faster than float32 ones, it's faster to do three multiplications and 2 additions with bfloat16 precision than it is to do a single multiplication with float32 precision. .. [Henry2019] http://arxiv.org/abs/1904.06376 .. note:: This does not change the output dtype of float32 matrix multiplications, it controls how the internal computation of the matrix multiplication is performed. .. note:: This does not change the precision of convolution operations. Other flags, like `torch.backends.cudnn.allow_tf32`, may control the precision of convolution operations. .. note:: This flag currently only affects one native device type: CUDA. If "high" or "medium" are set then the TensorFloat32 datatype will be used when computing float32 matrix multiplications, equivalent to setting `torch.backends.cuda.matmul.allow_tf32 = True`. When "highest" (the default) is set then the float32 datatype is used for internal computations, equivalent to setting `torch.backends.cuda.matmul.allow_tf32 = False`. Args: precision(str): can be set to "highest" (default), "high", or "medium" (see above). N)rZ_set_float32_matmul_precision)rrrrr@ns?)rrrcCst|dS)aWhen this flag is False (default) then some PyTorch warnings may only appear once per process. This helps avoid excessive warning information. Setting it to True causes these warnings to always appear, which may be helpful when debugging. Args: b (:class:`bool`): If True, force warnings to always be emitted If False, set to the default behaviour N)rZ_set_warnAlways)rrrrrBs cCstS)zReturns True if the global warn_always flag is turned on. Refer to :func:`torch.set_warn_always` documentation for more details. )rZ_get_warnAlwaysrrrrrCscondmessagecCst|tjtjfs$tdt|tjjj |r8dSt |t rLt |t rPt|dkr^d}nt|sntdt|}||dS)Nzcond must be a bool, but got zExpected cond to be True, but got False. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)zmessage must be a callable)rrrrrFrrZfxZ experimentalZsymbolic_shapesZ expect_true issubclass ExceptionWarningricallabler) error_typerrZmessage_evaluatedrrr _check_withs rcCstt||dS)aThrows error containing an optional message if the specified condition is False. Error type: ``RuntimeError`` C++ equivalent: ``TORCH_CHECK`` Args: cond (:class:`bool`): If False, throw error message (Callable, optional): Callable that returns either a string or an object that has a ``__str__()`` method to be used as the error message. Default: ``None`` N)rrrrrr_checksrcCstt||dS)aThrows error containing an optional message if the specified condition is False. Error type: ``IndexError`` C++ equivalent: ``TORCH_CHECK_INDEX`` Args: cond (:class:`bool`): If False, throw error message (Callable, optional): Callable that returns either a string or an object that has a ``__str__()`` method to be used as the error message. Default: ``None`` N)r IndexErrorrrrr _check_indexsrcCstt||dS)aThrows error containing an optional message if the specified condition is False. Error type: ``ValueError`` C++ equivalent: ``TORCH_CHECK_VALUE`` Args: cond (:class:`bool`): If False, throw error message (Callable, optional): Callable that returns either a string or an object that has a ``__str__()`` method to be used as the error message. Default: ``None`` N)rrjrrrr _check_valuesrcCstt||dS)aThrows error containing an optional message if the specified condition is False. Error type: ``TypeError`` C++ equivalent: ``TORCH_CHECK_TYPE`` Args: cond (:class:`bool`): If False, throw error message (Callable, optional): Callable that returns either a string or an object that has a ``__str__()`` method to be used as the error message. Default: ``None`` N)rrrrrr _check_typesrcCstt||dS)aThrows error containing an optional message if the specified condition is False. Error type: ``NotImplementedError`` C++ equivalent: ``TORCH_CHECK_NOT_IMPLEMENTED`` Args: cond (:class:`bool`): If False, throw error message (Callable, optional): Callable that returns either a string or an object that has a ``__str__()`` method to be used as the error message. Default: ``None`` N)rNotImplementedErrorrrrr_check_not_implemented$srcCsPt|stdt||jtjks8td|jt|||dS)Nzcond must be a tensor, but got z0cond tensor must have dtype torch.bool, but got ) rrrrdtyperrZ _is_all_trueitem)rrrrrr_check_tensor_all_with5s   rcCstt||dS)aThrows error containing an optional message if the specified condition is False. Error type: ``RuntimeError`` C++ equivalent: ``TORCH_CHECK_TENSOR_ALL`` Args: cond (:class:`torch.Tensor`): Tensor of dtype ``torch.bool``. If any element is ``False``, throw error message (Callable, optional): Callable that returns either a string or an object that has a ``__str__()`` method to be used as the error message. Default: ``None`` N)rrrrrr_check_tensor_all@sr)enaninfpirrrr)r9) _StorageBaser7_LegacyStorager8_warn_typed_storage_removalc@s$eZdZeddZeddZdS)r5cCs t|jSrr_dtyperrrrrfszByteStorage.dtypecCstjSr)rZuint8rrrrrkszByteStorage._dtypeNrrrr rrrrrrr5es c@s$eZdZeddZeddZdS)r/cCs t|jSrrrrrrrpszDoubleStorage.dtypecCstjSr)rdoublerrrrruszDoubleStorage._dtypeNrrrrrr/os c@s$eZdZeddZeddZdS)r0cCs t|jSrrrrrrrzszFloatStorage.dtypecCstjSr)rfloatrrrrrszFloatStorage._dtypeNrrrrrr0ys c@s$eZdZeddZeddZdS) HalfStoragecCs t|jSrrrrrrrszHalfStorage.dtypecCstjSr)rZhalfrrrrrszHalfStorage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS)r1cCs t|jSrrrrrrrszLongStorage.dtypecCstjSr)rlongrrrrrszLongStorage._dtypeNrrrrrr1s c@s$eZdZeddZeddZdS)r2cCs t|jSrrrrrrrszIntStorage.dtypecCstjSr)rrrrrrrszIntStorage._dtypeNrrrrrr2s c@s$eZdZeddZeddZdS)r3cCs t|jSrrrrrrrszShortStorage.dtypecCstjSr)rshortrrrrrszShortStorage._dtypeNrrrrrr3s c@s$eZdZeddZeddZdS)r4cCs t|jSrrrrrrrszCharStorage.dtypecCstjSr)rZint8rrrrrszCharStorage._dtypeNrrrrrr4s c@s$eZdZeddZeddZdS)r6cCs t|jSrrrrrrrszBoolStorage.dtypecCstjSr)rrrrrrrszBoolStorage._dtypeNrrrrrr6s c@s$eZdZeddZeddZdS)BFloat16StoragecCs t|jSrrrrrrrszBFloat16Storage.dtypecCstjSr)rZbfloat16rrrrrszBFloat16Storage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS)ComplexDoubleStoragecCs t|jSrrrrrrrszComplexDoubleStorage.dtypecCstjSr)rZcdoublerrrrrszComplexDoubleStorage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS)ComplexFloatStoragecCs t|jSrrrrrrrszComplexFloatStorage.dtypecCstjSr)rZcfloatrrrrrszComplexFloatStorage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS) QUInt8StoragecCs t|jSrrrrrrrszQUInt8Storage.dtypecCstjSr)rZquint8rrrrrszQUInt8Storage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS) QInt8StoragecCs t|jSrrrrrrrszQInt8Storage.dtypecCstjSr)rZqint8rrrrrszQInt8Storage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS) QInt32StoragecCs t|jSrrrrrrrszQInt32Storage.dtypecCstjSr)rZqint32rrrrrszQInt32Storage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS)QUInt4x2StoragecCs t|jSrrrrrrrszQUInt4x2Storage.dtypecCstjSr)rZquint4x2rrrrrszQUInt4x2Storage._dtypeNrrrrrrs rc@s$eZdZeddZeddZdS)QUInt2x4StoragecCs t|jSrrrrrrrszQUInt2x4Storage.dtypecCstjSr)rZquint2x4rrrrr szQUInt2x4Storage._dtypeNrrrrrr s r _tensor_classes)r r!r"r#r$)r%r&)r'cCsPtstdkrdStddd}ttdtj|sFtd|| dS)NrsrrQZtorch_shm_managerz$Unable to find torch_shm_manager at zutf-8) rrgrhr r rTrUrVrencode)rUrrr manager_path#s    r )autocast)Z unique_dim__segment_reduce)_disable_dynamocCsHddlm}m}t|tjk r8||fr8|t|f||S|sDt|dS)zFA wrapper around Python's assert which is symbolically traceable. r )has_torch_functionhandle_torch_functionN)Z overridesrrrrr9_assertri) conditionrrrrrrrusr)cpu)mps)autograd)r,r-set_grad_enabledr.)fft)futures)_awaits)nested)nn)windows)optim)multiprocessing)sparse)special)jit)linalg)hub)random) distributions)testing)backends) __config__) __future__)profiler)ao) _torch_docs _tensor_docs _storage_docscCstjS)z?Returns whether PyTorch was built with _GLIBCXX_USE_CXX11_ABI=1)rZ_GLIBCXX_USE_CXX11_ABIrrrrcompiled_with_cxx11_abisr2)ops)classes) quantization) quasirandom)register_after_fork)r:) from_dlpack to_dlpack)masked) matrix_rankeigsolvelstsq)_symeigc@sdeZdZdZddZddZeedddZee ee fd d d Z d d Z ddZ ddZdS)_TorchCompileInductorWrapperinductorcCs>t|_||_|||||jddr:dtjd<dS)Ntriton.cudagraphsF1ZDISABLE_CUPTI_LAZY_REINIT)dictconfigdynamic apply_mode apply_optionsrrTenviron)rrvoptionsrFrrrrs   z%_TorchCompileInductorWrapper.__init__cCs"t|to |j|jko |j|jkSr)rr@rErFrrrrrs    z#_TorchCompileInductorWrapper.__eq__rucCsN|dksJ|dkrn8|dkr:ddlm}||||jntd|ddS)Nr)zreduce-overheadz max-autotunezmax-autotune-no-cudagraphsr)list_mode_optionszUnrecognized mode=zV, should be one of: default, reduce-overhead, max-autotune, max-autotune-no-cudagraphs)torch._inductorrKrHrFr)rrvrKrrrrGs  z'_TorchCompileInductorWrapper.apply_mode)rJc Cs|sdSddlm}|}|D]\}}|dd}||kr\td|dt|t|t||k rt|j }t||j }td|d|d |||j|<q$dS) NrrE-r`zUnexpected optimization option z, known options are zUnexpected type of attr z, got z should be ) rLrEto_dictrreplacerlistkeysrr) rrJrEZcurrent_configkeyval attr_nameZ val_type_strZexpected_type_strrrrrHs"   z*_TorchCompileInductorWrapper.apply_optionscCsddlm}||||jdS)Nr) compile_fxZconfig_patches)torch._inductor.compile_fxrVrE)rmodel_inputs_rVrrr__call__s z%_TorchCompileInductorWrapper.__call__cCsddlm}||jdS)Nr)get_patched_config_dictrW)rXr\rE)rr\rrrget_compiler_config"s z0_TorchCompileInductorWrapper.get_compiler_configcCsBddlm}d|jks|jjr>|jddr>ddlm}|dS)NrrMrBT)reset_cudagraph_trees)rLrEtritonZ cudagraphsrZtorch._inductor.cudagraph_treesr^)rrEr^rrrreset&s   z"_TorchCompileInductorWrapper.resetN)rrr compiler_namerrrrrGrrrHr[r]r`rrrrr@s  r@c@s$eZdZddZddZddZdS)_TorchCompileWrappercCszddlm}t|tr||_nt|dr2|j|_n t||_||_|||_i|_ |rh|dkrh||j d<|rv||j d<dS)Nr)lookup_backendrrrvrJ) Ztorch._dynamo.backends.registryrcrrrarrrF compiler_fnkwargs)rbackendrvrJrFrcrrrr.s        z_TorchCompileWrapper.__init__cCs.t|to,|j|jko,|j|jko,|j|jkSr)rrbrdrerFrrrrr@s    z_TorchCompileWrapper.__eq__cCs|j||f|jSr)rdre)rrYrZrrrr[Fsz_TorchCompileWrapper.__call__N)rrrrrr[rrrrrb-srbrA fullgraphrFrfrvrJdisable)modelrhrFrfrvrJrirrcstdtjdkrtd|dkrDtdfdd }|Sdk r\dk r\tddkrpdkrpd d krtnttj j d |S) a Optimizes given model/function using TorchDynamo and specified backend. Concretely, for every frame executed within the compiled region, we will attempt to compile it and cache the compiled result on the code object for future use. A single frame may be compiled multiple times if previous compiled results are not applicable for subsequent calls (this is called a "guard failure), you can use TORCH_LOGS=guards to debug these situations. Multiple compiled results can be associated with a frame up to ``torch._dynamo.config.cache_size_limit``, which defaults to 64; at which point we will fall back to eager. Note that compile caches are per *code object*, not frame; if you dynamically create multiple copies of a function, they will all share the same code cache. Args: model (Callable): Module/function to optimize fullgraph (bool): Whether it is ok to break model into several subgraphs dynamic (bool or None): Use dynamic shape tracing. When this is True, we will up-front attempt to generate a kernel that is as dynamic as possible to avoid recompilations when sizes change. This may not always work as some operations/optimizations will force specialization; use TORCH_LOGS=dynamic to debug overspecialization. When this is False, we will NEVER generate dynamic kernels, we will always specialize. By default (None), we automatically detect if dynamism has occurred and compile a more dynamic kernel upon recompile. backend (str or Callable): backend to be used - "inductor" is the default backend, which is a good balance between performance and overhead - Non experimental in-tree backends can be seen with `torch._dynamo.list_backends()` - Experimental or debug in-tree backends can be seen with `torch._dynamo.list_backends(None)` - To register an out-of-tree custom backend: https://pytorch.org/docs/main/compile/custom-backends.html mode (str): Can be either "default", "reduce-overhead", "max-autotune" or "max-autotune-no-cudagraphs" - "default" is the default mode, which is a good balance between performance and overhead - "reduce-overhead" is a mode that reduces the overhead of python with CUDA graphs, useful for small batches. Reduction of overhead can come at the cost of more memory usage, as we will cache the workspace memory required for the invocation so that we do not have to reallocate it on subsequent runs. Reduction of overhead is not guaranteed to work; today, we only reduce overhead for CUDA only graphs which do not mutate inputs. There are other circumstances where CUDA graphs are not applicable; use TORCH_LOG=perf_hints to debug. - "max-autotune" is a mode that leverages Triton based matrix multiplications and convolutions It enables CUDA graphs by default. - "max-autotune-no-cudagraphs" is a mode similar to "max-autotune" but without CUDA graphs - To see the exact configs that each mode sets you can call `torch._inductor.list_mode_options()` options (dict): A dictionary of options to pass to the backend. Some notable ones to try out are - `epilogue_fusion` which fuses pointwise ops into templates. Requires `max_autotune` to also be set - `max_autotune` which will profile to pick the best matmul configuration - `fallback_random` which is useful when debugging accuracy issues - `shape_padding` which pads matrix shapes to better align loads on GPUs especially for tensor cores - `triton.cudagraphs` which will reduce the overhead of python with CUDA graphs - `trace.enabled` which is the most useful debugging flag to turn on - `trace.graph_diagram` which will show you a picture of your graph after fusion - For inductor you can see the full list of configs that it supports by calling `torch._inductor.list_options()` disable (bool): Turn torch.compile() into a no-op for testing Example:: @torch.compile(options={"triton.cudagraphs": True}, fullgraph=True) def foo(x): return torch.sin(x) + torch.cos(x) z torch.compile) z'Dynamo is not supported on Python 3.12+Nrjc s&|dkrtdt|dS)NzModel can't be Nonerg)rrLrmrfrirFrhrvrJrrfnszcompile..fnzVEither mode or options can be specified, but both can't be specified at the same time.rrA)rfZnopythonrFri) rZ_log_api_usage_oncer version_inforrr@rbr_dynamooptimize)rjrhrFrfrvrJrirorrnrrLJsU   )rNcCsdt|j}tjt}t||rr?rAr@rBrCrrrrrrrrrrrrZ_tensorr9Zstoragerr7rr8rr5r/r0rr1r2r3r4r6rrrrrrrr rsetr rr'r r!r"r#r$Z serializationr%r&Z _tensor_strr'r Z torch.amprrrrZtorch._C._VariableFunctionsrZ_segment_reduceZ PRIVATE_OPSZ_VariableFunctions startswithglobals_compilerZ functionalrrrrrZtorch.autogradr,r-rr.rrrrrZ torch.signalrr Ztorch.optim._multi_tensorr!r"r#Ztorch.utils.backcompatr$r%r&r(r)r*Ztorch.utils.datar+r,r-r.Ztorch.nn.quantizableZtorch.nn.quantizedZ torch.nn.qatZtorch.nn.intrinsicZ _init_namesrSr/r0r1r2Z torch._opsr3Ztorch._classesr4r5r6Zcontiguous_formatZlegacy_contiguous_formatZtorch.multiprocessing._atforkr7Zget_num_threadsZ_lobpcgr:ZatenZquantized_lstmZ quantized_gruZtorch.utils.dlpackr8r9r:Z _linalg_utilsr;r<r=r>r?Zsymeigr@rbrLrNrwrxryrzZtorch.cuda._sanitizerZ _sanitizerZcsanZenable_cuda_sanitizerZ%torch.fx.experimental.symbolic_shapesr{Z torch.funcrMr}_initr~rZis_builtrwZ is_availableZmkldnnrrqrrrrrZ _init_logsrrrrs  0               "    2,3                  0. *A  "                                                             *B   t