U Ò9%e­Jã @s˜ddlZddlmZddlmZmZddlZddgZeddƒZej edd d ƒej ed d d ƒej ed d d ƒiZ Gd d„dej ƒZdej eedœdd„ZdS)éN)Ú namedtuple)ÚAnyÚOptionalÚSparseSemiStructuredTensorÚto_sparse_semi_structuredÚ_SEMI_STRUCTURED_SPARSE_CONFIGz$compression_factor min_rows min_colsé é é€é é@c@s®eZdZdZdZdZdZedee j ee j ee j e dœdd„ƒZ edd„ƒZdee j ee j ee j e dd œd d „Zed œd d„Ze jjZeed œdd„ƒZdd„ZdS)raÇThis class implementes semi-structured sparsity as a Tensor subclass. Semi-structured sparsity describes a sparsity pattern where n in every 2n elements are sparse, depending on the datatype. It is also referred to as 2:4 sparsity or fine-grained structured sparsity. Currently, this class supports 2:4 sparsity for int8, float16 and bfloat16 dtypes. We also support 1:2 sparsity for float32 dtype. This subclass stores the dense tensor in a compressed form by only storing the specified elements and corresponding metadata. These two are stored next to each other in one contiguous tensor. We choose to store the specified elements and the metadata in a single tensor for compatibilty with cuSPARSELt, which expects the data to be stored in this format. compressed tensor = [ specified elements of original tensor | metadata ] For an original tensor of size (m, k) we expect the first m * k // 2 elements to be the kept elements The rest of the tensor is metadata. The subclass supports two backend, either CUTLASS or cuSPASRELt. When _FORCE_CUTLASS is set, or when cuSPARSELt is not available, this subclass calls into _sparse_semi_structured_linear and sparse_semi_structured_from_dense for conversion to the compressed format. When PyTorch is compiled with cuSPARSELt support, this subclass will call into _cslt_sparse_mm for sparse mm and _cslt_compress to convert into the compressed format. FN)Úoriginal_tensorÚoriginal_shapeÚcompressed_tensorÚ transposedcCs~|jst dt¡d|_|dk r,|}|j}n|dk r:|}ntdƒ‚i}|j|d<|j|d<|j|d<d|d <t j j ||f|ŽS) a Create a new instance of the class. When original_tensor is passed in, we compress it and store the compresed representation. We can also create new instance of the class from the compressed representation without the original tensor. Args: original_tensor: The original dense tensor, or None, if we have already compressed the tensor. original_shape: The shape of the original dense tensor compressed_tensor: A flattened tensor to store the specified elements and metadata. transposed: Whether the tensor is transposed or not. Returns: torch.Tensor: A torch.Tensor wrapper subclass. Raises: ValueError: If both original_tensor and compressed_tensor are None. zøThe PyTorch API of SparseSemiStructuredTensor is in prototype stage and will change in the near future. Please open a Github issue for features requests and see our documentation on the torch.sparse module for further information about the project.TNz4Both compressed_tensor and original_tensor are None!ÚdeviceÚdtypeÚlayoutFZ requires_grad) Ú_WARNING_SHOWNÚwarningsÚwarnÚ UserWarningÚshapeÚ ValueErrorrrrÚtorchÚTensorZ_make_wrapper_subclass)Úclsr rrrZprevious_tensorÚkwargs©rú[/var/www/html/Darija-Ai-API/env/lib/python3.8/site-packages/torch/sparse/semi_structured.pyÚ__new__:s$ù    z"SparseSemiStructuredTensor.__new__cCs:|tjkrtjS|tjtjfkr&tjStd|›dƒ‚dS)Nz Datatype z is not supported!)rÚint8Zint32Úfloat16Úbfloat16Zint16Ú RuntimeError)Z values_dtyperrrZ__get_indices_dtypeqs  z.SparseSemiStructuredTensor.__get_indices_dtype)r rrrÚreturncCsf|dk rP|js"td|j›dƒ‚| ¡dkrBtd| ¡›dƒ‚|jtkr^td|j›dƒ‚|j\}}t|jj}t|jj}||ks ||s ||ks ||r²td |j›d ƒ‚|j rF|  ¡} t|jj } | | d } t j | f|j|jd }d dlm} | |ƒ\} }|  d¡|d||d…<| |j¡ d¡|||dd…<n t  |¡}d|_||_||_dS)a SparseSemiStructuredTensor constructor. Args: original_tensor: The original dense tensor, or None, if we have already compressed the tensor. original_shape: The shape of the original dense tensor compressed_tensor: A flattened tensor to store the specified elements and metadata. transposed: Whether the tensor is transposed or not. Returns: None Raises: RuntimeError: If original_tensor is not a supported dtype, dim, shape, or device. NzError original_tensor.device= z= is not supported! Only CUDA tensors are currently supported.ézError original_tensor.dim = z; is not supported! Only 2d tensors are currently supported.zError original_tensor.dtype zZ is not a supported dtype! dtype must be one of: {_DTYPE_TO_SEMI_STRUCTURED_SPARSE_CONFIG}zError original_tensor.shape zj is not supported! Both dimensions must be larger or equal than and a multiple of ({min_rows}, {min_cols})é)rrr)Ú!sparse_semi_structured_from_denseéÿÿÿÿ)Zis_cudar$rÚdimrÚ'_DTYPE_TO_SEMI_STRUCTURED_SPARSE_CONFIGrÚmin_rowsÚmin_colsÚ_FORCE_CUTLASSZnelementÚcompression_factorrÚemptyÚ)torch.sparse._semi_structured_conversionsr(ÚviewZ_cslt_compressr rr)Úselfr rrrÚmÚnr,r-Z original_sizer/Zcompressed_sizer(ÚsparseÚmetarrrÚ__init__{sZ  ÿ ÿ  ÿ ÿÿ  ÿÿ ý   ÿ z#SparseSemiStructuredTensor.__init__)r%c Cs*d|j›d|j›d| ¡›d| ¡›d S)zžReturn string representation of SparseSemiStructuredTensor Returns: str: String representation Raises: None z!SparseSemiStructuredTensor(shape=z , transposed=zvalues=z metadata=ú))rrÚvaluesÚindices)r3rrrÚ__repr__Ûs (ÿz#SparseSemiStructuredTensor.__repr__cCsâ|tjjjjkr8t|dj|dj|dj|dj dS|tjjj jkrrt|dj|dj|dj|dj dS|tjjj jkrÒ|\}}}t ||ƒrÒ|j rÒ|j r¼tj|| ¡| ¡|dSt |j|j|¡  ¡S|tjjjjkr€|\}}t ||ƒr8|j s8|j r&t |  ¡| ¡| ¡¡  ¡St |j|d¡SnHt ||ƒr€|j r€|j rjt || ¡| ¡¡St |j|jd¡  ¡S|tjjjjkr|\}} }|j} t | |ƒr|j rÐtj||  ¡|  ¡|dSt | j| d| d¡  ¡|¡  ¡j| dd…d žŽS|tjjjjkrP|dj\} } | | d} |djd| … | | d¡S|tjjjjkr°|dj\} } | | d} |dj| d… | d¡}t |dj¡}| |¡Sd d|›d gd d „t|ƒDƒ¡}t|ƒ‚dS) a‡Overload __torch_dispatch__ to use torch._sparse_semi_structured_linear. `torch.structured_sparse_linear` uses accelerated sparse CUTLASS kernels. In the future we plan to also add in support for cuSPARSELt kernels. Args: func: The function being dispatched. types: The types of the arguments. args: The arguments passed to the function. kwargs: The keyword arguments passed to the function. Returns: Any: The result of the dispatched operation. Raises: NotImplementedError: If the dispatched operation is not implemented. r)rrr)ÚbiasNr)r&Ú zfunc z with args: cSs g|]\}}d|›d|›‘qS)Úargz: r)Ú.0Úir?rrrÚ jszASparseSemiStructuredTensor.__torch_dispatch__..)r))rZopsZatenÚdetachÚdefaultrr rrrÚtZaddmmÚ isinstancer.Z_sparse_semi_structured_linearr:r;Z_cslt_sparse_mmÚTÚmmZlinearr2Ú._SparseSemiStructuredTensor__get_indices_dtyperÚjoinÚ enumerateÚNotImplementedError)rÚfuncÚtypesÚargsrr=Zinput_AZinput_BZ input_tensorÚweightrr4ÚkZnum_kept_elementsÚmetadataÚ indices_dtypeZ error_stringrrrÚ__torch_dispatch__ís°ü  ü ÿÿÿÿÿ  üý ü  ÿ  ÿÿz-SparseSemiStructuredTensor.__torch_dispatch__cCsx|jdkrtdƒ‚|j\}}t |j¡}ddlm}||jd||d… |d¡|j||dd… |¡ |d¡ƒS)Nz6Compressed tensor is not set, cannot convert to dense!r)Úsparse_semi_structured_to_denser&r)) rr$rrrIrr1rUr2)r3r4r5rSrUrrrÚto_densens    "þz#SparseSemiStructuredTensor.to_dense)NNF)NNF)Ú__name__Ú __module__Ú __qualname__Ú__doc__Z_FUSE_TRANSPOSEr.rÚ staticmethodrrrÚSizeÚboolr rIr8Ústrr<Z_CZ_disabled_torch_function_implZ__torch_function__Ú classmethodrrTrVrrrrrs@ûû6 ûú `F)r rr%cCst||j|dS)a This function converts a dense tensor into a sparse semi-structured tensor. It will return a SparseSemiStructuredTensor, a subclass of torch.Tensor. This function will check to ensure the dense tensor has the right dtype, size, dims, and device. We currently only support semi-structured sparse tensors for 2d CUDA tensors. Additionally, your tensor must be a positive multiple of a block size given the dtype - torch.float16 (r, c) must be >= and a multiple of 64 - torch.int8 (r, c) must be >= and a multiple of 128 Args: original_tensor (Tensor): the dense tensor to convert transposed (bool, optional): whether the dense tensor is transposed Returns: SparseSemiStructuredTensor: A sparse semi-structured tensor created from the given original_tensor Raises: None Example: >>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA) >>> A = torch.Tensor([0, 0, 1, 1]).tile((128, 32)).half().cuda() tensor([[0., 0., 1., ..., 0., 1., 1.], [0., 0., 1., ..., 0., 1., 1.], [0., 0., 1., ..., 0., 1., 1.], ..., [0., 0., 1., ..., 0., 1., 1.], [0., 0., 1., ..., 0., 1., 1.], [0., 0., 1., ..., 0., 1., 1.]], device='cuda:0', dtype=torch.float16) >>> A_sparse = to_sparse_semi_structured(A) SparseSemiStructuredTensor(shape=torch.Size([128, 128]), transposed=False, values=tensor([[1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], ..., [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.]], device='cuda:0', dtype=torch.float16), metadata=tensor([[-4370, -4370, -4370, ..., -4370, -4370, -4370], [-4370, -4370, -4370, ..., -4370, -4370, -4370], [-4370, -4370, -4370, ..., -4370, -4370, -4370], ..., [-4370, -4370, -4370, ..., -4370, -4370, -4370], [-4370, -4370, -4370, ..., -4370, -4370, -4370], [-4370, -4370, -4370, ..., -4370, -4370, -4370]], device='cuda:0', dtype=torch.int16)) )rr)rr)r rrrrrs3)F)rÚ collectionsrÚtypingrrrÚ__all__rr!r"r#r+rrr]rrrrrÚs4 þÿ   ý kþý