U HÉcÁã@s¬dZddlZddlmZddlmmZGdd„dejƒZGdd„dejƒZ Gdd„dejƒZ Gd d „d ejƒZ Gd d „d ejƒZ Gd d„dejƒZ Gdd„dejƒZdS)z Activation functions éNc@seZdZedd„ƒZdS)ÚSiLUcCs|t |¡S©N)ÚtorchÚsigmoid©Úx©rúG/var/www/html/VideoAnalyticsDashboard/src/./yolov5/utils/activations.pyÚforward sz SiLU.forwardN©Ú__name__Ú __module__Ú __qualname__Ú staticmethodr rrrr r src@seZdZedd„ƒZdS)Ú HardswishcCs|t |ddd¡dS)Négg@)ÚFÚhardtanhrrrr r szHardswish.forwardNr rrrr rsrc@seZdZedd„ƒZdS)ÚMishcCs|t |¡ ¡Sr)rÚsoftplusÚtanhrrrr r sz Mish.forwardNr rrrr rsrc@s(eZdZGdd„dejjƒZdd„ZdS)ÚMemoryEfficientMishc@s$eZdZedd„ƒZedd„ƒZdS)zMemoryEfficientMish.FcCs | |¡| t t |¡¡¡Sr)Úsave_for_backwardÚmulrrrr)Úctxrrrr r "s zMemoryEfficientMish.F.forwardcCs>|jd}t |¡}t |¡ ¡}||||d||S)Nré)Ú saved_tensorsrrrrr)rÚ grad_outputrÚsxÚfxrrr Úbackward's  zMemoryEfficientMish.F.backwardN)r r rrr r rrrr r!s rcCs |j |¡Sr)rÚapply©Úselfrrrr r .szMemoryEfficientMish.forwardN)r r rrÚautogradÚFunctionrr rrrr r s rcs&eZdZd‡fdd„ Zdd„Z‡ZS)ÚFReLUrc s4tƒ ¡tj|||dd|dd|_t |¡|_dS)NrF)ÚgroupsÚbias)ÚsuperÚ__init__ÚnnÚConv2dÚconvÚ BatchNorm2dÚbn)r#Úc1Úk©Ú __class__rr r*4s zFReLU.__init__cCst || | |¡¡¡Sr)rÚmaxr/r-r"rrr r 9sz FReLU.forward)r)r r rr*r Ú __classcell__rrr2r r&3sr&cs(eZdZdZ‡fdd„Zdd„Z‡ZS)ÚAconCzì ACON activation (activate or not). AconC: (p1*x-p2*x) * sigmoid(beta*(p1*x-p2*x)) + p2*x, beta is a learnable parameter according to "Activate or Not: Learning Customized Activation" . csVtƒ ¡t t d|dd¡¡|_t t d|dd¡¡|_t t d|dd¡¡|_ dS)Nr) r)r*r+Ú ParameterrÚrandnÚp1Úp2ÚonesÚbeta)r#r0r2rr r*Ds zAconC.__init__cCs.|j|j|}|t |j|¡|j|Sr)r9r:rrr<)r#rÚdpxrrr r Jsz AconC.forward©r r rÚ__doc__r*r r5rrr2r r6>s r6cs*eZdZdZd‡fdd„ Zdd„Z‡ZS) Ú MetaAconCz÷ ACON activation (activate or not). MetaAconC: (p1*x-p2*x) * sigmoid(beta*(p1*x-p2*x)) + p2*x, beta is generated by a small network according to "Activate or Not: Learning Customized Activation" . récsxtƒ ¡t|||ƒ}t t d|dd¡¡|_t t d|dd¡¡|_tj ||||dd|_ tj ||||dd|_ dS)NrT)r() r)r*r4r+r7rr8r9r:r,Úfc1Úfc2)r#r0r1ÚsÚrÚc2r2rr r*Us  zMetaAconC.__init__cCsZ|jdddjddd}t | | |¡¡¡}|j|j|}|t ||¡|j|S)NéT)ÚdimÚkeepdimsr)ÚmeanrrrCrBr9r:)r#rÚyr<r=rrr r _szMetaAconC.forward)rrrAr>rrr2r r@Os r@)r?rÚtorch.nnr+Ztorch.nn.functionalÚ functionalrÚModulerrrrr&r6r@rrrr Ús