U *Ç-efã@sfUddlmZmZmZddlZddlmmZddlm Z gZ ee e d<ej jGdd„dƒƒZdS)é)ÚDictÚListÚOptionalN)ÚTensorÚ__all__c @sZeZdZdeeeeeeeeeedœ dd„Zeeedœdd „Z eeed œd d „Z d S)Ú_FunctionalSGDç{®Gáz„?çF) ÚparamsÚlrÚmomentumÚ dampeningÚ weight_decayÚnesterovÚmaximizeÚforeachÚ_allow_empty_param_listc Csl||||dœ|_||_||_||_tj ttjtt tjffi¡|_ t |ƒdkr^| s^t dƒ‚d|i|_ dS)N)r r r rrz%optimizer got an empty parameter listr )ÚdefaultsrrrÚtorchÚjitZannotaterrÚstrÚstateÚlenÚ ValueErrorÚ param_group) Úselfr r r r rrrrr©rúg/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/torch/distributed/optim/functional_sgd.pyÚ__init__s ü$z_FunctionalSGD.__init__)ÚparamÚgradc Csö|jd}|jd}|jd}|jd}|g}g}g} d} |dk r–|  |¡|jrVd} ||jkrji|j|<|j|} d| krˆ| d¡n| | d¡t ¡,tj|| ||||||j|j | |j d W5QRX|j|} |d } | dk rò| | d<dS) z[Similar to self.step, but operates on a single parameter and its gradient. rr r r FNTÚmomentum_buffer©rr r r rrÚhas_sparse_gradrr) rÚappendÚ is_sparserrÚno_gradÚFÚsgdrrr) rrr rr r r r Úmomentum_buffer_listÚgradsr#rr!rrrÚ step_param3sH          õ z_FunctionalSGD.step_param)Ú gradientscCsh|jd}g}g}g}|jd}|jd}|jd}|jd} t|ƒt|ƒkrttddt|ƒ›dd t|ƒ›ƒ‚d } t||ƒD]n\} } | dk r‚| | ¡| | ¡| jr°d } | |jkrÄi|j| <|j| } d | krâ| d¡q‚| | d ¡q‚t  ¡,t j ||||||| |j |j | |jd W5QRXt|ƒD].\}}|j|} ||}|dk r4|| d <q4dS)Nr r rr r zEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: FTr!r")rrrrÚzipr$r%rrr&r'r(rrrÚ enumerate)rr,r Zparams_with_gradr*r)r rr r r#rZgradientrÚiÚpr!rrrÚstepbs`     ÿ þÿ       õ  z_FunctionalSGD.stepN)rr r r FFFF) Ú__name__Ú __module__Ú __qualname__rrÚfloatÚboolrrr+r1rrrrrs*öö /r)ÚtypingrrrrZtorch.optim._functionalZoptimZ _functionalr'rrrÚ__annotations__rÚscriptrrrrrÚs