激活函数及其梯度
sigmoid
![[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-oWQDSPNI-1644334735092)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174300923.png)]](https://img-blog.csdnimg.cn/a709bfa6208b444988f83ece392286fb.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBAdmljdG9yX2d4,size_20,color_FFFFFF,t_70,g_se,x_16)
Derivative
![[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-6LnisWYL-1644334735095)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174355850.png)]](https://img-blog.csdnimg.cn/bafd5644b1044c08b22a9973edd9985c.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBAdmljdG9yX2d4,size_20,color_FFFFFF,t_70,g_se,x_16)
>>> a=torch.linspace(-100,100,10)
>>> a
tensor([-100.0000, -77.7778, -55.5556, -33.3333, -11.1111, 11.1111,
33.3333, 55.5556, 77.7778, 100.0000])
>>> torch.sigmoid(a)
tensor([0.0000e+00, 1.6655e-34, 7.4564e-25, 3.3382e-15, 1.4945e-05, 9.9999e-01,
1.0000e+00, 1.0000e+00, 1.0000e+00, 1.0000e+00])
Tanh
![[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kpXOeNqk-1644334735096)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174333037.png)]](https://img-blog.csdnimg.cn/8b79cc6159804259923d0c9b7b694e69.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBAdmljdG9yX2d4,size_20,color_FFFFFF,t_70,g_se,x_16)
Derivative
![[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QWxehowX-1644334735097)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174412319.png)]](https://img-blog.csdnimg.cn/b3ae16325397419c98214b6e874fd78f.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBAdmljdG9yX2d4,size_20,color_FFFFFF,t_70,g_se,x_16)
>>> a=torch.linspace(-1,1,10)
>>> a
tensor([-1.0000, -0.7778, -0.5556, -0.3333, -0.1111, 0.1111, 0.3333, 0.5556,
0.7778, 1.0000])
>>> torch.tanh(a)
tensor([-0.7616, -0.6514, -0.5047, -0.3215, -0.1107, 0.1107, 0.3215, 0.5047,
0.6514, 0.7616])
Rectified Linear Unit(ReLU)
![[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fdfxdvap-1644334735098)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174539911.png)]](https://img-blog.csdnimg.cn/3259ce9d50674198aad106321aa408a4.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBAdmljdG9yX2d4,size_20,color_FFFFFF,t_70,g_se,x_16)
Derivative
![[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-pLNFX21i-1644334735098)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174628247.png)]](https://img-blog.csdnimg.cn/b8abf7e33e704b16af19b28c903d0bea.png?x-oss-process=image/watermark,type_d3F5LXplbmhlaQ,shadow_50,text_Q1NETiBAdmljdG9yX2d4,size_20,color_FFFFFF,t_70,g_se,x_16)
>>> a=torch.linspace(-1,1,10)
>>> a
tensor([-1.0000, -0.7778, -0.5556, -0.3333, -0.1111, 0.1111, 0.3333, 0.5556,
0.7778, 1.0000])
>>> torch.relu(a)
tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1111, 0.3333, 0.5556, 0.7778,
1.0000])