理论:
LambdaLR更新学习率方式是 lr = lr*lr_lambda
其中,lr由optim系列优化器提供,lr_lambda由lr_scheduler>lambdaLR提供
假设,lr初始值为0.4,?
更新学习率函数lambda表达式为:lr_lambda = lambda epoch:0.1*epoch)
epoch的初始值为0
则,lr的变化规律应该是
0.4*0.1*0=0
0.4*0.1*1=0.04
0.4*0.1*2=0.08
......
实验:
import torch
from torch import nn
torch.manual_seed(0)
class model(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(in_channels=1,out_channels=1,kernel_size=2,stride=1,padding=0)
self.conv2 = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=2, stride=1, padding=0)
def forward(self,x):
out = self.conv1(x)
return out
net1 = model()
optimizer_1 = torch.optim.SGD(net1.parameters(),lr = 0.4)
scheduler_1 = torch.optim.lr_scheduler.LambdaLR(optimizer_1,lr_lambda = lambda epoch:0.1*epoch)
print('\n当前学习率')
print(scheduler_1.get_lr())
for i in range(10):
scheduler_1.step() # 更新学习率
print(scheduler_1.get_lr())
结果:
?证明完毕!
|