ResNet，是2015年何恺明大佬发表在CVPR上的一篇文章，运用了残差连接这个概念。该论文一出，直接引爆了整个cv界。并且在2016年ImageNet上ResNet获得第一名。而ResNet至今被用在AI各个领域内的前沿技术当中。
要是我以后的论文引用量有ResNet的十分之一我就满足了（笑）
ResNet介绍
ResNet解决的是深度网络的退化问题。按常理讲，网络越深模型就能拟合更复杂的结果。但是在实际训练中，模型一旦加深效果不一定会好很有可能会产生拟合效果差，梯度消失等缺点。比如论文中展示的在CIFAR-10上20层CNN和56层CNN测试精度。由图可知，56层CNN的精度还比20层CNN的精度差。
?在训练过程中，网络回传时是得到每一层网络的梯度再相乘。而越训练到后期或者较深的网络，它的梯度都非常的小，这样相乘后最后得到的总梯度也就很少甚至接近于0。为了解决这一问题，何博士在论文中提出了残差学习这一概念。
残差学习
当我们需要在一个网络的基础上再加几层网络时，常规的做法是直接在后面加网络原先网络的输出做加上网络的输入。但现在我们不这样做，根据残差学习当新网络输入为x时其学习到的特征记为 H(x)?，现在我们希望新网络可以学习到残差值?F(x)=H(x)-x?，这样其实原始的学习特征是?F(x)+x?。也就是说再最后的输出时，我们还是需要在F(x)的基础上加上x。
?在原网络的基础加上新增网络，容易使网络退化梯度变得非常小。而将其输出改为残差值和网络值相加时，在求梯度时就不会有产生小梯度的值。因为在求导时式子中有一个x，众所周知我们对变量进行求导时，x的导数就是1。也可以浅显的说，现在对该层的网络求导得到的梯度是一个小梯度再加上一个1。这样就增加了梯度的值而弥补了梯度会消失的缺点。当然残差梯度不会那么巧全为1，而且就算其比较小，有1的存在也不会导致梯度消失。所以残差学习会更容易。
网络结构
采用了类似与VGG的网络，并在其基础上进行了改进，并通过短路机制加入了残差单元。基本的单元结构还是卷积，BN，激活函数这个套路。但在每个单元的输出位置加上了残差连接，单元输出再加上单元输入最后通过一个激活函数做为最后的输出。
而对于不同层的ResNet来说，残差单元的结构也不相同
?在小于50层时一般残差单元内只有两层卷积，并且一个卷积是3*3卷积核大小然后填充是1不改变feature map的大小，而另一个卷积则将大小缩小一半这个操作为了不使信息丢失的太多将feature map的通道数增大一倍，而且也降低网络的复杂性。大于50层时，先用了一个1*1的卷积层将feature map的通道数映射回我需要的通道数，再通过与上述一样的3*3改变大小的卷积层。最后再通过一个将通道数乘四倍的卷积层。从图中可以看到，ResNet相比普通网络每两层间增加了短路机制，这就形成了残差学习，其中虚线表示feature map数量发生了改变。
?Pytorch实现ResNet
import torch
import time
from torch import nn
# 初始的卷积层,对输入的图片进行处理成feature map
class Conv1(nn.Module):
def __init__(self,inp_channels,out_channels,stride = 2):
super(Conv1,self).__init__()
self.net = nn.Sequential(
nn.Conv2d(inp_channels,out_channels,kernel_size=7,stride=stride,padding=3,bias=False),# 卷积的结果(i - k + 2*p)/s + 1,此时图像大小缩小一半
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
nn.MaxPool2d(kernel_size=3,stride=2,padding=1)# 根据卷积的公式，该feature map尺寸变为原来的一半
)
def forward(self,x):
y = self.net(x)
return y
class Simple_Res_Block(nn.Module):
def __init__(self,inp_channels,out_channels,stride=1,downsample = False,expansion_=False):
super(Simple_Res_Block,self).__init__()
self.downsample = downsample
if expansion_:
self.expansion = 4# 将维度扩展成expansion倍
else:
self.expansion = 1
self.block = nn.Sequential(
nn.Conv2d(inp_channels,out_channels,kernel_size=3,stride=stride,padding=1),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels,out_channels*self.expansion,kernel_size=3,padding=1),
nn.BatchNorm2d(out_channels*self.expansion)
)
if self.downsample:
self.down = nn.Sequential(
nn.Conv2d(inp_channels,out_channels*self.expansion,kernel_size=1,stride=stride,bias=False),
nn.BatchNorm2d(out_channels*self.expansion)
)
self.relu = nn.ReLU(inplace=True)
def forward(self,input):
residual = input
x = self.block(input)
if self.downsample:
residual = self.down(residual)# 使x和h的维度相同
out = residual + x
out = self.relu(out)
return out
class Residual_Block(nn.Module):
def __init__(self,inp_channels,out_channels,stride=1,downsample = False,expansion_=False):
super(Residual_Block,self).__init__()
self.downsample = downsample# 判断是否对x进行下采样使x和该模块输出值维度通道数相同
if expansion_:
self.expansion = 4# 将维度扩展成expansion倍
else:
self.expansion = 1
# 模块
self.conv1 = nn.Conv2d(inp_channels,out_channels,kernel_size=1,stride=1,bias=False)# 不对特征图尺寸发生改变，起映射作用
self.drop = nn.Dropout(0.5)
self.BN1 = nn.BatchNorm2d(out_channels)
self.conv2 = nn.Conv2d(out_channels,out_channels,kernel_size=3,stride=stride,padding=1,bias=False)# 此时卷积核大小和填充大小不会影响特征图尺寸大小,由步长决定
self.BN2 = nn.BatchNorm2d(out_channels)
self.conv3 = nn.Conv2d(out_channels,out_channels*self.expansion,kernel_size=1,stride=1,bias=False)# 改变通道数
self.BN3 = nn.BatchNorm2d(out_channels*self.expansion)
self.relu = nn.ReLU(inplace=True)
if self.downsample:
self.down = nn.Sequential(
nn.Conv2d(inp_channels,out_channels*self.expansion,kernel_size=1,stride=stride,bias=False),
nn.BatchNorm2d(out_channels*self.expansion)
)
def forward(self,input):
residual = input
x = self.relu(self.BN1(self.conv1(input)))
x = self.relu(self.BN2(self.conv2(x)))
h = self.BN3(self.conv3(x))
if self.downsample:
residual = self.down(residual)# 使x和h的维度相同
out = h + residual# 残差部分
out = self.relu(out)
return out
class Resnet(nn.Module):
def __init__(self,net_block,block,num_class = 1000,expansion_=False):
super(Resnet,self).__init__()
self.expansion_ = expansion_
if expansion_:
self.expansion = 4# 将维度扩展成expansion倍
else:
self.expansion = 1
# 输入的初始图像经过的卷积
# (3*64*64) --> (64*56*56)
self.conv = Conv1(3,64)
# 构建模块
# (64*56*56) --> (256*56*56)
self.block1 = self.make_layer(net_block,block[0],64,64,expansion_=self.expansion_,stride=1)# stride为1，不改变尺寸大小
# (256*56*56) --> (512*28*28)
self.block2 = self.make_layer(net_block,block[1],64*self.expansion,128,expansion_=self.expansion_,stride=2)
# (512*28*28) --> (1024*14*14)
self.block3 = self.make_layer(net_block,block[2],128*self.expansion,256,expansion_=self.expansion_,stride=2)
# (1024*14*14) --> (2048*7*7)
self.block4 = self.make_layer(net_block,block[3],256*self.expansion,512,expansion_=self.expansion_,stride=2)
self.avgPool = nn.AvgPool2d(7,stride=1)# (2048*7*7) --> (2048*1*1)经过平均池化层将所有像素融合并取平均
if expansion_:
length = 2048
else:
length = 512
self.linear = nn.Linear(length,num_class)
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.weight, 1)
nn.init.constant_(m.bias, 0)
def make_layer(self,net_block,layers,inp_channels,out_channels,expansion_=False,stride = 1):
block = []
block.append(net_block(inp_channels,out_channels,stride=stride,downsample=True,expansion_=expansion_))# 先将上一个模块的通道数缩小为该模块需要的通道数
if expansion_:
self.expansion = 4
else:
self.expansion = 1
for i in range(1,layers):
block.append(net_block(out_channels*self.expansion,out_channels,expansion_=expansion_))
return nn.Sequential(*block)
def forward(self,x):
x = self.conv(x)
x = self.block1(x)
x = self.block2(x)
x = self.block3(x)
x = self.block4(x)
# x = self.avgPool(x)
x = x.view(x.shape[0],-1)
x = self.linear(x)
return x
def Resnet18():
return Resnet(Simple_Res_Block,[2,2,2,2],num_class=10,expansion_=False)# 此时每个模块里面只有两层卷积
def Resnet34():
return Resnet(Simple_Res_Block,[3,4,6,3],num_class=10,expansion_=False)
def Resnet50():
return Resnet(Residual_Block,[3,4,6,3],expansion_=True)# 也叫50层resnet，这个网络有16个模块，每个模块有三层卷积，最后还剩下初始的卷积和最后的全连接层，总共50层
def Resnet101():
return Resnet(Residual_Block,[3,4,23,3],expansion_=True)
def Resnet152():
return Resnet(Residual_Block,[3,8,36,3],expansion_=True)
其中包括了ResNet18,34,50,101,152。
对CIFAR-10进行分类?
# 基于cifar10或cifar100的训练
import torch
import os
import time
import torchvision
import tqdm
import numpy as np
from torch.utils.data import Dataset,DataLoader
from ResNet import Resnet18,Resnet34,Resnet50,Resnet101,Resnet152
from visualizer import Vis
class opt():
model_name = 'Resnet18'
save_path = 'checkpoints'
save_name = 'lastest_param.pth'
device = 'cuda'
batch_size = 128
learning_rate = 0.001
epoch = 60
state_file = 'checkpoints/result/lastest_param.pth'
load_f = True
classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
train_transform = torchvision.transforms.Compose([
torchvision.transforms.RandomCrop(32,padding=4),
torchvision.transforms.RandomHorizontalFlip(p=0.5),
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
])
test_transform = torchvision.transforms.Compose([
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
])
def load_save(model,load_f = False):
if load_f:
state = torch.load(opt.state_file)
model.load_state_dict(state)
return model
else:
return model
# model
if opt.model_name == "Resnet18":
model = Resnet18()
model.to(opt.device)
elif opt.model_name == "Resnet34":
model = Resnet34()
model.to(opt.device)
elif opt.model_name == "Resnet50":
model = Resnet50()
model.to(opt.device)
load_save(model,opt.load_f)
# dataset
train_dataset = torchvision.datasets.CIFAR10(
root = 'data',
train = True,
transform = opt.train_transform,
download=True
)
test_dataset = torchvision.datasets.CIFAR10(
root = 'data',
train = False,
transform = opt.test_transform,
download=True
)
# dataloader
train_loader = DataLoader(
train_dataset,
batch_size=opt.batch_size,
shuffle=True,
num_workers=6
)
test_loader = DataLoader(
test_dataset,
batch_size=100,
shuffle=False,
num_workers=6
)
# loss
loss_fn = torch.nn.CrossEntropyLoss()# 交叉熵
# 优化器
optim = torch.optim.SGD(model.parameters(),lr=opt.learning_rate,momentum=0.9,weight_decay=5e-4)# 对权重做衰减，也就是给损失函数加一个l2正则项，若模型没有较好收敛，则降低参数
flag = 0
def reverse_norm(img,mean=None,std=None):
imgs = []
for i in range(img.size(0)):
image = img[i].data.cpu().numpy().transpose(1, 2, 0)
if (mean is not None) and (std is not None):
image = (image * std + mean) * 255
else: # 如果只是经过了ToTensor()
image = image * 255
imgs.append(image.transpose(2,0,1))
return np.stack(imgs)
for epoch in range(opt.epoch):
now = time.time()
print('---epoch{}---'.format(epoch))
model.train()
loss_epoch = 0
true_pre_epoch = 0
correct = 0
for i,(img,label) in enumerate(tqdm.tqdm(train_loader)):
img,label = img.to(opt.device),label.to(opt.device)
output = model(img)
loss = loss_fn(output,label)
loss.backward()
optim.step()
optim.zero_grad()
flag += 1
loss_epoch += loss.data
pre = torch.argmax(output, dim=1)
num_true = (pre == label).sum()
true_pre_epoch += num_true
correct += label.shape[0]
if (i+1)%100 == 0:
print('epoch {} iter {} loss : {}'.format(epoch,i+1,loss_epoch/(i+1)))
if (i+1)%200 == 0:
acc = true_pre_epoch/correct
print('epoch {} iter {} train_acc : {}'.format(epoch,i+1,acc))
imgs = reverse_norm(img,mean=(0.4914, 0.4822, 0.4465),std=(0.2023, 0.1994, 0.2010))
# 可视化
vis = Vis()
vis.linee(Y=loss_epoch/(i+1),X=flag,win='loss')
vis.linee(Y=acc,X=flag,win='acc')
vis.Image(imgs,pre,opt.classes)
# save
model_path = os.path.join(opt.save_path,opt.save_name)
torch.save(model.state_dict(),model_path)
# test
model.eval()
num = 0
labels = 0
for img ,label in test_loader:
img, label = img.to(opt.device), label.to(opt.device)
output = model(img)
num += (torch.argmax(output,dim=1).data == label.data).sum()
labels += label.shape[0]
fin = time.time()
print('epoch {} test_acc : {} 运行一个epoch花费时间：{}s'.format(epoch,num/labels,fin-now))结果
因为CIFAR-10的数据集较小也只是一个简单的10分类，图片才32*32的大小。所以我选择的是ResNet18去进行训练。在手动调整学习率之后，模型的测试精度能达到87%。我采用了三个学习率去训练，先用0.1训练了150个epoch，后面又分别用了0.01和0.001训练了60个epoch。训练时的loss大小和训练精度如下图，图像中每次值的突变代表我手动调整了学习率。
?测试精度
---epoch57---
25%|██▍ | 97/391 [00:03<00:08, 35.42it/s]epoch 57 iter 100 loss : 0.01788470149040222
50%|█████ | 197/391 [00:05<00:05, 35.00it/s]Setting up a new session...
epoch 57 iter 200 loss : 0.019015971571207047
epoch 57 iter 200 train_acc : 0.9937499761581421
77%|███████▋ | 301/391 [00:09<00:02, 32.77it/s]epoch 57 iter 300 loss : 0.01771947182714939
100%|██████████| 391/391 [00:11<00:00, 32.87it/s]
epoch 57 test_acc : 0.8694999814033508 运行一个epoch花费时间：12.92395305633545s
---epoch58---
25%|██▍ | 97/391 [00:03<00:08, 33.84it/s]epoch 58 iter 100 loss : 0.01748574711382389
50%|█████ | 197/391 [00:06<00:06, 32.05it/s]Setting up a new session...
epoch 58 iter 200 loss : 0.016185222193598747
epoch 58 iter 200 train_acc : 0.9952343702316284
77%|███████▋ | 301/391 [00:09<00:02, 35.15it/s]epoch 58 iter 300 loss : 0.015332281589508057
100%|██████████| 391/391 [00:11<00:00, 33.29it/s]
epoch 58 test_acc : 0.8686999678611755 运行一个epoch花费时间：12.811056137084961s
---epoch59---
26%|██▌ | 101/391 [00:03<00:08, 35.97it/s]epoch 59 iter 100 loss : 0.01672389917075634
50%|█████ | 197/391 [00:05<00:05, 32.87it/s]Setting up a new session...
epoch 59 iter 200 loss : 0.0159761980175972
epoch 59 iter 200 train_acc : 0.9956249594688416
76%|███████▌ | 297/391 [00:08<00:02, 35.49it/s]epoch 59 iter 300 loss : 0.016513127833604813
100%|██████████| 391/391 [00:11<00:00, 33.80it/s]
epoch 59 test_acc : 0.8678999543190002 运行一个epoch花费时间：12.58652377128601s
进程已结束，退出代码为 0
调参总结
1.给SGD加一个权重衰减，要不然会过拟合导致训练精度很高，测试精度很低。 2.再加一个momentum,并将值设为0.9 3.将权重衰减的参数调为5e-4 4.将batch_size调为128，一开始设置的64不足以使模型收敛的很好 5.训练时无法收敛的很好时，可以多加一些数据增强 6.为了提高训练精度，采用手动调学习率的方法。100个epoch之后，将学习率改为1e-3在训练60个epoch
这一部分的调参具体参考了Pytorch实战2：ResNet-18实现Cifar-10图像分类（测试集分类准确率95.170%）_sunqiande88的博客-CSDN博客
我相信针对该经典模型还有更好的trick或者调参来提高测试精度，如果你有更好的精度还请不吝惜你的方法在评论区留言告诉我，谢谢！