1. 前言
本文使用飞桨（PaddlePaddle）复现卷积神经网络GoogLeNet。
本人全部文章请参见：博客文章导航目录
本文归属于：经典CNN复现系列
前文：VGGNet
2. GoogLeNet模型结构
GoogLeNet是2014年ImageNet比赛的冠军，它的主要特点是网络不仅有深度，还在横向上具有“宽度”。GoogLeNet提出了一种被称为Inception的模块，该模块具有不同尺寸的卷积核，可以使用较大的卷积核提取空间分布范围更广的图像信息的特征，使用较小的卷积核提取空间分布范围较小的图像信息的特征。
Google的研究人员为了向LeNet致敬，特地将模型命名为GoogLeNet。GoogLeNet也被称为Inception V1，其中Inception一词来源于电影《盗梦空间》(Inception)，Inception模块结构如下图所示：
图一(a)是Inception模块的设计思想，使用3个不同大小的卷积核对输入图片进行卷积操作，并附加最大池化，将这4个操作的输出沿着通道这一维度进行拼接，构成的输出特征图将会包含经过不同大小的卷积核提取出来的特征，从而达到捕捉不同尺度信息的效果。
Inception模块采用多通路(multi-path)的设计形式，每个支路使用不同大小的卷积核，最终输出特征图的通道数是每个支路输出通道数的总和，这将会导致输出通道数变得很大，尤其是使用多个Inception模块串联操作的时候，模型参数量会变得非常大。为了减小参数量，Inception模块具体实现时使用了图一(b)中的设计方式，在每个3x3和5x5的卷积层之前，增加1x1的卷积层来控制输出通道数。同时在最大池化层后面增加1x1卷积层减小输出通道数。
GoogLeNet模型结构如图二所示，在主体卷积部分中使用了5个模块（block），每个模块之间使用步长为2的3×3最大池化层来减小输出高宽。
- 第一模块使用一个64通道的7×7卷积层；
- 第二模块使用2个卷积层：首先是64通道的1×1卷积层，然后是将通道增大3倍的3×3卷积层；
- 第三模块串联2个Inception块；
- 第四模块串联了5个Inception块；
- 第五模块串联了2 个Inception块；
- 第五模块的后面紧跟输出层，使用全局平均池化层来将每个通道的高和宽变成1，最后接上一个输出个数为标签类别数的全连接层。
GoogLeNet是一个相对较深较复杂的网络，为了缓解当网络较深时存在的梯度消失现象，GoogLeNet添加了图二中所示的softmax1和softmax2两个辅助分类器（Auxililary Classifier），训练时将三个分类器的损失函数进行加权求和，辅助训练网络浅层模块的参数。辅助分类器的结构信息如下：
- 步长为3的5x5平均池化；
- 1x1的128通道卷积；
- 1024个节点的全连接层 + 概率为0.7的Dropout；
- 节点数等于输出类别数的全连接输出层。
由于在深度学习发展过程中发现LRN（Local Response Normalization）实际作用不大，现在主流深度神经网络已经不再使用LRN层了，PaddlePaddle框架中也没有集成LRN层。因此本文说明GoogLeNet时忽略了其中的LRN层，下面的模型复现也没有在代码里实现LRN层。想了解LRN层结构及实现，请参考我的另一篇博客AlexNet。
3. GoogLeNet模型复现
使用飞桨（PaddlePaddle）复现GoogLeNet，首先定义继承自paddle.nn.Layer的Inception模块，具体代码如下所示：
# -*- coding: utf-8 -*-
# @Time : 2021/8/8 21:58
# @Author : He Ruizhi
# @File : googlenet.py
# @Software: PyCharm
import paddle
class Inception(paddle.nn.Layer):
"""
定义Inception模块
input_channels：上一层输入特征图通道数
p1_channels：第一条支路1x1卷积的输出通道数，type > int
p2_channels：第二条支路的输出通道数，type > tuple|list，p2_channels[0]：1x1卷积输出通道数，p2_channels[1]：3x3卷积输出通道数
p3_channels：第三条支路的输出通道数，type > tuple|list，p3_channels[0]：1x1卷积输出通道数，p3_channels[1]：5x5卷积输出通道数
p4_channels：第四条支路1x1卷积的输出通道数，type > int
"""
def __init__(self, input_channels, p1_channels, p2_channels, p3_channels, p4_channels):
super(Inception, self).__init__()
# 依次初始化Inception模块各条支路
self.p1 = paddle.nn.Sequential(
paddle.nn.Conv2D(in_channels=input_channels, out_channels=p1_channels, kernel_size=1, stride=1),
paddle.nn.ReLU()
)
self.p2 = paddle.nn.Sequential(
paddle.nn.Conv2D(in_channels=input_channels, out_channels=p2_channels[0], kernel_size=1, stride=1),
paddle.nn.ReLU(),
paddle.nn.Conv2D(in_channels=p2_channels[0], out_channels=p2_channels[1], kernel_size=3, padding=1, stride=1),
paddle.nn.ReLU()
)
self.p3 = paddle.nn.Sequential(
paddle.nn.Conv2D(in_channels=input_channels, out_channels=p3_channels[0], kernel_size=1, stride=1),
paddle.nn.ReLU(),
paddle.nn.Conv2D(in_channels=p3_channels[0], out_channels=p3_channels[1], kernel_size=5, padding=2, stride=1),
paddle.nn.ReLU()
)
self.p4 = paddle.nn.Sequential(
paddle.nn.MaxPool2D(kernel_size=3, stride=1, padding=1),
paddle.nn.Conv2D(in_channels=input_channels, out_channels=p4_channels, kernel_size=1, stride=1),
paddle.nn.ReLU()
)
def forward(self, x):
# 分别计算各条支路的输出
p1 = self.p1(x)
p2 = self.p2(x)
p3 = self.p3(x)
p4 = self.p4(x)
# 将各条支路的输出特征图拼接，作为最终的输出结果，axis=1因为第0维是batch_size
return paddle.concat([p1, p2, p3, p4], axis=1)
设置input_channels=192、p1_channels=64、p2_channels=(96, 128)、p3_channels=(16, 32)、p4_channels=32，实例化Inception对象，并使用paddle.summary查看Inception结构：
inception = Inception(192, 64, (96, 128), (16, 32), 32)
paddle.summary(inception, input_size=(None, 192, 224, 224))
打印Inception结构信息如下：
---------------------------------------------------------------------------
Layer (type) Input Shape Output Shape Param #
===========================================================================
Conv2D-1 [[1, 192, 224, 224]] [1, 64, 224, 224] 12,352
ReLU-1 [[1, 64, 224, 224]] [1, 64, 224, 224] 0
Conv2D-2 [[1, 192, 224, 224]] [1, 96, 224, 224] 18,528
ReLU-2 [[1, 96, 224, 224]] [1, 96, 224, 224] 0
Conv2D-3 [[1, 96, 224, 224]] [1, 128, 224, 224] 110,720
ReLU-3 [[1, 128, 224, 224]] [1, 128, 224, 224] 0
Conv2D-4 [[1, 192, 224, 224]] [1, 16, 224, 224] 3,088
ReLU-4 [[1, 16, 224, 224]] [1, 16, 224, 224] 0
Conv2D-5 [[1, 16, 224, 224]] [1, 32, 224, 224] 12,832
ReLU-5 [[1, 32, 224, 224]] [1, 32, 224, 224] 0
MaxPool2D-1 [[1, 192, 224, 224]] [1, 192, 224, 224] 0
Conv2D-6 [[1, 192, 224, 224]] [1, 32, 224, 224] 6,176
ReLU-6 [[1, 32, 224, 224]] [1, 32, 224, 224] 0
===========================================================================
Total params: 163,696
Trainable params: 163,696
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 36.75
Forward/backward pass size (MB): 355.25
Params size (MB): 0.62
Estimated Total Size (MB): 392.62
---------------------------------------------------------------------------
定义继承自paddle.nn.Layer的GoogLeNet类，在__init__方法中定义卷积等模块，在forward函数中实现网络前向计算流程。具体代码如下：
class GoogLeNet(paddle.nn.Layer):
"""
搭建GoogLeNet
"""
def __init__(self, num_classes=1000):
super(GoogLeNet, self).__init__()
# 第一个模块：7x7的64通道padding为3步长为2的卷积 + 步长为2的3x3池化
self.conv_block1 = paddle.nn.Sequential(
paddle.nn.Conv2D(in_channels=3, out_channels=64, kernel_size=7, stride=2, padding=3),
paddle.nn.ReLU(),
paddle.nn.MaxPool2D(kernel_size=3, stride=2, padding=1)
)
# 第二个模块：1x1的64通道卷积 + 3x3的192通道卷积 + 步长为2的3x3池化
self.conv_block2 = paddle.nn.Sequential(
paddle.nn.Conv2D(in_channels=64, out_channels=64, kernel_size=1, stride=1),
paddle.nn.ReLU(),
paddle.nn.Conv2D(in_channels=64, out_channels=192, kernel_size=3, stride=1, padding=1),
paddle.nn.ReLU(),
paddle.nn.MaxPool2D(kernel_size=3, stride=2, padding=1)
)
# 第三个模块：2个Inception模块 + 步长为2的3x3池化
self.inception_block3 = paddle.nn.Sequential(
Inception(192, 64, (96, 128), (16, 32), 32),
Inception(256, 128, (128, 192), (32, 96), 64),
paddle.nn.MaxPool2D(kernel_size=3, stride=2, padding=1)
)
# 由于GoogLeNet存在三个输出，因此以下模块不使用paddle.nn.Sequential打包
# 第四个模块：4个Inception模块 + 步长为2的3x3池化
self.inception4a = Inception(480, 192, (96, 208), (16, 48), 64)
self.inception4b = Inception(512, 160, (112, 224), (24, 64), 64)
self.inception4c = Inception(512, 128, (128, 256), (24, 64), 64)
self.inception4d = Inception(512, 112, (144, 288), (32, 64), 64)
self.inception4e = Inception(528, 256, (160, 320), (32, 128), 128)
self.max_pool4 = paddle.nn.MaxPool2D(kernel_size=3, stride=2, padding=1)
# 第五个模块：2个Inception模块
self.inception_block5 = paddle.nn.Sequential(
Inception(832, 256, (160, 320), (32, 128), 128),
Inception(832, 384, (192, 384), (48, 128), 128),
)
# 主干输出模块
self.out_main = paddle.nn.Sequential(
paddle.nn.AvgPool2D(kernel_size=7),
paddle.nn.Flatten(),
paddle.nn.Dropout(0.4),
paddle.nn.Linear(in_features=1024, out_features=num_classes)
)
# 分支输出1
self.out1 = paddle.nn.Sequential(
paddle.nn.AvgPool2D(kernel_size=5, stride=3),
paddle.nn.Conv2D(in_channels=512, out_channels=128, kernel_size=1),
paddle.nn.Flatten(),
paddle.nn.Linear(in_features=2048, out_features=1024),
paddle.nn.ReLU(),
paddle.nn.Dropout(0.7),
paddle.nn.Linear(in_features=1024, out_features=num_classes)
)
# 分支输出2
self.out2 = paddle.nn.Sequential(
paddle.nn.AvgPool2D(kernel_size=5, stride=3),
paddle.nn.Conv2D(in_channels=528, out_channels=128, kernel_size=1),
paddle.nn.Flatten(),
paddle.nn.Linear(in_features=2048, out_features=1024),
paddle.nn.ReLU(),
paddle.nn.Dropout(0.7),
paddle.nn.Linear(in_features=1024, out_features=num_classes)
)
def forward(self, x):
x = self.conv_block1(x)
x = self.conv_block2(x)
x = self.inception_block3(x)
# 第四个模块须记录Inception4a和Inception4d的输出
ince4a = self.inception4a(x)
x = self.inception4b(ince4a)
x = self.inception4c(x)
ince4d = self.inception4d(x)
x = self.inception4e(ince4d)
x = self.max_pool4(x)
x = self.inception_block5(x)
# 主干输出
out = self.out_main(x)
# 分支输出1
out1 = self.out1(ince4a)
# 分支输出2
out2 = self.out2(ince4d)
return out, out1, out2
实例化GoogLeNet模型对象，并使用paddle.summary查看GoogLeNet结构信息：
if __name__ == '__main__':
# inception = Inception(192, 64, (96, 128), (16, 32), 32)
# paddle.summary(inception, input_size=(None, 192, 224, 224))
googlenet = GoogLeNet()
paddle.summary(googlenet, input_size=(None, 3, 224, 224))
打印GoogLeNet模型结构信息如下：
---------------------------------------------------------------------------
Layer (type) Input Shape Output Shape Param #
===========================================================================
Conv2D-1 [[1, 3, 224, 224]] [1, 64, 112, 112] 9,472
ReLU-1 [[1, 64, 112, 112]] [1, 64, 112, 112] 0
MaxPool2D-1 [[1, 64, 112, 112]] [1, 64, 56, 56] 0
Conv2D-2 [[1, 64, 56, 56]] [1, 64, 56, 56] 4,160
ReLU-2 [[1, 64, 56, 56]] [1, 64, 56, 56] 0
Conv2D-3 [[1, 64, 56, 56]] [1, 192, 56, 56] 110,784
ReLU-3 [[1, 192, 56, 56]] [1, 192, 56, 56] 0
MaxPool2D-2 [[1, 192, 56, 56]] [1, 192, 28, 28] 0
Conv2D-4 [[1, 192, 28, 28]] [1, 64, 28, 28] 12,352
ReLU-4 [[1, 64, 28, 28]] [1, 64, 28, 28] 0
Conv2D-5 [[1, 192, 28, 28]] [1, 96, 28, 28] 18,528
ReLU-5 [[1, 96, 28, 28]] [1, 96, 28, 28] 0
Conv2D-6 [[1, 96, 28, 28]] [1, 128, 28, 28] 110,720
ReLU-6 [[1, 128, 28, 28]] [1, 128, 28, 28] 0
Conv2D-7 [[1, 192, 28, 28]] [1, 16, 28, 28] 3,088
ReLU-7 [[1, 16, 28, 28]] [1, 16, 28, 28] 0
Conv2D-8 [[1, 16, 28, 28]] [1, 32, 28, 28] 12,832
ReLU-8 [[1, 32, 28, 28]] [1, 32, 28, 28] 0
MaxPool2D-3 [[1, 192, 28, 28]] [1, 192, 28, 28] 0
Conv2D-9 [[1, 192, 28, 28]] [1, 32, 28, 28] 6,176
ReLU-9 [[1, 32, 28, 28]] [1, 32, 28, 28] 0
Inception-1 [[1, 192, 28, 28]] [1, 256, 28, 28] 0
Conv2D-10 [[1, 256, 28, 28]] [1, 128, 28, 28] 32,896
ReLU-10 [[1, 128, 28, 28]] [1, 128, 28, 28] 0
Conv2D-11 [[1, 256, 28, 28]] [1, 128, 28, 28] 32,896
ReLU-11 [[1, 128, 28, 28]] [1, 128, 28, 28] 0
Conv2D-12 [[1, 128, 28, 28]] [1, 192, 28, 28] 221,376
ReLU-12 [[1, 192, 28, 28]] [1, 192, 28, 28] 0
Conv2D-13 [[1, 256, 28, 28]] [1, 32, 28, 28] 8,224
ReLU-13 [[1, 32, 28, 28]] [1, 32, 28, 28] 0
Conv2D-14 [[1, 32, 28, 28]] [1, 96, 28, 28] 76,896
ReLU-14 [[1, 96, 28, 28]] [1, 96, 28, 28] 0
MaxPool2D-4 [[1, 256, 28, 28]] [1, 256, 28, 28] 0
Conv2D-15 [[1, 256, 28, 28]] [1, 64, 28, 28] 16,448
ReLU-15 [[1, 64, 28, 28]] [1, 64, 28, 28] 0
Inception-2 [[1, 256, 28, 28]] [1, 480, 28, 28] 0
MaxPool2D-5 [[1, 480, 28, 28]] [1, 480, 14, 14] 0
Conv2D-16 [[1, 480, 14, 14]] [1, 192, 14, 14] 92,352
ReLU-16 [[1, 192, 14, 14]] [1, 192, 14, 14] 0
Conv2D-17 [[1, 480, 14, 14]] [1, 96, 14, 14] 46,176
ReLU-17 [[1, 96, 14, 14]] [1, 96, 14, 14] 0
Conv2D-18 [[1, 96, 14, 14]] [1, 208, 14, 14] 179,920
ReLU-18 [[1, 208, 14, 14]] [1, 208, 14, 14] 0
Conv2D-19 [[1, 480, 14, 14]] [1, 16, 14, 14] 7,696
ReLU-19 [[1, 16, 14, 14]] [1, 16, 14, 14] 0
Conv2D-20 [[1, 16, 14, 14]] [1, 48, 14, 14] 19,248
ReLU-20 [[1, 48, 14, 14]] [1, 48, 14, 14] 0
MaxPool2D-6 [[1, 480, 14, 14]] [1, 480, 14, 14] 0
Conv2D-21 [[1, 480, 14, 14]] [1, 64, 14, 14] 30,784
ReLU-21 [[1, 64, 14, 14]] [1, 64, 14, 14] 0
Inception-3 [[1, 480, 14, 14]] [1, 512, 14, 14] 0
Conv2D-22 [[1, 512, 14, 14]] [1, 160, 14, 14] 82,080
ReLU-22 [[1, 160, 14, 14]] [1, 160, 14, 14] 0
Conv2D-23 [[1, 512, 14, 14]] [1, 112, 14, 14] 57,456
ReLU-23 [[1, 112, 14, 14]] [1, 112, 14, 14] 0
Conv2D-24 [[1, 112, 14, 14]] [1, 224, 14, 14] 226,016
ReLU-24 [[1, 224, 14, 14]] [1, 224, 14, 14] 0
Conv2D-25 [[1, 512, 14, 14]] [1, 24, 14, 14] 12,312
ReLU-25 [[1, 24, 14, 14]] [1, 24, 14, 14] 0
Conv2D-26 [[1, 24, 14, 14]] [1, 64, 14, 14] 38,464
ReLU-26 [[1, 64, 14, 14]] [1, 64, 14, 14] 0
MaxPool2D-7 [[1, 512, 14, 14]] [1, 512, 14, 14] 0
Conv2D-27 [[1, 512, 14, 14]] [1, 64, 14, 14] 32,832
ReLU-27 [[1, 64, 14, 14]] [1, 64, 14, 14] 0
Inception-4 [[1, 512, 14, 14]] [1, 512, 14, 14] 0
Conv2D-28 [[1, 512, 14, 14]] [1, 128, 14, 14] 65,664
ReLU-28 [[1, 128, 14, 14]] [1, 128, 14, 14] 0
Conv2D-29 [[1, 512, 14, 14]] [1, 128, 14, 14] 65,664
ReLU-29 [[1, 128, 14, 14]] [1, 128, 14, 14] 0
Conv2D-30 [[1, 128, 14, 14]] [1, 256, 14, 14] 295,168
ReLU-30 [[1, 256, 14, 14]] [1, 256, 14, 14] 0
Conv2D-31 [[1, 512, 14, 14]] [1, 24, 14, 14] 12,312
ReLU-31 [[1, 24, 14, 14]] [1, 24, 14, 14] 0
Conv2D-32 [[1, 24, 14, 14]] [1, 64, 14, 14] 38,464
ReLU-32 [[1, 64, 14, 14]] [1, 64, 14, 14] 0
MaxPool2D-8 [[1, 512, 14, 14]] [1, 512, 14, 14] 0
Conv2D-33 [[1, 512, 14, 14]] [1, 64, 14, 14] 32,832
ReLU-33 [[1, 64, 14, 14]] [1, 64, 14, 14] 0
Inception-5 [[1, 512, 14, 14]] [1, 512, 14, 14] 0
Conv2D-34 [[1, 512, 14, 14]] [1, 112, 14, 14] 57,456
ReLU-34 [[1, 112, 14, 14]] [1, 112, 14, 14] 0
Conv2D-35 [[1, 512, 14, 14]] [1, 144, 14, 14] 73,872
ReLU-35 [[1, 144, 14, 14]] [1, 144, 14, 14] 0
Conv2D-36 [[1, 144, 14, 14]] [1, 288, 14, 14] 373,536
ReLU-36 [[1, 288, 14, 14]] [1, 288, 14, 14] 0
Conv2D-37 [[1, 512, 14, 14]] [1, 32, 14, 14] 16,416
ReLU-37 [[1, 32, 14, 14]] [1, 32, 14, 14] 0
Conv2D-38 [[1, 32, 14, 14]] [1, 64, 14, 14] 51,264
ReLU-38 [[1, 64, 14, 14]] [1, 64, 14, 14] 0
MaxPool2D-9 [[1, 512, 14, 14]] [1, 512, 14, 14] 0
Conv2D-39 [[1, 512, 14, 14]] [1, 64, 14, 14] 32,832
ReLU-39 [[1, 64, 14, 14]] [1, 64, 14, 14] 0
Inception-6 [[1, 512, 14, 14]] [1, 528, 14, 14] 0
Conv2D-40 [[1, 528, 14, 14]] [1, 256, 14, 14] 135,424
ReLU-40 [[1, 256, 14, 14]] [1, 256, 14, 14] 0
Conv2D-41 [[1, 528, 14, 14]] [1, 160, 14, 14] 84,640
ReLU-41 [[1, 160, 14, 14]] [1, 160, 14, 14] 0
Conv2D-42 [[1, 160, 14, 14]] [1, 320, 14, 14] 461,120
ReLU-42 [[1, 320, 14, 14]] [1, 320, 14, 14] 0
Conv2D-43 [[1, 528, 14, 14]] [1, 32, 14, 14] 16,928
ReLU-43 [[1, 32, 14, 14]] [1, 32, 14, 14] 0
Conv2D-44 [[1, 32, 14, 14]] [1, 128, 14, 14] 102,528
ReLU-44 [[1, 128, 14, 14]] [1, 128, 14, 14] 0
MaxPool2D-10 [[1, 528, 14, 14]] [1, 528, 14, 14] 0
Conv2D-45 [[1, 528, 14, 14]] [1, 128, 14, 14] 67,712
ReLU-45 [[1, 128, 14, 14]] [1, 128, 14, 14] 0
Inception-7 [[1, 528, 14, 14]] [1, 832, 14, 14] 0
MaxPool2D-11 [[1, 832, 14, 14]] [1, 832, 7, 7] 0
Conv2D-46 [[1, 832, 7, 7]] [1, 256, 7, 7] 213,248
ReLU-46 [[1, 256, 7, 7]] [1, 256, 7, 7] 0
Conv2D-47 [[1, 832, 7, 7]] [1, 160, 7, 7] 133,280
ReLU-47 [[1, 160, 7, 7]] [1, 160, 7, 7] 0
Conv2D-48 [[1, 160, 7, 7]] [1, 320, 7, 7] 461,120
ReLU-48 [[1, 320, 7, 7]] [1, 320, 7, 7] 0
Conv2D-49 [[1, 832, 7, 7]] [1, 32, 7, 7] 26,656
ReLU-49 [[1, 32, 7, 7]] [1, 32, 7, 7] 0
Conv2D-50 [[1, 32, 7, 7]] [1, 128, 7, 7] 102,528
ReLU-50 [[1, 128, 7, 7]] [1, 128, 7, 7] 0
MaxPool2D-12 [[1, 832, 7, 7]] [1, 832, 7, 7] 0
Conv2D-51 [[1, 832, 7, 7]] [1, 128, 7, 7] 106,624
ReLU-51 [[1, 128, 7, 7]] [1, 128, 7, 7] 0
Inception-8 [[1, 832, 7, 7]] [1, 832, 7, 7] 0
Conv2D-52 [[1, 832, 7, 7]] [1, 384, 7, 7] 319,872
ReLU-52 [[1, 384, 7, 7]] [1, 384, 7, 7] 0
Conv2D-53 [[1, 832, 7, 7]] [1, 192, 7, 7] 159,936
ReLU-53 [[1, 192, 7, 7]] [1, 192, 7, 7] 0
Conv2D-54 [[1, 192, 7, 7]] [1, 384, 7, 7] 663,936
ReLU-54 [[1, 384, 7, 7]] [1, 384, 7, 7] 0
Conv2D-55 [[1, 832, 7, 7]] [1, 48, 7, 7] 39,984
ReLU-55 [[1, 48, 7, 7]] [1, 48, 7, 7] 0
Conv2D-56 [[1, 48, 7, 7]] [1, 128, 7, 7] 153,728
ReLU-56 [[1, 128, 7, 7]] [1, 128, 7, 7] 0
MaxPool2D-13 [[1, 832, 7, 7]] [1, 832, 7, 7] 0
Conv2D-57 [[1, 832, 7, 7]] [1, 128, 7, 7] 106,624
ReLU-57 [[1, 128, 7, 7]] [1, 128, 7, 7] 0
Inception-9 [[1, 832, 7, 7]] [1, 1024, 7, 7] 0
AvgPool2D-1 [[1, 1024, 7, 7]] [1, 1024, 1, 1] 0
Flatten-1 [[1, 1024, 1, 1]] [1, 1024] 0
Dropout-1 [[1, 1024]] [1, 1024] 0
Linear-1 [[1, 1024]] [1, 1000] 1,025,000
AvgPool2D-2 [[1, 512, 14, 14]] [1, 512, 4, 4] 0
Conv2D-58 [[1, 512, 4, 4]] [1, 128, 4, 4] 65,664
Flatten-2 [[1, 128, 4, 4]] [1, 2048] 0
Linear-2 [[1, 2048]] [1, 1024] 2,098,176
ReLU-58 [[1, 1024]] [1, 1024] 0
Dropout-2 [[1, 1024]] [1, 1024] 0
Linear-3 [[1, 1024]] [1, 1000] 1,025,000
AvgPool2D-3 [[1, 528, 14, 14]] [1, 528, 4, 4] 0
Conv2D-59 [[1, 528, 4, 4]] [1, 128, 4, 4] 67,712
Flatten-3 [[1, 128, 4, 4]] [1, 2048] 0
Linear-4 [[1, 2048]] [1, 1024] 2,098,176
ReLU-59 [[1, 1024]] [1, 1024] 0
Dropout-3 [[1, 1024]] [1, 1024] 0
Linear-5 [[1, 1024]] [1, 1000] 1,025,000
===========================================================================
Total params: 13,378,280
Trainable params: 13,378,280
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 0.57
Forward/backward pass size (MB): 69.75
Params size (MB): 51.03
Estimated Total Size (MB): 121.36
---------------------------------------------------------------------------