|
流程：首先把model中的每一层网络激活hook，完成正向传播的框架搭建，搭建的过程中顺带将每一层的信息进行统计。然后输出input，完成model的正向传播。
x是input
type(x)
Out: list
type(x[0])
Out: torch.Tensor
type(*x)
Out: torch.Tensor
x[0].size()
Out: torch.Size([2, 3, 418, 418])
(*x).size()
error:...
x
x
Out:
[tensor([[[[0.3102, 0.8596, 0.1304, ..., 0.6594, 0.8860, 0.5789],
[0.8040, 0.2697, 0.2976, ..., 0.5797, 0.4450, 0.5615],
[0.2392, 0.5074, 0.0021, ..., 0.6517, 0.7355, 0.6797],
...,
[0.3715, 0.2288, 0.7578, ..., 0.0383, 0.3627, 0.5528],
[0.1345, 0.3320, 0.5228, ..., 0.4717, 0.4596, 0.7026],
[0.1294, 0.0535, 0.3622, ..., 0.0159, 0.0584, 0.2387]],
[[0.7831, 0.7086, 0.5249, ..., 0.9599, 0.1168, 0.2670],
[0.5323, 0.9834, 0.9716, ..., 0.2078, 0.9068, 0.4957],
[0.0579, 0.6982, 0.6790, ..., 0.3966, 0.2998, 0.8798],
...,
[0.6805, 0.0573, 0.6803, ..., 0.1519, 0.4711, 0.7284],
[0.9284, 0.0585, 0.2972, ..., 0.7272, 0.9505, 0.2277],
[0.4118, 0.6793, 0.3267, ..., 0.3202, 0.8033, 0.5302]],
[[0.1242, 0.6022, 0.3829, ..., 0.9428, 0.2495, 0.6285],
[0.3000, 0.0764, 0.7873, ..., 0.9511, 0.7078, 0.0534],
[0.5872, 0.4522, 0.9887, ..., 0.1104, 0.2189, 0.2736],
...,
[0.1302, 0.7966, 0.5835, ..., 0.7391, 0.1846, 0.4527],
[0.5570, 0.7166, 0.3877, ..., 0.1735, 0.9202, 0.5621],
[0.5085, 0.6321, 0.1718, ..., 0.0255, 0.3761, 0.8534]]],
[[[0.3739, 0.1188, 0.4226, ..., 0.6244, 0.2109, 0.9015],
[0.1861, 0.7112, 0.1474, ..., 0.7150, 0.9982, 0.2875],
[0.4693, 0.2805, 0.1399, ..., 0.0569, 0.6584, 0.2678],
...,
[0.1254, 0.3277, 0.9279, ..., 0.4351, 0.6777, 0.7394],
[0.9116, 0.8686, 0.2146, ..., 0.3590, 0.0505, 0.5434],
[0.4119, 0.8424, 0.3614, ..., 0.3036, 0.2965, 0.1025]],
[[0.1698, 0.3996, 0.4214, ..., 0.4053, 0.1676, 0.8579],
[0.2363, 0.2986, 0.4431, ..., 0.0400, 0.9726, 0.8652],
[0.3392, 0.2502, 0.0449, ..., 0.5490, 0.1409, 0.7616],
...,
[0.8507, 0.5658, 0.5749, ..., 0.5579, 0.7436, 0.0212],
[0.8636, 0.6931, 0.7503, ..., 0.2917, 0.9506, 0.2678],
[0.4792, 0.2392, 0.3151, ..., 0.8005, 0.4148, 0.6095]],
[[0.2562, 0.2324, 0.1086, ..., 0.6738, 0.6497, 0.9233],
[0.4929, 0.7130, 0.7814, ..., 0.4813, 0.5589, 0.3298],
[0.3308, 0.5899, 0.0266, ..., 0.5381, 0.2375, 0.5328],
...,
[0.8276, 0.9921, 0.5365, ..., 0.1358, 0.3378, 0.5708],
[0.5240, 0.9950, 0.0833, ..., 0.7758, 0.6101, 0.0930],
[0.9067, 0.3900, 0.4336, ..., 0.4563, 0.3291, 0.6772]]]],
device='cuda:0')]
torchsummary.py
import torch
import torch.nn as nn
from torch.autograd import Variable
from collections import OrderedDict
import numpy as np
def summary(model, input_size, batch_size=-1, device="cuda"):
def register_hook(module):
def hook(module, input, output):
class_name = str(module.__class__).split(".")[-1].split("'")[0]
module_idx = len(summary)
m_key = "%s-%i" % (class_name, module_idx + 1)
summary[m_key] = OrderedDict()
summary[m_key]["input_shape"] = list(input[0].size())
summary[m_key]["input_shape"][0] = batch_size
if isinstance(output, (list, tuple)):
summary[m_key]["output_shape"] = [
[-1] + list(o.size())[1:] for o in output
]
else:
summary[m_key]["output_shape"] = list(output.size())
summary[m_key]["output_shape"][0] = batch_size
params = 0
if hasattr(module, "weight") and hasattr(module.weight, "size"):
params += torch.prod(torch.LongTensor(list(module.weight.size())))
summary[m_key]["trainable"] = module.weight.requires_grad
if hasattr(module, "bias") and hasattr(module.bias, "size"):
params += torch.prod(torch.LongTensor(list(module.bias.size())))
summary[m_key]["nb_params"] = params
if (
not isinstance(module, nn.Sequential)
and not isinstance(module, nn.ModuleList)
and not (module == model)
):
hooks.append(module.register_forward_hook(hook))
device = device.lower()
assert device in [
"cuda",
"cpu",
], "Input device is not valid, please specify 'cuda' or 'cpu'"
if device == "cuda" and torch.cuda.is_available():
dtype = torch.cuda.FloatTensor
else:
dtype = torch.FloatTensor
# multiple inputs to the network
if isinstance(input_size, tuple):
input_size = [input_size]
# batch_size of 2 for batchnorm
x = [torch.rand(2, *in_size).type(dtype) for in_size in input_size]
# print(type(x[0]))
# create properties
summary = OrderedDict()
hooks = []
# register hook
model.apply(register_hook)
# make a forward pass
# print(x.shape)
model(*x)
# remove these hooks
for h in hooks:
h.remove()
print("----------------------------------------------------------------")
line_new = "{:>20} {:>25} {:>15}".format("Layer (type)", "Output Shape", "Param #")
print(line_new)
print("================================================================")
total_params = 0
total_output = 0
trainable_params = 0
for layer in summary:
# input_shape, output_shape, trainable, nb_params
line_new = "{:>20} {:>25} {:>15}".format(
layer,
str(summary[layer]["output_shape"]),
"{0:,}".format(summary[layer]["nb_params"]),
)
total_params += summary[layer]["nb_params"]
total_output += np.prod(summary[layer]["output_shape"])
if "trainable" in summary[layer]:
if summary[layer]["trainable"] == True:
trainable_params += summary[layer]["nb_params"]
print(line_new)
# assume 4 bytes/number (float on cuda).
total_input_size = abs(np.prod(input_size) * batch_size * 4. / (1024 ** 2.))
total_output_size = abs(2. * total_output * 4. / (1024 ** 2.)) # x2 for gradients
total_params_size = abs(total_params.numpy() * 4. / (1024 ** 2.))
total_size = total_params_size + total_output_size + total_input_size
print("================================================================")
print("Total params: {0:,}".format(total_params))
print("Trainable params: {0:,}".format(trainable_params))
print("Non-trainable params: {0:,}".format(total_params - trainable_params))
print("----------------------------------------------------------------")
print("Input size (MB): %0.2f" % total_input_size)
print("Forward/backward pass size (MB): %0.2f" % total_output_size)
print("Params size (MB): %0.2f" % total_params_size)
print("Estimated Total Size (MB): %0.2f" % total_size)
print("----------------------------------------------------------------")
# return summary
/home/wangbin/anaconda3/envs/deep_learning/bin/python3.7 /media/wangbin/F/深度学习_程序/yolo_practice/resnet_yolo.py
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Conv2d-1 [-1, 64, 209, 209] 9,408
BatchNorm2d-2 [-1, 64, 209, 209] 128
ReLU-3 [-1, 64, 209, 209] 0
MaxPool2d-4 [-1, 64, 105, 105] 0
Conv2d-5 [-1, 64, 105, 105] 4,096
BatchNorm2d-6 [-1, 64, 105, 105] 128
ReLU-7 [-1, 64, 105, 105] 0
Conv2d-8 [-1, 64, 105, 105] 36,864
BatchNorm2d-9 [-1, 64, 105, 105] 128
ReLU-10 [-1, 64, 105, 105] 0
Conv2d-11 [-1, 256, 105, 105] 16,384
BatchNorm2d-12 [-1, 256, 105, 105] 512
Conv2d-13 [-1, 256, 105, 105] 16,384
BatchNorm2d-14 [-1, 256, 105, 105] 512
ReLU-15 [-1, 256, 105, 105] 0
Bottleneck-16 [-1, 256, 105, 105] 0
Conv2d-17 [-1, 64, 105, 105] 16,384
BatchNorm2d-18 [-1, 64, 105, 105] 128
ReLU-19 [-1, 64, 105, 105] 0
Conv2d-20 [-1, 64, 105, 105] 36,864
BatchNorm2d-21 [-1, 64, 105, 105] 128
ReLU-22 [-1, 64, 105, 105] 0
Conv2d-23 [-1, 256, 105, 105] 16,384
BatchNorm2d-24 [-1, 256, 105, 105] 512
ReLU-25 [-1, 256, 105, 105] 0
Bottleneck-26 [-1, 256, 105, 105] 0
Conv2d-27 [-1, 64, 105, 105] 16,384
BatchNorm2d-28 [-1, 64, 105, 105] 128
ReLU-29 [-1, 64, 105, 105] 0
Conv2d-30 [-1, 64, 105, 105] 36,864
BatchNorm2d-31 [-1, 64, 105, 105] 128
ReLU-32 [-1, 64, 105, 105] 0
Conv2d-33 [-1, 256, 105, 105] 16,384
BatchNorm2d-34 [-1, 256, 105, 105] 512
ReLU-35 [-1, 256, 105, 105] 0
Bottleneck-36 [-1, 256, 105, 105] 0
Conv2d-37 [-1, 128, 105, 105] 32,768
BatchNorm2d-38 [-1, 128, 105, 105] 256
ReLU-39 [-1, 128, 105, 105] 0
Conv2d-40 [-1, 128, 53, 53] 147,456
BatchNorm2d-41 [-1, 128, 53, 53] 256
ReLU-42 [-1, 128, 53, 53] 0
Conv2d-43 [-1, 512, 53, 53] 65,536
BatchNorm2d-44 [-1, 512, 53, 53] 1,024
Conv2d-45 [-1, 512, 53, 53] 131,072
BatchNorm2d-46 [-1, 512, 53, 53] 1,024
ReLU-47 [-1, 512, 53, 53] 0
Bottleneck-48 [-1, 512, 53, 53] 0
Conv2d-49 [-1, 128, 53, 53] 65,536
BatchNorm2d-50 [-1, 128, 53, 53] 256
ReLU-51 [-1, 128, 53, 53] 0
Conv2d-52 [-1, 128, 53, 53] 147,456
BatchNorm2d-53 [-1, 128, 53, 53] 256
ReLU-54 [-1, 128, 53, 53] 0
Conv2d-55 [-1, 512, 53, 53] 65,536
BatchNorm2d-56 [-1, 512, 53, 53] 1,024
ReLU-57 [-1, 512, 53, 53] 0
Bottleneck-58 [-1, 512, 53, 53] 0
Conv2d-59 [-1, 128, 53, 53] 65,536
BatchNorm2d-60 [-1, 128, 53, 53] 256
ReLU-61 [-1, 128, 53, 53] 0
Conv2d-62 [-1, 128, 53, 53] 147,456
BatchNorm2d-63 [-1, 128, 53, 53] 256
ReLU-64 [-1, 128, 53, 53] 0
Conv2d-65 [-1, 512, 53, 53] 65,536
BatchNorm2d-66 [-1, 512, 53, 53] 1,024
ReLU-67 [-1, 512, 53, 53] 0
Bottleneck-68 [-1, 512, 53, 53] 0
Conv2d-69 [-1, 128, 53, 53] 65,536
BatchNorm2d-70 [-1, 128, 53, 53] 256
ReLU-71 [-1, 128, 53, 53] 0
Conv2d-72 [-1, 128, 53, 53] 147,456
BatchNorm2d-73 [-1, 128, 53, 53] 256
ReLU-74 [-1, 128, 53, 53] 0
Conv2d-75 [-1, 512, 53, 53] 65,536
BatchNorm2d-76 [-1, 512, 53, 53] 1,024
ReLU-77 [-1, 512, 53, 53] 0
Bottleneck-78 [-1, 512, 53, 53] 0
Conv2d-79 [-1, 256, 53, 53] 131,072
BatchNorm2d-80 [-1, 256, 53, 53] 512
ReLU-81 [-1, 256, 53, 53] 0
Conv2d-82 [-1, 256, 27, 27] 589,824
BatchNorm2d-83 [-1, 256, 27, 27] 512
ReLU-84 [-1, 256, 27, 27] 0
Conv2d-85 [-1, 1024, 27, 27] 262,144
BatchNorm2d-86 [-1, 1024, 27, 27] 2,048
Conv2d-87 [-1, 1024, 27, 27] 524,288
BatchNorm2d-88 [-1, 1024, 27, 27] 2,048
ReLU-89 [-1, 1024, 27, 27] 0
Bottleneck-90 [-1, 1024, 27, 27] 0
Conv2d-91 [-1, 256, 27, 27] 262,144
BatchNorm2d-92 [-1, 256, 27, 27] 512
ReLU-93 [-1, 256, 27, 27] 0
Conv2d-94 [-1, 256, 27, 27] 589,824
BatchNorm2d-95 [-1, 256, 27, 27] 512
ReLU-96 [-1, 256, 27, 27] 0
Conv2d-97 [-1, 1024, 27, 27] 262,144
BatchNorm2d-98 [-1, 1024, 27, 27] 2,048
ReLU-99 [-1, 1024, 27, 27] 0
Bottleneck-100 [-1, 1024, 27, 27] 0
Conv2d-101 [-1, 256, 27, 27] 262,144
BatchNorm2d-102 [-1, 256, 27, 27] 512
ReLU-103 [-1, 256, 27, 27] 0
Conv2d-104 [-1, 256, 27, 27] 589,824
BatchNorm2d-105 [-1, 256, 27, 27] 512
ReLU-106 [-1, 256, 27, 27] 0
Conv2d-107 [-1, 1024, 27, 27] 262,144
BatchNorm2d-108 [-1, 1024, 27, 27] 2,048
ReLU-109 [-1, 1024, 27, 27] 0
Bottleneck-110 [-1, 1024, 27, 27] 0
Conv2d-111 [-1, 256, 27, 27] 262,144
BatchNorm2d-112 [-1, 256, 27, 27] 512
ReLU-113 [-1, 256, 27, 27] 0
Conv2d-114 [-1, 256, 27, 27] 589,824
BatchNorm2d-115 [-1, 256, 27, 27] 512
ReLU-116 [-1, 256, 27, 27] 0
Conv2d-117 [-1, 1024, 27, 27] 262,144
BatchNorm2d-118 [-1, 1024, 27, 27] 2,048
ReLU-119 [-1, 1024, 27, 27] 0
Bottleneck-120 [-1, 1024, 27, 27] 0
Conv2d-121 [-1, 256, 27, 27] 262,144
BatchNorm2d-122 [-1, 256, 27, 27] 512
ReLU-123 [-1, 256, 27, 27] 0
Conv2d-124 [-1, 256, 27, 27] 589,824
BatchNorm2d-125 [-1, 256, 27, 27] 512
ReLU-126 [-1, 256, 27, 27] 0
Conv2d-127 [-1, 1024, 27, 27] 262,144
BatchNorm2d-128 [-1, 1024, 27, 27] 2,048
ReLU-129 [-1, 1024, 27, 27] 0
Bottleneck-130 [-1, 1024, 27, 27] 0
Conv2d-131 [-1, 256, 27, 27] 262,144
BatchNorm2d-132 [-1, 256, 27, 27] 512
ReLU-133 [-1, 256, 27, 27] 0
Conv2d-134 [-1, 256, 27, 27] 589,824
BatchNorm2d-135 [-1, 256, 27, 27] 512
ReLU-136 [-1, 256, 27, 27] 0
Conv2d-137 [-1, 1024, 27, 27] 262,144
BatchNorm2d-138 [-1, 1024, 27, 27] 2,048
ReLU-139 [-1, 1024, 27, 27] 0
Bottleneck-140 [-1, 1024, 27, 27] 0
Conv2d-141 [-1, 512, 27, 27] 524,288
BatchNorm2d-142 [-1, 512, 27, 27] 1,024
ReLU-143 [-1, 512, 27, 27] 0
Conv2d-144 [-1, 512, 14, 14] 2,359,296
BatchNorm2d-145 [-1, 512, 14, 14] 1,024
ReLU-146 [-1, 512, 14, 14] 0
Conv2d-147 [-1, 2048, 14, 14] 1,048,576
BatchNorm2d-148 [-1, 2048, 14, 14] 4,096
Conv2d-149 [-1, 2048, 14, 14] 2,097,152
BatchNorm2d-150 [-1, 2048, 14, 14] 4,096
ReLU-151 [-1, 2048, 14, 14] 0
Bottleneck-152 [-1, 2048, 14, 14] 0
Conv2d-153 [-1, 512, 14, 14] 1,048,576
BatchNorm2d-154 [-1, 512, 14, 14] 1,024
ReLU-155 [-1, 512, 14, 14] 0
Conv2d-156 [-1, 512, 14, 14] 2,359,296
BatchNorm2d-157 [-1, 512, 14, 14] 1,024
ReLU-158 [-1, 512, 14, 14] 0
Conv2d-159 [-1, 2048, 14, 14] 1,048,576
BatchNorm2d-160 [-1, 2048, 14, 14] 4,096
ReLU-161 [-1, 2048, 14, 14] 0
Bottleneck-162 [-1, 2048, 14, 14] 0
Conv2d-163 [-1, 512, 14, 14] 1,048,576
BatchNorm2d-164 [-1, 512, 14, 14] 1,024
ReLU-165 [-1, 512, 14, 14] 0
Conv2d-166 [-1, 512, 14, 14] 2,359,296
BatchNorm2d-167 [-1, 512, 14, 14] 1,024
ReLU-168 [-1, 512, 14, 14] 0
Conv2d-169 [-1, 2048, 14, 14] 1,048,576
BatchNorm2d-170 [-1, 2048, 14, 14] 4,096
ReLU-171 [-1, 2048, 14, 14] 0
Bottleneck-172 [-1, 2048, 14, 14] 0
Conv2d-173 [-1, 256, 14, 14] 524,288
BatchNorm2d-174 [-1, 256, 14, 14] 512
Conv2d-175 [-1, 256, 14, 14] 589,824
BatchNorm2d-176 [-1, 256, 14, 14] 512
Conv2d-177 [-1, 256, 14, 14] 65,536
BatchNorm2d-178 [-1, 256, 14, 14] 512
Conv2d-179 [-1, 256, 14, 14] 524,288
BatchNorm2d-180 [-1, 256, 14, 14] 512
detnet_bottleneck-181 [-1, 256, 14, 14] 0
Conv2d-182 [-1, 256, 14, 14] 65,536
BatchNorm2d-183 [-1, 256, 14, 14] 512
Conv2d-184 [-1, 256, 14, 14] 589,824
BatchNorm2d-185 [-1, 256, 14, 14] 512
Conv2d-186 [-1, 256, 14, 14] 65,536
BatchNorm2d-187 [-1, 256, 14, 14] 512
detnet_bottleneck-188 [-1, 256, 14, 14] 0
Conv2d-189 [-1, 256, 14, 14] 65,536
BatchNorm2d-190 [-1, 256, 14, 14] 512
Conv2d-191 [-1, 256, 14, 14] 589,824
BatchNorm2d-192 [-1, 256, 14, 14] 512
Conv2d-193 [-1, 256, 14, 14] 65,536
BatchNorm2d-194 [-1, 256, 14, 14] 512
detnet_bottleneck-195 [-1, 256, 14, 14] 0
Conv2d-196 [-1, 30, 14, 14] 69,120
BatchNorm2d-197 [-1, 30, 14, 14] 60
================================================================
Total params: 26,728,060
Trainable params: 26,728,060
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 2.00
Forward/backward pass size (MB): 1038.47
Params size (MB): 101.96
Estimated Total Size (MB): 1142.43
----------------------------------------------------------------
Process finished with exit code 0
|