[人工智能]Bert model output不会随意改变

Bert model output不会随意改变

随便load两个bert model进行验证。

>>>model1 = BertModel.from_pretrained(checkpoint)
>>>model2 = BertModel.from_pretrained(checkpoint)

>>>a = tokenizer("Hello, my dog is cute", return_tensors="pt")
>>>b = tokenizer("Good Morning every day!", return_tensors="pt")

>>>output_a = model1(**a)[0]
>>>output_b = model2(**b)[0]

>>>output_a
>>>tensor([[[-0.1144,  0.1937,  0.1250,  ..., -0.3827,  0.2107,  0.5407],
         [ 0.5308,  0.3207,  0.3665,  ..., -0.0036,  0.7579,  0.0388],
         [-0.4877,  0.8849,  0.4256,  ..., -0.6976,  0.4458,  0.1231],
         ...,
         [-0.7003, -0.1815,  0.3297,  ..., -0.4838,  0.0680,  0.8901],
         [-1.0355, -0.2567, -0.0317,  ...,  0.3197,  0.3999,  0.1795],
         [ 0.6080,  0.2610, -0.3131,  ...,  0.0311, -0.6283, -0.1994]]],
       grad_fn=<NativeLayerNormBackward>)

>>>output_b
>>>tensor([[[ 0.1286,  0.4435, -0.0013,  ..., -0.0280,  0.0254,  0.1479],
         [ 0.4923,  0.2065,  0.7322,  ...,  0.3164,  0.9537, -0.2725],
         [ 0.2850,  0.5958,  0.2142,  ...,  0.2637,  0.6935, -1.3050],
         ...,
         [-0.5116, -0.4607,  0.2709,  ...,  0.2901,  0.5506, -0.7153],
         [-0.1843, -0.0266,  0.0022,  ...,  0.9494,  0.2513, -0.1752],
         [ 0.8566,  0.1451, -0.2009,  ...,  0.1922, -0.6008, -0.1922]]],
       grad_fn=<NativeLayerNormBackward>)

分别用两个模型分别对两句话encode。

>>>output_b1 = model1(**b)[0]
>>>output_b1
>>>tensor([[[ 0.1286,  0.4435, -0.0013,  ..., -0.0280,  0.0254,  0.1479],
         [ 0.4923,  0.2065,  0.7322,  ...,  0.3164,  0.9537, -0.2725],
         [ 0.2850,  0.5958,  0.2142,  ...,  0.2637,  0.6935, -1.3050],
         ...,
         [-0.5116, -0.4607,  0.2709,  ...,  0.2901,  0.5506, -0.7153],
         [-0.1843, -0.0266,  0.0022,  ...,  0.9494,  0.2513, -0.1752],
         [ 0.8566,  0.1451, -0.2009,  ...,  0.1922, -0.6008, -0.1922]]],
       grad_fn=<NativeLayerNormBackward>)
>>>output_a1 = model2(**a)[0]
>>>output_a1
>>>tensor([[[-0.1144,  0.1937,  0.1250,  ..., -0.3827,  0.2107,  0.5407],
         [ 0.5308,  0.3207,  0.3665,  ..., -0.0036,  0.7579,  0.0388],
         [-0.4877,  0.8849,  0.4256,  ..., -0.6976,  0.4458,  0.1231],
         ...,
         [-0.7003, -0.1815,  0.3297,  ..., -0.4838,  0.0680,  0.8901],
         [-1.0355, -0.2567, -0.0317,  ...,  0.3197,  0.3999,  0.1795],
         [ 0.6080,  0.2610, -0.3131,  ...,  0.0311, -0.6283, -0.1994]]],
       grad_fn=<NativeLayerNormBackward>)

用给input_a编码的model1再encode input_b，未变。

可以验证 bert model一般默认是冻结全部权重，保证不会因为encode了几句话就变了内部参数结构。