|
自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Translations with Trax
Translations with Trax
Google Brain谷歌大脑研发了Tensor2Tensor (T2T)，使得深度学习更容易，T2T是TensorFlow的扩展，包含了一个深度学习库，模型包含许多Transformer示例。
Tensor2Tensor (T2T)是一个良好的开端，谷歌Brain产生了端到端Trax深度学习库，Trax包含可应用于翻译的Transformer模型, 谷歌Brain团队目前维护着Trax。
本文我们将重点讨论Vaswani等人(2017)描述的英德翻译最小函数初始化问题，以说明Transformer的的性能。我们将使用预处理的英语和德语数据集来显示Transformer体系结构与语言无关。
Trax_Translation.ipynb从安装需要的模块开始。
import os
import numpy as np
import trax
是的，就是这么简单!
现在，让我们创建Transformer 模型。
Creating a Transformer model
我们从Transformer的模型体系结构开始,Trax函数将按配置检索预训练的模型.
model = trax.models.Transformer(
input_vocab_size=33300,
d_model=512, d_ff=2048,
n_heads=8, n_encoder_layers=6, n_decoder_layers=6,
max_len=2048, mode='predict')
该模型是带有编码器和解码器堆栈的Transformer。每个堆栈包含6层和8个头部 ,D_model =512与Transformer有相同的架构。
Transformer需要运行预训练的权重 。
Initializing the model using pretrained weights
预训练权重构成了Transformer的语言表示，权重可以表示为一些参数，这些参数将产生某种形式的机器智商IQ。
初始化权重 :
model.init_from_file('gs://trax-ml/models/translation/ende_wmt32k.pkl.gz',
weights_only=True)
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
(((),
((), ((), ((), (), ()))),
((array([[ 0.05705298, 0.0897216 , 0.02124788, ..., -0.07733061,
-0.00756849, 0.0863874 ],
[-0.6957528 , 0.52017885, 0.01700701, ..., -0.12895925,
0.35437337, -0.9630883 ],
[ 0.05140841, -0.02195717, -0.09755829, ..., 0.08997008,
0.13417095, -0.26320305],
...,
[ 0.10300644, -0.06085985, -0.27162302, ..., 0.05554791,
-0.11951675, 0.17515816],
[-0.03211264, 0.01250579, -0.7359762 , ..., 0.39658797,
0.6097316 , 0.38914362],
[-0.1319148 , 0.13255426, -0.2606699 , ..., 0.02071916,
0.1670416 , 0.03873056]], dtype=float32),
(),
array([[[ 1.143651 , 1.0403643 , 0.1692191 , ..., 0.4217065 ,
-0.3508536 , 0.24478772],
[ 0.8617227 , -0.77985394, 1.7588066 , ..., 0.11070859,
0.2525967 , 0.11612047],
[-0.14974353, -0.88516223, 0.0342847 , ..., 0.26336393,
0.28872377, 0.22468647],
...,
[ 0.06449221, -0.36093104, -0.06822655, ..., 0.35782918,
0.0771453 , 0.3584378 ],
[-0.26886657, -0.2492804 , 0.25719792, ..., 0.3578206 ,
0.07718264, 0.35842982],
[-0.35503155, 0.09155753, 0.36127755, ..., 0.35781205,
0.07721993, 0.35842174]]], dtype=float32),
(((),
((),
((array([0.78396314, 0.58032054, 0.6696018 , 0.6808054 , 0.6434321 ,
0.68325233, 0.77864265, 0.54177266, 0.6939366 , 0.53155243,
0.7506562 , 0.57536435, 0.7102179 , 0.7888767 , 0.6595054 ,
0.6525936 , 0.77018315, 0.73037016, 0.47918084, 0.6590534 ,
....
0.85482347, 0.7594529 , 0.3780823 , 0.80268675, 0.8431776 ,
0.815761 , 0.58810884, 0.8992868 , 1.0852215 , 0.8452261 ,
0.8531468 , 0.8723731 , 0.8734694 , 0.76962787, 0.7416202 ,
0.75197744, 0.8319315 , 0.79353607, 0.9299328 , 0.81899166,
0.7162788 , 0.82320136], dtype=float32),
array([ 4.05150140e-03, 5.27852308e-03, 1.14727684e-03, 4.59734630e-03,
-8.10241268e-04, 1.08174770e-03, 8.98309168e-04, 4.83894954e-03,
.....
2.92939553e-03, -2.00807024e-03, 5.71298879e-03, -5.04219951e-03],
dtype=float32)),
((),
((),
((),
(((array([[-0.00983255, -0.01889868, 0.04254947, ..., -0.04968164,
-0.02589379, -0.02438862],
[-0.01838136, 0.10078885, 0.00802404, ..., -0.09020539,
0.09048726, -0.07120325],
[ 0.05777453, -0.02979612, 0.00717621, ..., 0.00313391,
-0.01440868, 0.01646473],
...,
[ 0.01317722, 0.06473751, -0.04793017, ..., -0.06019548,
0.08053429, -0.08464969],
[ 0.08383329, -0.02072529, -0.02371077, ..., 0.06034886,
-0.00062477, -0.00554504],
[-0.04712729, 0.03563578, -0.04299119, ..., -0.02282772,
0.00213189, -0.01969139]], dtype=float32),
array([ 6.30819658e-03, -2.42519332e-03, 9.65738320e-04, 3.94032802e-03,
2.98257056e-03, -5.72120352e-03, 2.40378827e-03, 8.10004422e-04,
.....
-7.49644870e-03, -3.16923484e-02, -5.97513281e-03, -1.41367707e-02],
dtype=float32)),
(array([[-0.07449927, 0.04084178, -0.00152788, ..., 0.01092932,
-0.1081888 , -0.02391315],
[-0.01409085, -0.01460742, 0.01363205, ..., -0.03848151,
0.01406184, -0.0111003 ],
[-0.08024514, 0.08280817, -0.04111585, ..., 0.03955729,
0.06565299, -0.10677323],
...,
[-0.02786698, 0.12181292, 0.07619616, ..., 0.03162721,
-0.01636616, -0.04887632],
[-0.00037219, 0.11405649, 0.05229751, ..., 0.05899883,
0.0139249 , 0.04951515],
[ 0.00647625, 0.07594762, 0.02067034, ..., -0.01015966,
-0.05069298, 0.00357397]], dtype=float32),
array([-1.56847504e-03, -3.91228852e-04, -1.39212934e-03, -2.41137110e-04,
1.19162106e-03, 1.20998919e-03, 9.27774163e-05, 6.73537899e-04,
....
7.84029035e-05, -1.81641860e-03, 4.92170628e-04, 2.46795971e-04,
3.19628190e-04, -4.55839545e-05, 9.98356962e-04, 7.42866483e-04],
dtype=float32)),
(array([[-0.02352409, 0.0112926 , 0.03170136, ..., 0.02165376,
0.00294732, -0.02305261],
[ 0.02488195, -0.03432996, -0.00839461, ..., -0.006901 ,
0.04413575, -0.00692465],
[ 0.01638577, -0.02584397, -0.02642302, ..., 0.00746474,
-0.01324901, -0.00802223],
...,
[-0.00637522, 0.05794188, 0.00741955, ..., 0.02341847,
0.00567059, 0.00102247],
[ 0.01990202, -0.00079649, 0.02237192, ..., -0.02036999,
-0.02263673, 0.01997334],
[ 0.00122439, 0.04450148, -0.03642509, ..., -0.05580654,
-0.01226757, -0.02887065]], dtype=float32),
array([ 5.01244387e-04, -5.75548504e-04, -2.16729793e-04, -5.48165874e-04,
1.77858848e-04, -2.97376799e-04, -2.99024192e-04, -2.45218311e-04,
9.87236708e-05, 3.96675052e-04, -1.24326893e-04, -1.43147816e-04,
.....
2.88170791e-04, -7.95034284e-04, 3.27270507e-04, -4.24875107e-05],
dtype=float32))),
(),
(array([[-0.05587342, -0.01420343, -0.01025653, ..., 0.00373129,
-0.04279305, -0.03390695],
[ 0.01339305, 0.00235549, 0.03377225, ..., 0.0140237 ,
0.01085103, -0.02298495],
[ 0.01897964, -0.00565846, 0.01265932, ..., -0.01241283,
0.01481292, 0.06486797],
...,
[ 0.04784625, -0.00418825, -0.02697581, ..., 0.00899825,
-0.00204265, -0.01294921],
[ 0.0273882 , 0.0447928 , 0.01065076, ..., -0.03944066,
-0.05660933, -0.05735212],
[-0.00538272, -0.01057845, -0.04171268, ..., 0.00693211,
0.02464722, 0.01105996]], dtype=float32),
array([-1.27969717e-03, -5.42177586e-04, -5.80465829e-04, -9.99189506e-05,
-3.36362456e-04, -6.61161786e-04, 2.39674249e-04, -3.19434970e-04,
.....
-1.34417182e-03, 2.41281974e-04, -3.33459873e-04, 2.78818829e-04],
dtype=float32))),
())),
()),
()))),
()),
(((),
((),
((array([1.102134 , 1.0123407 , 0.8853125 , 0.8507046 , 0.77589196,
0.8322442 , 0.7730745 , 0.6186642 , 0.8086139 , 0.48227188,
0.7269428 , 0.7032602 , 0.6900958 , 1.0526972 , 0.6340356 ,
0.72584444, 0.06727506, 0.2045577 , 0.6879156 , 0.65786254,
....
0.9952131 , 0.9630924 ], dtype=float32),
array([-1.44699141e-01, -1.40607193e-01, -5.93346953e-02, -4.71118242e-02,
1.17056109e-02, 7.82178864e-02, 7.93147311e-02, -6.87639490e-02,
-3.68599147e-02, -1.12718925e-01, -8.64906162e-02, -2.32138380e-01,
......
-2.71341860e-01, 2.26536542e-01, -9.21389833e-02, 1.64688647e-01],
dtype=float32)),
(array([[-0.01093922, -0.02913175, -0.00902539, ..., -0.03531323,
0.01737153, 0.0329526 ],
[ 0.02787731, 0.05038743, -0.01863357, ..., 0.02651414,
0.039524 , -0.00425833],
[ 0.00379309, -0.02712435, 0.00365311, ..., -0.01453952,
-0.0352793 , 0.03874848],
...,
[-0.02661362, 0.00248995, -0.01399386, ..., -0.00703348,
0.00660558, -0.03778128],
[ 0.00498781, 0.01402566, 0.03397956, ..., 0.00439327,
0.0298744 , -0.02996395],
[-0.02134071, -0.00769194, -0.01524895, ..., -0.01931463,
-0.01611234, -0.00385839]], dtype=float32),
array([-0.00217734, -0.00234205, -0.00039003, ..., -0.0017114 ,
-0.00117945, -0.00120653], dtype=float32)),
((), (), ()),
(),
(array([[ 2.8253358e-05, -2.0919122e-02, 2.7881092e-03, ...,
2.0900771e-02, -3.4419663e-02, -1.8172318e-02],
[-9.7109034e-04, -2.0357359e-02, 5.3021088e-03, ...,
2.5118981e-02, 2.4128985e-02, -3.2650912e-03],
[-6.4096176e-03, 1.3212421e-02, 2.9458484e-02, ...,
-3.0640023e-02, -3.6055189e-02, -2.5525706e-02],
...,
[-1.7361064e-02, 1.3076221e-05, -2.4035037e-02, ...,
-3.0048754e-02, -3.6518399e-02, -1.7251316e-02],
[ 4.5724425e-02, 5.5810958e-02, -2.2222040e-02, ...,
5.7667037e-03, 1.4665924e-02, -2.5030434e-02],
[ 2.5763189e-02, -4.0483389e-02, 4.4222028e-04, ...,
3.1355478e-02, -5.4799533e-03, 1.1510235e-03]], dtype=float32),
array([ 2.18823133e-03, 2.79318169e-03, 4.10764507e-04, 1.19657838e-03,
2.02708715e-03, 9.37822973e-04, 1.03020156e-03, -1.98605958e-05,
-1.59321658e-04, 6.48561690e-04, 4.19441465e-04, -2.00567651e-04,
1.08302478e-03, 2.32592877e-03, 2.37255031e-03, 5.53323945e-04,
.....
7.44775694e-04, 5.85523201e-04, 9.04807996e-04, -2.55566207e-03,
1.85412948e-03, 1.40106387e-03, 2.00630259e-03, -1.14389742e-03],
dtype=float32)),
()))),
()),
(((),
((),
((array([0.5621848 , 0.6937268 , 0.8574977 , 0.86983985, 0.6055682 ,
0.83376575, 0.74329376, 0.9448553 , 0.8519927 , 1.1055367 ,
1.045579 , 0.73620087, 0.8716647 , 0.9349683 , 0.8104241 ,
0.9176326 , 1.969923 , 1.877579 , 0.7240503 , 0.92385656,
1.0188955 , 0.7490151 , 0.8387114 , 0.8565147 , 0.78953433,
0.7144554 , 0.6806402 , 0.7426592 , 0.71180755, 0.7283262 ,
0.49806422, 0.5552644 , 0.5617643 , 0.66560435, 0.5917875 ,
0.5699419 , 0.6815993 , 0.6134473 , 0.6449534 , 0.6297416 ,
0.8026556 , 0.39487982, 0.64342576, 0.54356563, 0.5880048 ,
.....
0.6289075 , 0.5845859 , 0.6637251 , 0.8297579 , 0.5819279 ,
0.705869 , 0.70433384, 0.6484329 , 0.38405302, 0.54425037,
0.60226005, 0.6035507 , 0.61098737, 0.7589033 , 0.60919267,
0.58097786, 0.6475794 ], dtype=float32),
array([ 4.68320725e-03, 2.20737609e-04, 5.78584732e-04, 3.01261223e-03,
-3.56059405e-03, -6.14388919e-05, 2.78966408e-03, -9.55396463e-05,
-1.93396234e-03, 2.60684430e-03, -1.17261072e-04, 4.50838450e-03,
3.55625432e-03, -2.05258932e-03, 1.40377332e-03, 9.09683586e-04,
1.29926542e-03, 1.79099524e-03, 1.92633295e-03, 2.60444870e-03,
.....
-2.19818763e-03, -4.21653851e-04, 3.08287679e-04, -9.23711620e-03,
2.13323883e-03, 1.09106395e-03, 3.59544158e-03, 2.00573821e-04,
4.03929222e-03, -4.05074703e-03, 3.18779581e-04, -6.07974827e-04],
dtype=float32)),
((),
((),
((),
(((array([[ 0.04436483, 0.04771446, -0.13436686, ..., -0.14226885,
0.04808403, -0.06580725],
[-0.05019807, 0.04349452, -0.06982896, ..., -0.00121004,
0.02473206, 0.06915052],
[ 0.01451271, 0.06423736, -0.08250674, ..., 0.02034138,
0.0450182 , -0.01055613],
...,
[ 0.00239087, 0.03666861, -0.03787184, ..., 0.05566741,
-0.0082335 , -0.03599276],
[-0.07235014, 0.02153421, -0.03740313, ..., 0.04786579,
0.09983153, 0.02284449],
[ 0.03088157, 0.00420653, -0.01671258, ..., -0.01756766,
-0.08169941, -0.04031602]], dtype=float32),
array([ 1.13366377e-02, -1.10620924e-03, 9.06919315e-03, -1.00711482e-02,
1.33993514e-02, -5.50241116e-03, -5.42777684e-03, -5.38460212e-03,
-1.24234017e-02, -4.58599720e-03, 3.08457180e-03, -1.14587438e-03,
5.95363928e-03, -1.06670782e-02, 3.97509756e-03, -1.81169831e-03,
......
-1.85013898e-02, -8.74073058e-03, 1.42205199e-02, -8.69629625e-03,
1.16403296e-03, 1.02645494e-02, 1.13955443e-03, 1.22828763e-02,
7.10584084e-03, -7.90077262e-03, 1.89334620e-03, -7.57595571e-03,
-1.44457836e-02, 6.83462294e-03, -2.23615784e-02, 1.28387229e-03,
6.32393174e-03, -6.34944299e-03, 1.49642797e-02, 1.05375294e-02,
-2.64909188e-03, -8.88486300e-03, -1.75779350e-02, -1.28543675e-02],
dtype=float32)),
(array([[ 0.01685577, -0.07905079, 0.13963036, ..., -0.00482945,
-0.00159723, 0.01762539],
[-0.01799799, -0.00388364, -0.0637244 , ..., -0.04484536,
-0.0224951 , -0.07293875],
[-0.04229953, -0.02852584, -0.05636088, ..., -0.02659514,
-0.00414677, -0.02763357],
...,
[ 0.01015009, -0.01773522, 0.04523581, ..., -0.0336475 ,
-0.09781008, 0.04131836],
[-0.01548378, 0.04935797, -0.00822115, ..., -0.03964528,
0.00336613, 0.0690268 ],
[-0.00976845, 0.01454094, 0.04269084, ..., 0.07415776,
-0.01026561, 0.00940721]], dtype=float32),
array([-6.29140064e-04, -1.23964055e-04, 3.10195464e-04, -5.62312052e-05,
-7.78545975e-04, -1.88774997e-04, 4.71177264e-05, -5.38174529e-04,
8.57309246e-08, -4.89368627e-04, -4.30835149e-04, 4.68431885e-04,
2.51237303e-04, 3.94859351e-04, 1.33680369e-04, 7.28936109e-04,
9.22648251e-05, -3.49204027e-04, 1.04444311e-03, 8.40976951e-04,
7.20981392e-04, -9.21021507e-04, 2.96629209e-04, -4.43926314e-04,
......
-8.88829411e-04, -5.05586388e-04, 1.45784987e-04, -6.02267624e-04,
4.33463319e-05, 2.98810715e-04, -2.18872956e-04, -8.17719119e-05],
dtype=float32)),
(array([[ 0.02876149, -0.00048357, -0.01141942, ..., 0.00579668,
0.00868742, -0.01983118],
[ 0.03238959, 0.02785595, -0.00096219, ..., 0.03324736,
-0.00416722, 0.00544274],
[ 0.04681065, 0.04084571, -0.02620286, ..., -0.01916425,
0.01792918, -0.01645469],
...,
[-0.03378287, 0.01039147, 0.03121787, ..., -0.00649906,
-0.02564578, 0.00225107],
[-0.01079457, 0.03634041, 0.00541349, ..., 0.01231498,
0.0269435 , 0.03434065],
[-0.03088765, 0.06068046, -0.0220051 , ..., 0.04710462,
-0.02975444, -0.04363554]], dtype=float32),
array([-7.65498553e-04, 2.74482096e-04, -4.90025384e-04, 1.04152400e-03,
-2.08146557e-05, 2.44983676e-04, 6.64887717e-04, 3.47387337e-04,
3.27068090e-04, -4.93900734e-04, -2.89052899e-04, 6.88469852e-04,
5.25962270e-04, 3.63405794e-04, -2.17920722e-04, 6.80900936e-04,
.........
9.90654924e-04, 6.16334903e-04, 6.50273694e-04, -5.07433084e-04,
-3.08774645e-04, -1.34287006e-03, -7.97831162e-04, -1.04969240e-05,
2.54309125e-04, 6.50372589e-04, 1.05812016e-03, -4.23422491e-04],
dtype=float32))),
(),
(array([[ 0.02718984, 0.02894734, 0.00693258, ..., -0.02390157,
-0.01956652, 0.02613108],
[-0.01051425, -0.00247402, 0.00437519, ..., 0.01355012,
-0.01375171, -0.00030979],
[ 0.03315658, -0.03321722, 0.00592927, ..., -0.02752117,
-0.033825 , 0.00401202],
...,
[-0.01505975, 0.00909435, 0.02610086, ..., -0.03313698,
-0.00125495, 0.03623522],
[-0.0577226 , 0.06591149, -0.01649481, ..., 0.00336204,
-0.0452969 , 0.00461343],
[ 0.00459657, -0.04356053, 0.03990532, ..., 0.05888681,
-0.0265279 , -0.02966733]], dtype=float32),
array([-4.33505338e-04, 2.64111964e-04, 3.18850856e-04, -6.49480862e-05,
1.00082438e-03, -5.91101008e-04, -3.16322141e-04, -6.61888975e-04,
-8.47268850e-04, -9.25756234e-04, 4.87258425e-04, -7.85998709e-04,
-7.00545148e-04, -4.11492510e-04, 1.87516984e-04, -1.46421080e-03,
-7.37535243e-04, 5.65405004e-04, 5.31164347e-04, 6.41480554e-04,
........
4.31178807e-04, 1.96512628e-05, -4.00108023e-04, 3.00906342e-03,
-7.28485815e-04, 1.08956163e-04, 1.01793812e-04, 7.35571230e-05,
-1.44627306e-03, 6.67295302e-04, -4.26555023e-04, 8.56978004e-04],
dtype=float32))),
())),
()),
()))),
()),
(((),
((),
((array([2.0435145 , 1.616267 , 1.433323 , 1.7765775 , 1.4481372 ,
1.3071214 , 1.7041181 , 0.9068846 , 1.0360289 , 0.6299846 ,
1.3046974 , 1.0437144 , 1.1240528 , 1.2478328 , 0.70106614,
0.9303421 , 0.07168967, 0.05904389, 0.72973365, 1.1609468 ,
.......
2.0218782 , 1.7913011 , 2.041556 , 2.5451207 , 1.989467 ,
1.8323137 , 1.9481978 , 2.0517547 , 2.3482096 , 2.0776472 ,
1.7166487 , 1.5429945 ], dtype=float32),
array([-2.45205268e-01, -1.52529493e-01, -1.71360388e-01, -7.36087188e-02,
-1.33912817e-01, -2.11142879e-02, 7.53392465e-03, -4.98694777e-02,
6.73716376e-03, -7.67695308e-02, -2.28923991e-01, -2.33232781e-01,
-5.40046878e-02, 2.79090349e-02, -7.98283368e-02, -9.01315585e-02,
.......
-2.66913950e-01, -3.12474836e-02, -1.22217901e-01, 2.91088462e-01,
-2.00575441e-01, 2.24337459e-01, -1.79571509e-01, 1.84825465e-01],
dtype=float32)),
(array([[-0.00034249, -0.02955409, 0.05068079, ..., 0.01298233,
-0.00082925, 0.01931314],
[ 0.01880113, 0.04796297, 0.04905764, ..., 0.01041629,
-0.00433718, -0.00569308],
[ 0.00526232, -0.00966249, 0.03375914, ..., -0.01586324,
0.04436447, 0.01092449],
...,
[ 0.00228407, -0.00670767, -0.00993229, ..., -0.04431191,
0.03151484, 0.0065571 ],
[ 0.00749401, 0.00570948, 0.00931196, ..., 0.05612922,
0.03447838, -0.0015762 ],
[ 0.00420937, 0.00590031, -0.00693895, ..., -0.03597783,
-0.02307665, -0.00371054]], dtype=float32),
array([-0.0012094 , -0.0020339 , -0.00134537, ..., -0.00121453,
-0.00122303, -0.000543 ], dtype=float32)),
((), (), ()),
(),
(array([[ 0.00058908, -0.015174 , -0.02469857, ..., 0.02299635,
-0.04136869, 0.01211076],
[ 0.00726052, 0.01816828, 0.00306317, ..., 0.00619089,
-0.01264757, -0.00242481],
[ 0.00536577, -0.01337963, 0.02425298, ..., -0.00369241,
-0.00613173, -0.01018831],
...,
[ 0.0238857 , -0.01658127, -0.0214825 , ..., -0.05886063,
0.00421863, 0.00267504],
[-0.01767615, -0.00770401, -0.0072536 , ..., -0.01277949,
0.06342866, -0.00072114],
[-0.00200121, 0.00901863, 0.02561563, ..., 0.0446266 ,
0.0006642 , -0.0045839 ]], dtype=float32),
array([ 1.32858241e-03, 3.96640645e-03, 6.98459160e-04, 3.32345226e-04,
2.33496120e-03, 8.87550996e-04, 4.92309744e-04, 2.26677774e-04,
-1.30674453e-03, 1.40756631e-04, 6.79603196e-04, 6.61036174e-04,
1.03546621e-03, 1.17660221e-03, 1.31010485e-03, 6.19838305e-04,
-1.11619721e-03, 4.58938244e-04, -4.06333274e-04, 2.10141228e-03,
3.27350805e-04, 8.49412754e-04, -1.11570978e-03, -7.72173749e-04,
1.27991475e-03, 5.07304037e-04, 7.70613318e-04, -1.18349039e-03,
1.49820640e-03, 1.28383853e-03, -1.42487756e-04, 7.45691301e-04,
-5.91686257e-05, 3.54575051e-04, 8.64963164e-04, -1.61387463e-04,
........
1.33650808e-03, 7.63861870e-04, 5.34134451e-05, -1.54597452e-03,
9.91755049e-04, 8.49092961e-04, 1.52893877e-03, -6.81481499e-04],
dtype=float32)),
()))),
()),
(((),
((),
((array([0.8566098 , 0.8261439 , 0.91383135, 1.0060054 , 0.6466727 ,
0.9427771 , 0.8478438 , 1.0285401 , 0.8312587 , 1.347589 ,
1.1794763 , 0.85727376, 0.9290037 , 1.0969888 , 0.9709032 ,
0.98100406, 2.1127503 , 1.9069791 , 0.83928233, 1.0415456 ,
1.2178383 , 0.97466844, 1.1155393 , 0.8499728 , 0.89386004,
.......
0.71305907, 0.64385504, 0.47255883, 0.76376677, 0.1295525 ,
0.53655064, 0.78703696, 0.39029637, 0.83108765, 0.81934404,
0.7972474 , 0.7136157 , 0.7192456 , 1.0058353 , 0.70057535,
0.6779891 , 0.6950896 , 0.7216264 , 0.44179034, 0.64593446,
0.7800695 , 0.7560659 , 1.0054858 , 0.8252287 , 0.79663163,
0.7966191 , 0.6117478 ], dtype=float32),
array([ 3.8292690e-03, 4.1993810e-03, 4.8097083e-03, 3.3849715e-03,
3.3917250e-03, -3.0078106e-03, -2.3171555e-05, 6.2169752e-04,
-8.1133441e-04, 1.6350171e-03, 9.5927238e-04, 2.7699617e-03,
-1.6039971e-04, -6.6661322e-04, -3.9064401e-04, 3.3372839e-03,
1.1640869e-03, 2.1356158e-03, 4.3786098e-03, 4.7431393e-03,
.........
1.3311458e-03, -6.6650445e-03, -1.5684001e-03, 1.6583666e-03,
-4.3004551e-03, 3.2472201e-03, 1.6087124e-03, 2.9584568e-03,
-9.4123656e-04, 5.2291555e-03, 1.6761480e-05, -3.1569730e-03,
-1.9260325e-03, -1.7602117e-05, -3.2044963e-03, -7.8132609e-03,
4.8444029e-03, 2.1238631e-04, 3.5557058e-03, -3.6434764e-03,
5.7954984e-03, -1.2477320e-04, 2.6847082e-03, -4.8846123e-03],
dtype=float32)),
((),
((),
((),
(((array([[-0.00389616, 0.05882167, -0.09089489, ..., -0.00266514,
-0.01033158, 0.01096952],
[-0.08967611, -0.08763819, -0.0821876 , ..., 0.05575378,
-0.00908734, -0.03308633],
[-0.08291744, 0.01129396, -0.04385214, ..., 0.02803485,
0.06249096, -0.027956 ],
...,
[ 0.02164162, -0.03804163, -0.0266341 , ..., -0.0932552 ,
-0.00640112, 0.03252052],
[-0.02288172, -0.03197748, -0.05085482, ..., -0.02749549,
-0.00046983, -0.04685643],
[ 0.02428026, -0.01089646, -0.0402813 , ..., 0.00024081,
0.00835228, 0.01939279]], dtype=float32),
array([ 2.30888557e-03, 5.60043904e-04, -3.11916252e-03, -1.00670138e-03,
-3.30467476e-03, 4.09853086e-03, 4.96401684e-03, 2.73018517e-03,
-4.88881068e-03, 4.85400110e-03, -4.55736089e-03, -3.95556400e-03,
-3.85924289e-03, 3.73602263e-03, -3.79285077e-03, 4.07824339e-03,
5.77018037e-03, 5.90089289e-03, -9.37878247e-03, -1.81852980e-03,
.......
2.71145487e-03, -5.68174990e-04, -1.53347978e-03, -6.90627785e-04,
-4.41261975e-04, -1.54486357e-03, -3.32891266e-03, 6.22269406e-04,
-6.07610564e-04, -3.28153907e-03, 1.65029394e-03, 1.86938350e-03,
1.11232381e-04, 2.40195586e-04, -2.31642523e-04, 1.45475462e-03],
dtype=float32)),
(array([[-0.00912927, 0.00248751, -0.00162794, ..., 0.02239549,
0.04036287, -0.07744382],
[ 0.0066331 , -0.03722198, -0.02241602, ..., 0.01498746,
-0.04410183, -0.02918728],
[ 0.08993914, 0.05555706, -0.03422953, ..., -0.00722888,
0.03674918, 0.02092756],
...,
[ 0.04052105, 0.00810029, 0.04122091, ..., 0.05442077,
0.04312225, -0.02330261],
[ 0.08667028, 0.04664524, 0.00437689, ..., -0.02887452,
0.01824609, 0.03905721],
[-0.02360163, -0.07396459, 0.01954039, ..., -0.01249475,
0.00280824, -0.02435429]], dtype=float32),
array([ 5.57447667e-04, 1.17520962e-04, -2.08095857e-03, 5.12223982e-04,
1.35292867e-04, 3.68826120e-04, -4.50300460e-04, -2.39803398e-04,
-2.69127981e-04, -3.25544941e-04, 8.56164552e-05, 8.21966096e-04,
9.36341821e-05, -5.87902498e-04, 4.37240320e-04, 1.41791638e-03,
........
-4.13642323e-04, 4.49604384e-04, 7.92738691e-04, 5.34845240e-05,
5.74861770e-04, -4.07628162e-04, -1.50003831e-03, -2.24982228e-04,
-9.24676773e-04, 1.00160250e-04, -1.95888322e-04, 3.35855380e-04,
-1.72361706e-05, 9.77941439e-04, -8.20175046e-04, -3.08826275e-04,
-1.55481498e-03, -6.24585780e-04, -9.48479283e-04, 2.36428878e-05,
-6.87443244e-04, -3.62270774e-04, -5.06167940e-04, 9.64527891e-04,
1.25815405e-03, -5.56849525e
模型配置现在可以运行了
星空智能对话机器人系列博客
|