开发: C++知识库 Java知识库 JavaScript Python PHP知识库人工智能区块链大数据移动开发嵌入式开发工具数据结构与算法开发测试游戏开发网络协议系统运维
教程: HTML教程 CSS教程 JavaScript教程 Go语言教程 JQuery教程 VUE教程 VUE3教程 Bootstrap教程 SQL数据库教程 C语言教程 C++教程 Java教程 Python教程 Python3教程 C#教程
数码: 电脑笔记本显卡显示器固态硬盘硬盘耳机手机 iphone vivo oppo 小米华为单反装机图拉丁

-> 人工智能 -> 自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Generating unconditional samples with GPT-2 -> 正文阅读

[人工智能]自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Generating unconditional samples with GPT-2

自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Generating unconditional samples with GPT-2

Generating unconditional samples with GPT-2

在医学数据集上训练模型后，接下来生成无条件样本 python generate_unconditional_samples.py --model_name ‘117M’

#@title Step 11: Generating Unconditional Samples
import os # import after runtime is restarted
os.chdir("/content/gpt-2/src")
!python generate_unconditional_samples.py --model_name '117M'

在服务器命令行输入脚本，运行结果如下

(dl) root@59db211f1693:/Chapter08_2/gpt-2-master/src# 
(dl) root@59db211f1693:/Chapter08_2/gpt-2-master/src# python generate_unconditional_samples.py --model_name '117M'
WARNING: Logging before flag parsing goes to stderr.
W1020 04:38:12.259888 140070790878976 deprecation_wrapper.py:119] From generate_unconditional_samples.py:54: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-10-20 04:38:12.261520: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2021-10-20 04:38:12.279426: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.635
pciBusID: 0000:02:00.0
2021-10-20 04:38:12.279702: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2021-10-20 04:38:12.280899: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2021-10-20 04:38:12.281981: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2021-10-20 04:38:12.282288: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2021-10-20 04:38:12.283607: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2021-10-20 04:38:12.284625: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2021-10-20 04:38:12.287597: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2021-10-20 04:38:12.289040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2021-10-20 04:38:12.312183: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2021-10-20 04:38:12.345429: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2500005000 Hz
2021-10-20 04:38:12.347781: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55b06cd2e530 executing computations on platform Host. Devices:
2021-10-20 04:38:12.347831: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2021-10-20 04:38:12.484360: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55b06cd8cef0 executing computations on platform CUDA. Devices:
2021-10-20 04:38:12.484444: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
2021-10-20 04:38:12.486321: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.635
pciBusID: 0000:02:00.0
2021-10-20 04:38:12.486443: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2021-10-20 04:38:12.486501: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2021-10-20 04:38:12.486553: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2021-10-20 04:38:12.486604: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2021-10-20 04:38:12.486656: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2021-10-20 04:38:12.486710: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2021-10-20 04:38:12.486777: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2021-10-20 04:38:12.489994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2021-10-20 04:38:12.490099: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2021-10-20 04:38:12.493143: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-10-20 04:38:12.493206: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0 
2021-10-20 04:38:12.493234: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N 
2021-10-20 04:38:12.496501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10309 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:02:00.0, compute capability: 7.5)
W1020 04:38:12.499568 140070790878976 deprecation_wrapper.py:119] From generate_unconditional_samples.py:56: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

W1020 04:38:12.505186 140070790878976 deprecation_wrapper.py:119] From /Chapter08_2/gpt-2-master/src/sample.py:51: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.

W1020 04:38:12.505573 140070790878976 deprecation_wrapper.py:119] From /Chapter08_2/gpt-2-master/src/model.py:148: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1020 04:38:12.508140 140070790878976 deprecation_wrapper.py:119] From /Chapter08_2/gpt-2-master/src/model.py:152: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.

W1020 04:38:14.277628 140070790878976 deprecation.py:323] From /Chapter08_2/gpt-2-master/src/sample.py:64: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1020 04:38:14.298571 140070790878976 deprecation.py:323] From /Chapter08_2/gpt-2-master/src/sample.py:39: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1020 04:38:14.299457 140070790878976 deprecation.py:323] From /Chapter08_2/gpt-2-master/src/sample.py:67: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
W1020 04:38:16.669366 140070790878976 deprecation_wrapper.py:119] From generate_unconditional_samples.py:65: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

W1020 04:38:16.808486 140070790878976 deprecation.py:323] From /usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
2021-10-20 04:38:19.525713: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
======================================== SAMPLE 1 ========================================
When the ground will be completely shaken from left to right by a better surface abutting the ground, it will already be dry, what is less, shall seem left unsatisfactory.

It is still more certain that I have borrowed from a proof;
whether my assertions are sufficiently strong to resist comparison with the rest of
the syllogisms connected with this part of the Critique, even though I am from
the point of view of a sensuous attribute, or whether impressions of this character
form the basis for many other self-evident assertions, in which, while a very
simple proof, the applicability of the rule must have been immediately
distinguished from the principles. We may therefore have at my disposal two
complicated opinions—on the one hand, on the one side of the question whether the
critique is merely formless, and whether it does not assume the identity of a
divine principle and based upon a certain à priori condition, so as to
proceed beyond the limits of experience, and on the other hand, on the
hypothesis-like principle of the deducibility of all appearances after
our own, and the attainment of a transcendental perfection of all
possible experience by means of the experience of a person, I shall, therefore,
be obliged to employ as little auxiliary, as to elicit, the most
right of the apperception of the object in whatever presence that
has been presented to it.

The view of the metaphysic of body is that of the former, which
exists merely to the extent of complex criticism, but is not to be
ferred to above in any other way; but that of the latter takes this
path, and extends it beyond the limits of experience; and we have not
embraced the question out of sight, in the pure teacherical way,
but merely in spirit, if, as numerous sophists now suggest, no
principles of the conception of the object arise from it, and in this
way the categorical system of the axioms of pure perception
is nevertheless a definite field in which to attend altogether, with a man
who is in no certainty certain of his content. But we pursue this path only
unselfishly, and not at the expense of the object which for the
general
reason is of itself less amenable to plurality of ideas, than of great
transcendental demonstrations, before the greatest rational world
could be equipped. But these are not sought after for a purpose
referring only to this faculty, but that they should be employed as some
important means of advancing the speculative interests of reason
attainably into a second, tertiary, and final triumph of the natural
limits of the powers of thought. Now, in the former we lost all hope of
further ascendancy in a certain private sphere without the
maintaining these remarks. But their application still rested, in
the former instance, on the conception of an object other than that of
things—a conception which, as we have stated above, had
always been, of necessity, speculative. If we were to acknowledge on
the part of our fancy an object that had before it nothing of the kind
covered by variety, but which was conceived and prepared immediately and
declared to be accordingly; and in conformity with this conception, having
it in its power to advance the field of practical experience with unforeseen
successions, it seems that we could plainly call the apprehension of
something particularly so, even noble, at the sight of the conceptions
own by those who in a teleological persuasion know that they ought
to contain the conception of reason, the declarations of empirical
practical conviction, and, with equal circumspection, the
descriptive ground of the principle of a transcendental precept.

I now require us to apply the criterion of a maxim of pure reason
to every of the arts of thought. But we must not read about the
representations which the mind has to develop with this capacity of
reception; they may alone suffice for the guiding principles of the will, and
for the final completion of this work, we must refer with this reference
state of mind to the hypothesis that the mind is always an object of
experience—of the power good or evil, with its aswellissimo expression
as a faculty to progress a series of conditions of experience, in
whole, causality—of the power of reason to guide a series of conditions
of experience, in this case alone, over a space of time still smaller than
the series in which we discover itself. These hypotheses admit of
apart from very little empirical experience, and fill quite
manifold grounds for the construction of our investigations in it, and
would not have proceeded so far in any other manner. That, however,
which among self-satisfied
======================================== SAMPLE 2 ========================================
the quantum will not allow
to itself view form, that is to say, of any internal
reality. But
(1) There cannot be a quantum quantity
above the given size, that is to say,
-it must be unbounded in its matter. Consequently, the
quantum cannot be given beyond this limit.

Section II. Of the Fundamental Postulates and the Function of the
Limits of the Pure Understanding

Section I. Of the Different Mathematical Descriptions

Section III. Of the Principles of Conceptions

Section IV. Of the Equations of Conceptions

Section V. Of Conceptions and the Principles of Conceptions

Section VI. Of Conceptions and its ?sthetic

Section VII. Conceptions which are Dialectical

Section VIII. Of the Logical Undress

Section IX. Of Conceptions of Transcendental and
Transcendental Objects

Section X. Of Transcendental Objects

Section XI. Of Transcendental Objects

Section XII. Of Transcendental Deities

Section XI. Of Deities of—

Section XIII. Of Completeness, Quality and Empiricism

Section XIIIII. Of Empirical Science and Empiricism

Section XIIIV. Of Judgment and Happiness

Section XIIV. Of Empirical Judgements


*

Hence, indeed the only distinction which I can make in
comparing their conception of power and its relation to nature,
relates to the only synthetical distinction of the two. The
conception of power is, as the reader will understand it,
a conception derived from experience found in experience, and which
persists in the use of the predicate of a hypothetical argument. Without
explanation the reader is bound to infer, that the impossibility of
maxims of a given synthetical THought can be met only when changed by
an intuition or a definition. Secondly, the dialectical nature
of the conception required to refute the necessity
of one conception to apply the other by means of a second requires different
approaches to the former. Here also the rules of the understanding have their place
insensitively and thoroughly in the exercise of this centre. Thus,
the first rule that we are to imitate “one cannot prove one
thing” is no ground of contentment from the exertions of reason; but
upon which, when the contradictory dialectic is suppressed in
question, we must again imitate a rule similar: “All things
must be known”; while on the other hand, that there should appear
in the first place an à priori synthesis which could only be demonstrated
by a supersensible intuition, the second rule that “all
things are possible” is not obtained, but necessarily must be
uncovered in the foregoing path.



Against internal contradictions can be easily opposed to external
opposition. For example, we are urged to uproot contradictions
of universal knowledge by answering, “All objects are communicated by the
principle of universality,” and, in the following case, it is
always imperative, from the inability of the determining least
of our objects—mere ideas—that we should leave a pure respect for
solve the internal contradictions by showed and simple answers.
In a correspondence with the above procedure, the material
momentaplii
would be exactly the same as that proposed in § 2. That is,
this would be just the same as would be produced in conformity with the
undisclosed internal distinction à priori of their nature. For example,
the opposite opposite of contradictions is ever so thus criticized;
that is to say, there is no partiality supposed under the conditions
of these internal dialectical analytical doubts—which is clearly a
possible consequence of the explanation of internal contradictions and
how they are at least hidden under empirical conditions. Now, by
the disposition of the discussion to questions, we may be expected of
something quite different than in the case of internal contradictions, which
cannot be simple contradictions, without being at once understood. It ceases to
be natural to go to these interests, to ask how their
determination is always presented as imperatively objective or as an absolute
account of the relation of a given idea to a possible object. As
the questioner purss forward, he knows only that he has to answer a
question concerning the application of principles borrowed by experience, in
other words, respecting relations which do not necessarily follow
from empirical considerations. If reason conducts us to certain
articular facts in understanding, such as the principles which are
ready to put into practice by the introduction and adducing of sufficient
quantities of materials, without being disposed to employ them, we
cannot be sure that such principles can be proved to be abstracted
from

如果仔细观察输出，我们会注意到以下几点：

生成的句子的结构相对可以接受
输出的语法不错
对于非专业人士来说，输出可能看起来像人类

然而，生成的内容毫无意义,Transformer无法生产与我们训练的医学论文相关的实际内容，获得更好的结果将有助于工作。我们总是可以增加数据集的大小。但它会包含我们正在寻找的东西吗？我们能找到更多数据的不良相关性吗？

设想涉及新冠病毒COVID-19的医疗项目，数据集包含以下内容的句子：

“新冠病毒-19不是一种危险的病毒，但它与普通流感病毒相似”
“新冠病毒-19是一种非常危险的病毒”
“新冠病毒-19不是一种病毒，而是实验室创造的东西”
“新冠病毒-19肯定不是实验室创造的！”
“疫苗是危险的！”
“疫苗是救命稻草！”
“各国政府没有正确管理这一流行病”
“政府做了必要的事”

还有很多像这样矛盾的句子。假设您有一个包含数十亿个单词的数据集，但其内容是如此冲突和嘈杂，无论发生什么，你都无法获得可靠的结果！这可能意味着数据集必须更小，并且仅限于来自科学论文的内容，但即便如此，科学家们也常常意见相左。结论是，这将需要大量的艰苦工作和一个坚实的团队来完成可靠的结果。

让我们进一步调查并控制标记化数据。

Controlling tokenized data

在本节中，我们将阅读GPT-2模型用其预训练标记器编码的第一个单词。
我们将使用Training_OpenAI_GPT_2_CH08.ipynb 的标记化数据代码，解压文件out.npz，其包含数据集mdset中的医疗论文编码

#@title Additional Tools : Controlling Tokenized Data
#Unzip out.npz
import zipfile
with zipfile.ZipFile('/content/gpt-2/src/out.npz', 'r') as zip_ref:
zip_ref.extractall('/content/gpt-2/src/')

out.npz已解压缩，我们可以读取arr_0.npy，该NumPy数组包含我们正在查找的编码数据集：

#@title Additional Tools: Controlling Tokenized Data
#Unzip out.npz
import zipfile
with zipfile.ZipFile('/Chapter08_2/gpt-2-master/src/out.npz', 'r') as zip_ref:
    zip_ref.extractall('/Chapter08_2/gpt-2-master/src/src/')

out.npz已解压缩，我们可以读取arr_0.npy，该NumPy数组包含我们正在查找的编码数据集：

#Load arr_0.npy which contains encoded dset
import numpy as np
f=np.load('/Chapter08_2/gpt-2-master/src/src/arr_0.npy')
print(f)
print(f.shape)
for i in range(0,10):
    print(f[i])

运行结果如下：


[1212 5644  326 ...   13  198 2682]
(29379,)
1212
5644
326
11
355
716
78
1765
1868
4778

现在，我们将打开encoder.json并将其转换为Python字典

#We first import encoder.json
import json
i=0
with open("/Chapter08_2/gpt-2-master/models/117M/encoder.json", "r") as read_file:
    print("Converting the JSON encoded data into a Python dictionary")
    developer = json.load(read_file) #converts the encoded data into a Python dictionary
    for key, value in developer.items(): #we parse the decoded json data
        i+=1
        if(i>10):
            break;
        print(key, ":", value)

运行结果如下

Converting the JSON encoded data into a Python dictionary
! : 0
" : 1
# : 2
$ : 3
% : 4
& : 5
' : 6
( : 7
) : 8
* : 9

显示编码数据集的前500个标记的key和值

#We will now search for the key and value for each encoded token
    for i in range(0,500):
        for key, value in developer.items():
            if f[i]==value:
                print(key, ":", value)

添加的这些单词，以确保GPT-2预训练标记器能够轻松识别它们，即：

This suggests that

为了确保GPT-2预训练标记器能够轻松识别它们，事实就是如此

This : 1212
?suggests : 5644
?that : 326

我们可以很容易地识别以初始空格开头的初始标记字符（?）。我们在医学论文中使用以下词语：amoeboid，“amoeboid”是一个生僻词，我们可以看到GPT-2标记器将它分解为几个子词：

?am : 716
o : 78
eb : 1765
oid : 1868

让我们跳过空格，看看发生了什么。“Amoeboid”变成了“am”+“o”+“eb”+“oid”，这里不存在未知标记：[unk]，因为使用了字节级BPE策略。

但是，transformer的注意层可能会关联：

“am”与其他序列关联，如“I am”
“o”与任何被拆分的序列关联，并包含一个“o”
“oid”，另一个序列包含“oid”，可能是带有一些算法的“tabloid”

这不是一个好消息。让我们进一步说明这一点：

amoeboid and mesenchymal

以下输出清楚地显示“and”，但至于其余的标记令人困惑：

在这里插入图片描述
人们可能想知道为什么这是一个问题。原因可以归结为一个单词：“一词多义。”如果我们使用word2vec标记器，字典可能不包含生僻词，如“amoeboid”，我们会想出一个未知的标记。如果我们使用字节级BPE，我们将获得总体上更好的结果，因为我们排除了同一单词的较少变体，如“go”和“go”+“ing”，然而，“amoeboid”中的“am”标记将一词多义在低层带入了这个问题，“am”可以是一种前缀x，“am”可以是“I”+“am”中的单词“am”，也可以是类似的子单词，就像“am”+“bush”一样。注意力层可以将“am”作为一个标记与其他“am”，创建不存在的关系。这是NLU中的多义现象的一个核心问题。

我们可以说正在取得进展，但我们需要更加努力地改进NLP。

以上代码的运行结果如下

This : 1212
?suggests : 5644
?that : 326
, : 11
?as : 355
?am : 716
o : 78
eb : 1765
oid : 1868
?cells : 4778
?are : 389
?less : 1342
?contract : 2775
ile : 576
, : 11
?while : 981
?mes : 18842
ench : 24421
ym : 4948
al : 282
? : 198
cells : 46342
?are : 389
?more : 517
?contract : 2775
ile : 576
, : 11
?and : 290
?there : 612
?may : 743
?be : 307
?a : 257
?switching : 15430
?between : 1022
?am : 716
o : 78
eb : 1765
oid : 1868
?and : 290
?mes : 18842
ench : 24421
ym : 4948
al : 282
? : 198
m : 76
igration : 4254
, : 11
?perhaps : 3737
?there : 612
?can : 460
?also : 635
?be : 307
?a : 257
?switching : 15430
?between : 1022
?the : 262
?dominance : 18648
?of : 286
?chem : 4607
ot : 313
axis : 22704
?( : 357
amo : 18811
eb : 1765
oid : 1868
? : 198
m : 76
igration : 4254
) : 8
?and : 290
?contact : 2800
?guidance : 11154
?( : 357
mes : 6880
ench : 24421
ym : 4948
al : 282
?migration : 13472
) : 8
?[ : 685
60 : 1899
]. : 4083
?One : 1881
?of : 286
?the : 262
?most : 749
?interesting : 3499
?2 : 362
D : 35
? : 198
platform : 24254
s : 82
, : 11
?allowing : 5086
?to : 284
?study : 2050
?contact : 2800
?guidance : 11154
?and : 290
?chem : 4607
ot : 313
axis : 22704
, : 11
?was : 373
?proposed : 5150
?in : 287
?[ : 685
57 : 3553
], : 4357
?in : 287
?which : 543
?the : 262
? : 198
authors : 41617
?demonstrated : 9555
?an : 281
?additive : 38298
?effect : 1245
?of : 286
?chemical : 5931
?grad : 3915
ients : 2334
?and : 290
?fiber : 13608
?alignment : 19114
?by : 416
?measuring : 15964
? : 198
the : 1169
?persistence : 30802
?time : 640
; : 26
?they : 484
?also : 635
?observed : 6515
?that : 326
?cells : 4778
?were : 547
?directed : 7924
?by : 416
?fiber : 13608
?alignment : 19114
?and : 290
?there : 612
?was : 373
? : 198
no : 3919
?effect : 1245
?of : 286
?the : 262
?chemical : 5931
?gradient : 31312
?when : 618
?fibers : 26742
?were : 547
?aligned : 19874
?perpendicular : 47190
?to : 284
?it : 340
. : 13
?A : 317
?similar : 2092
?setting : 4634
? : 198
was : 9776
?also : 635
?used : 973
?for : 329
?studying : 11065
?the : 262
?dependence : 21403
?of : 286
?contact : 2800
?guidance : 11154
?on : 319
?the : 262
?cell : 2685
?cycle : 6772
?[ : 685
48 : 2780
]. : 4083
?However : 2102
, : 11
?In : 554
? : 198
the : 1169
?case : 1339
?of : 286
?different : 1180
?multi : 5021
- : 12
direction : 37295
al : 282
?cues : 25288
, : 11
?totally : 6635
?different : 1180
?scenarios : 13858
?may : 743
?happen : 1645
, : 11
?e : 304
. : 13
g : 70
. : 13
?in : 287
?[ : 685
51 : 4349
] : 60
?it : 340
?is : 318
? : 198
shown : 42579
?that : 326
?for : 329
?contact : 2800
?guidance : 11154
?and : 290
?elect : 1742
rot : 10599
axis : 22704
?in : 287
?the : 262
?cor : 1162
nea : 39718
, : 11
?elect : 1742
rot : 10599
axis : 22704
?wins : 7864
?when : 618
?competing : 11780
? : 198
with : 4480
?the : 262
?direction : 4571
?of : 286
?alignment : 19114
?of : 286
?the : 262
?fibers : 26742
. : 13
? : 198
Multi : 29800
- : 12
cue : 15509
?kinetic : 37892
?model : 2746
?with : 351
?non : 1729
- : 12
local : 12001
?sensing : 34244
?for : 329
?cell : 2685
? : 198
m : 76
igration : 4254
?on : 319
?a : 257
?fibers : 26742
?network : 3127
?with : 351
?chem : 4607
ot : 313
axis : 22704
? : 198
Mart : 13143
ina : 1437
?Con : 1482
te : 660
?aī : 18872
? : 245
?Nad : 21877
ia : 544
?L : 406
oy : 726
?a? : 564
? : 254
a? : 447
? : 94
? : 198
June : 15749
?18 : 1248
, : 11
?2020 : 12131
? : 198
Abstract : 23839
? : 198
C : 34
ells : 19187
?perform : 1620
?directed : 7924
?motion : 6268
?in : 287
?response : 2882
?to : 284
?external : 7097
?stimuli : 25973
?that : 326
?they : 484
?detect : 4886
?by : 416
?sensing : 34244
? : 198
the : 1169
?environment : 2858
?with : 351
?their : 511
?membrane : 25019
?prot : 1237
rus : 14932
ions : 507
. : 13
?In : 554
?particular : 1948
, : 11
?several : 1811
?biochemical : 47685
?and : 290
?bi : 3182
ophysical : 41789
?cues : 25288
?give : 1577
?rise : 4485
?to : 284
?tactic : 18543
?migration : 13472
?in : 287
?the : 262
?direction : 4571
?of : 286
?their : 511
?specific : 2176
?targets : 6670
. : 13
?This : 770
?defines : 15738
? : 198
a : 64
?multi : 5021
- : 12
cue : 15509
?environment : 2858
?in : 287
?which : 543
?cells : 4778
?have : 423
?to : 284
?sort : 3297
?and : 290
?combine : 12082
?different : 1180
, : 11
?and : 290
?potentially : 6196
? : 198
competitive : 46131
, : 11
?stimuli : 25973
. : 13
?We : 775
?propose : 18077
?a : 257
?non : 1729
- : 12
local : 12001
?kinetic : 37892
?model : 2746
?for : 329
?cell : 2685
?migration : 13472
?in : 287
?presence : 4931
?of : 286
? : 198
two : 11545
?external : 7097
?factors : 5087
?both : 1111
?influencing : 32596
?cell : 2685
?polarization : 42704
: : 25
?contact : 2800
?guidance : 11154
?and : 290
?chem : 4607
ot : 313
axis : 22704
. : 13
?We : 775
? : 198
pro : 1676
pose : 3455
?two : 734
?different : 1180
?sensing : 34244
?strategies : 10064
?and : 290
?we : 356
?analyze : 16602
?the : 262
?two : 734
?resulting : 7186
?models : 4981
?by : 416
?recovering : 20222
? : 198
the : 1169
?appropriate : 5035
?macro : 15021
sc : 1416
opic : 16603
?limit : 4179
?in : 287
?different : 1180
?regimes : 25879
, : 11
?in : 287
?order : 1502
?to : 284
?see : 766
?how : 703
?the : 262
?size : 2546
?of : 286
?the : 262
?cell : 2685
, : 11
? : 198
with : 4480
?respect : 2461
?to : 284
?the : 262
?variation : 12291
?of : 286
?both : 1111
?external : 7097
?fields : 7032
, : 11
?influences : 16717
?the : 262
?overall : 4045
?behavior : 4069
. : 13
?Moreover : 10968
, : 11
? : 198
we : 732
?integrate : 19386
?numer : 5470
ically : 1146
?the : 262
?kinetic : 37892
?transport : 4839
?equation : 16022
?in : 287
?a : 257
?two : 734
- : 12
dimensional : 19577
?setting : 4634
?in : 287
?order : 1502
? : 198
to : 1462
?investigate : 9161
?qual : 4140
itatively : 48668
?various : 2972
?scenarios : 13858
. : 13
? : 198
Key : 9218
word : 4775
. : 13
?Kin : 16645
etic : 5139
?equations : 27490
, : 11
?mult : 1963
isc : 2304
ale : 1000
?modeling : 21128
, : 11
?multi : 5021
- : 12
cue : 15509
, : 11
?non : 1729
- : 12
local : 12001
, : 11
?hyd : 7409
rod : 14892
ynamic : 28995
?limit : 4179
, : 11
? : 198

Generating trained conditional samples

在服务器命令行运行 python interactive_conditional_samples.py --temperature 0.8 --top_k 40 --model_name ‘117M’ --length 50


(dl) root@060ffd5da87e:/Chapter08_2/gpt-2-master/src# python interactive_conditional_samples.py --temperature 0.8 --top_k 40 --model_name '117M' --length 50
WARNING: Logging before flag parsing goes to stderr.
W1020 04:47:45.025585 140575232579328 deprecation_wrapper.py:119] From interactive_conditional_samples.py:57: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-10-20 04:47:45.027247: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2021-10-20 04:47:45.044515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.635
pciBusID: 0000:02:00.0
2021-10-20 04:47:45.044825: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2021-10-20 04:47:45.046033: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2021-10-20 04:47:45.047131: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2021-10-20 04:47:45.047448: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2021-10-20 04:47:45.048776: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2021-10-20 04:47:45.049804: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2021-10-20 04:47:45.052806: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2021-10-20 04:47:45.054260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2021-10-20 04:47:45.054613: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2021-10-20 04:47:45.085437: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2500005000 Hz
2021-10-20 04:47:45.088032: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x560494de83f0 executing computations on platform Host. Devices:
2021-10-20 04:47:45.088134: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2021-10-20 04:47:45.232670: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x560494e46d80 executing computations on platform CUDA. Devices:
2021-10-20 04:47:45.232753: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
2021-10-20 04:47:45.234588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.635
pciBusID: 0000:02:00.0
2021-10-20 04:47:45.234708: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2021-10-20 04:47:45.234764: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2021-10-20 04:47:45.234814: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2021-10-20 04:47:45.234862: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2021-10-20 04:47:45.234911: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2021-10-20 04:47:45.234959: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2021-10-20 04:47:45.235008: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2021-10-20 04:47:45.238164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2021-10-20 04:47:45.238264: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2021-10-20 04:47:45.241313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-10-20 04:47:45.241363: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0 
2021-10-20 04:47:45.241390: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N 
2021-10-20 04:47:45.244651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10309 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:02:00.0, compute capability: 7.5)
W1020 04:47:45.247674 140575232579328 deprecation_wrapper.py:119] From interactive_conditional_samples.py:58: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W1020 04:47:45.250178 140575232579328 deprecation_wrapper.py:119] From interactive_conditional_samples.py:60: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

W1020 04:47:45.250591 140575232579328 deprecation_wrapper.py:119] From /Chapter08_2/gpt-2-master/src/sample.py:51: The name tf.AUTO_REUSE is deprecated. Please use tf.compat.v1.AUTO_REUSE instead.

W1020 04:47:45.250924 140575232579328 deprecation_wrapper.py:119] From /Chapter08_2/gpt-2-master/src/model.py:148: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1020 04:47:47.850759 140575232579328 deprecation.py:323] From /Chapter08_2/gpt-2-master/src/sample.py:64: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
W1020 04:47:47.867138 140575232579328 deprecation.py:323] From /Chapter08_2/gpt-2-master/src/sample.py:16: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1020 04:47:47.889066 140575232579328 deprecation.py:323] From /Chapter08_2/gpt-2-master/src/sample.py:67: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.random.categorical` instead.
W1020 04:47:50.205072 140575232579328 deprecation_wrapper.py:119] From interactive_conditional_samples.py:68: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

W1020 04:47:50.345160 140575232579328 deprecation.py:323] From /usr/local/miniconda3/envs/dl/lib/python3.6/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Model prompt >>> 


Model prompt >>> Welcome to the gavin's star space intelligent dialogue robot course
2021-10-20 04:49:23.677542: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
======================================== SAMPLE 1 ========================================
, and the


29. The gavin's star space intelligent dialogue robot course, and the final


30. The gavin's star space intelligent dialogue robot course, 
 31.

       
================================================================================
Model prompt >>> Welcome to the gavin's star space intelligent dialogue robot course
======================================== SAMPLE 1 ========================================


(a) in Space and Time;

(b) in Space and Time: Principles,
(c) in Space and Time: Speculation in Space and Time
(d), and in Space and Time: Speculation in
================================================================================
Model prompt >>> Welcome to the gavin's star space intelligent dialogue robot course
======================================== SAMPLE 1 ========================================


(taken from the gavin's website), or this page for other Gist
searchable content:

http://www.gist.com/p/2d1c6a7e68e1e3
================================================================================
Model prompt >>> 



Model prompt >>> During such processes, cells sense the environment and respond to
external factors that induce a certain direction of motion towards
specific targets (taxis): this results in a persistent migration in a
certain preferential direction. The guidance cues leading to directed
migration may be biochemical or biophysical. Biochemical cues can
be, for example, soluble factors or growth factors that give rise to
chemotaxis, which involves a mono-directional stimulus. Other cues
generating mono-directional stimuli include, for instance, bound
ligands to the substratum that induce haptotaxis, durotaxis, that
involves migration towards regions with an increasing stiffness of
the ECM, electrotaxis, also known as galvanotaxis, that prescribes
a directed motion guided by an electric field or current, or
phototaxis, referring to the movement oriented by a stimulus of light
[34]. Important biophysical cues are some of the properties of the
extracellular matrix (ECM), first among all the alignment of collagen
fibers and its stiffness. In particular, the fiber alignment is shown
to stimulate contact guidance [22, 21]. TL;DR:======================================== SAMPLE 1 ========================================
 it as if they were just objects of the senses. But the same is true of matter, which, when it is in motion, finds its way to the surface and is able to connect the matter with the matter of a thing, and when it
================================================================================
Model prompt >>> ======================================== SAMPLE 1 ========================================
 a point
of view, and with them I may know how far we can go in the direction of
motion.

If we consider this question from the standpoint of the reason of
the transcendent, and do not take it that the
================================================================================
Model prompt >>> ======================================== SAMPLE 1 ========================================
 system which is not yet at the end of its state, and consequently, for a given state, a free choice of the system of which this choice is a part. The whole of its internal contingency, therefore, is not to be found in an
================================================================================
Model prompt >>> ======================================== SAMPLE 1 ========================================
 action are, therefore, only the product of a priori principles, and not of an agent determined by principles.

The principle which determines the law of nature must, therefore, be an action of reason. Thus the understanding, by the rule
================================================================================
Model prompt >>> ======================================== SAMPLE 1 ========================================
 be detected by means of a chemical reaction—the internal sense of smell—a phenomenon we call an
antagonism. Antagonism has the meaning of a phenomenon (of sense) which is not merely
phenomenal, but is itself
================================================================================
Model prompt >>> ======================================== SAMPLE 1 ========================================
 the changes in the growth of the organism that make a change in the organism and so on.

But