torch.nn.Embedding(num_embeddings: int, embedding_dim: int, padding_idx: Optional[int] = None,
max_norm: Optional[float] = None, norm_type: float = 2., scale_grad_by_freq: bool = False,
sparse: bool = False, _weight: Optional[Tensor] = None)
参数：
"""
num_embeddings (int) – 词典的大小尺寸，比如总共出现2000个词，那就输入2000。此时index为（0-1999）
embedding_dim (int) – 嵌入向量的维度，即用多少维来表示一个符号。
padding_idx (int, optional) – 填充id，比如，输入长度为100，但是每次的句子长度并不一样，
后面就需要用统一的数字填充，而这里就是指定这个数字，这样，
网络在遇到填充id时，就不会计算其与其它符号的相关性。（初始化为0）
max_norm (float, optional) – 最大范数，如果嵌入向量的范数超过了这个界限，就要进行再归一化。
norm_type (float, optional) – 指定利用什么范数计算，并用于对比max_norm，默认为2范数。
scale_grad_by_freq (boolean, optional) – 根据单词在mini-batch中出现的频率，对梯度进行放缩。默认为False.
sparse (bool, optional) – 若为True,则与权重矩阵相关的梯度转变为稀疏张量
"""
"""
num_embeddings (int): size of the dictionary of embeddings
embedding_dim (int): the size of each embedding vector
padding_idx (int, optional): If given, pads the output with the embedding vector at :attr:`padding_idx`
(initialized to zeros) whenever it encounters the index.
max_norm (float, optional): If given, each embedding vector with norm larger than :attr:`max_norm`
is renormalized to have norm :attr:`max_norm`.
norm_type (float, optional): The p of the p-norm to compute for the :attr:`max_norm` option. Default ``2``.
scale_grad_by_freq (boolean, optional): If given, this will scale gradients by the inverse of frequency of
the words in the mini-batch. Default ``False``.
sparse (bool, optional): If ``True``, gradient w.r.t. :attr:`weight` matrix will be a sparse tensor.
See Notes for more details regarding sparse gradients.
"""
????????其中较为常用的参数为num_embeddings, embedding_dim
? ? ?torch.nn.Embedding(num_embeddings,embedding_dim)的意思是创建一个词嵌入模型，num_embeddings代表词数量, embedding_dim代表为每个词创建一个多少维的向量来表示。
? ? ? ?用一个简单例子来解释：
import torch
import torch.nn as nn
from torch.autograd import Variable
word_index = {'hello': 0, 'what': 1,'are': 2, 'welcome': 3,'you': 4, 'i': 5}
# 首先创建6个词的词嵌套模型，每个词用10维的向量表示
embeds = nn.Embedding(6, 10)
print('embeds weight:',embeds.weight)
word = [[1, 2, 4],
[4, 2, 3]]
# 表示两个句子,[what are you]和[you are welcome]
embed = embeds(torch.LongTensor(word))
print('embed:',embed)
print('embed size:',embed.size())
hello_idx = torch.LongTensor([word_index['hello']])
hello_idx = Variable(hello_idx)
hello_embed = embeds(hello_idx)
print('hello_embed :',hello_embed)
print('hello_embed size:',hello_embed.size())
输出：
embeds weight: Parameter containing:
tensor([[ 1.4825e+00, 3.0712e-01, -2.3293e-01, -4.1670e-01, 1.2044e+00,
-2.9397e-01, 1.7059e+00, -1.1970e+00, -6.0050e-02, -5.3276e-02],
[ 2.5928e-01, 4.3114e-04, -6.0456e-01, 1.1260e+00, -9.2605e-02,
1.7866e+00, 1.9495e+00, 1.3268e-01, 3.4928e-01, -3.7590e+00],
[-1.2764e+00, -8.0230e-01, 3.6574e-01, -1.1873e+00, -1.4793e+00,
-1.9998e+00, 2.9790e+00, -2.0439e-01, 2.4888e+00, -1.6975e+00],
[-1.7675e-01, -7.6663e-01, -1.3094e-01, -4.7818e-01, -1.4898e+00,
1.0922e+00, -1.2935e-01, -1.8294e+00, -1.1626e+00, 1.2329e+00],
[ 9.8236e-01, 1.2997e+00, -6.3932e-01, 1.1221e+00, -8.2662e-02,
-3.9220e+00, 4.4613e-01, -4.9158e-01, -1.5463e+00, -2.4569e+00],
[-3.3445e-01, 1.7175e+00, 6.3186e-01, 8.1554e-01, -8.2884e-01,
5.4600e-01, 1.2782e-01, 8.5362e-01, -1.1766e+00, 2.5298e-01]],
requires_grad=True)
embed: tensor([[[ 2.5928e-01, 4.3114e-04, -6.0456e-01, 1.1260e+00, -9.2605e-02,
1.7866e+00, 1.9495e+00, 1.3268e-01, 3.4928e-01, -3.7590e+00],
[-1.2764e+00, -8.0230e-01, 3.6574e-01, -1.1873e+00, -1.4793e+00,
-1.9998e+00, 2.9790e+00, -2.0439e-01, 2.4888e+00, -1.6975e+00],
[ 9.8236e-01, 1.2997e+00, -6.3932e-01, 1.1221e+00, -8.2662e-02,
-3.9220e+00, 4.4613e-01, -4.9158e-01, -1.5463e+00, -2.4569e+00]],
[[ 9.8236e-01, 1.2997e+00, -6.3932e-01, 1.1221e+00, -8.2662e-02,
-3.9220e+00, 4.4613e-01, -4.9158e-01, -1.5463e+00, -2.4569e+00],
[-1.2764e+00, -8.0230e-01, 3.6574e-01, -1.1873e+00, -1.4793e+00,
-1.9998e+00, 2.9790e+00, -2.0439e-01, 2.4888e+00, -1.6975e+00],
[-1.7675e-01, -7.6663e-01, -1.3094e-01, -4.7818e-01, -1.4898e+00,
1.0922e+00, -1.2935e-01, -1.8294e+00, -1.1626e+00, 1.2329e+00]]],
grad_fn=<EmbeddingBackward>)
embed size: torch.Size([2, 3, 10])
hello_embed : tensor([[ 1.4825, 0.3071, -0.2329, -0.4167, 1.2044, -0.2940, 1.7059, -1.1970,
-0.0601, -0.0533]], grad_fn=<EmbeddingBackward>)
hello_embed size: torch.Size([1, 10])
????????可以看到embed输出的size是[2,3,10],这表示输入维度为2x3的两个由三个单词组成的句子，其中的每个词由一个10维的向量来表示。单个单词“hello”由一个[1,10]的向量来表示。
|