@Author：Runsen
文章目录
本次博客将介绍如何在Keras中，对文本进行处理Text preprocessing
Text preprocessing
- Keras API
- 文档: https://keras.io/preprocessing/text/
from tensorflow.keras.preprocessing.text import Tokenizer, text_to_word_sequence, one_hot
from tensorflow.keras.preprocessing.sequence import pad_sequences
Tokenization of a sentence
- Tokenizatio：将字符序列转换成符号序列的过程 (https://en.wikipedia.org/wiki/Lexical_analysis#Token)
sentences = ['Curiosity killed the cat.', 'But satisfaction brought it back']
tk = Tokenizer() # create Tokenizer instance
tk.fit_on_texts(sentences) # tokenizer should be fit with text data in advance
-
文本建模的一种简单方法是为每个句子创建整数序列
-
通过这样做，可以保留有关单词顺序的信息
seq = tk.texts_to_sequences(sentences)
print(seq)
[[1, 2, 3, 4], [5, 6, 7, 8, 9]]
One-hot encoding
有时，只需要检查某个词是否出现在句子中，这种文本处理的方式被称为“一One-hot encoding ”。
- 如果单词出现在句子中，它被编码为“1”
- 如果不是，则编码为“0”
mat = tk.sequences_to_matrix(seq)
print(mat)
[[0. 1. 1. 1. 1. 0. 0. 0. 0. 0.]
[0. 0. 0. 0. 0. 1. 1. 1. 1. 1.]]
Padding sequences
- 通常，为了保持句子的维数，会执行zero padding
- Padding 就像填充图像格式，但适用于序列
# if set padding to 'pre', zeros are appended to start of sentences
pad_seq = pad_sequences(seq, padding='pre')
print(pad_seq)
[[0 1 2 3 4]
[5 6 7 8 9]]
、pad_seq = pad_sequences(seq, padding='post')
print(pad_seq)
[[1 2 3 4 0]
[5 6 7 8 9]]
Word Embeddings
- 另一种单词嵌入的方法是通过在word embedding中预先训练单词向量（例如word2vec、GloVe、fasttext等）
Word vectors
单词嵌入是将每个单词转换成一个固定维度（单词）向量的过程。嵌入空间（即向量空间）的维数是一个超参数；可以将维数设置为任何正整数
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import *
from tensorflow.keras.datasets import reuters
from tensorflow.keras.preprocessing import sequence
from tensorflow.keras.utils import to_categorical
Embedding layer
因此，Embedding layer具有三维张量
- Output 输出形状= (batch_size, input_length, output_dim)
- input_dim 输入尺寸：输入空间的维数（感兴趣的唯一标记的数量）
- output_dim 输出尺寸：嵌入空间的维数
- input_length 输入长度：输入序列的长度（如果没有，可以改变）
#输入长度不变时
model = Sequential()
model.add(Embedding(input_dim = 10, output_dim = 5, input_length = 3))
model.output_shape
(None, 3, 5)
# when input length varies
model = Sequential()
model.add(Embedding(input_dim = 10, output_dim = 5, input_length = None))
model.output_shape
(None, None, 5)
在model中，通常采用 embedding layer作为第一层对文本格式的数据进行建模
# parameters to import dataset
num_words = 3000
maxlen = 50
(X_train, y_train), (X_test, y_test) = reuters.load_data(num_words = num_words, maxlen = maxlen)
X_train = sequence.pad_sequences(X_train, maxlen = maxlen, padding = 'post')
X_test = sequence.pad_sequences(X_test, maxlen = maxlen, padding = 'post')
y_train = to_categorical(y_train, num_classes = 46)
y_test = to_categorical(y_test, num_classes = 46)
print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)
(1595, 50)
(399, 50)
(1595, 46)
(399, 46)
input_dim = num_words
output_dim = 100 # we set dimensionality of embedding space as 100
input_length = maxlen
def reuters_model():
model = Sequential()
model.add(Embedding(input_dim = input_dim, output_dim = output_dim, input_length = input_length))
model.add(GRU(50, return_sequences = False))
model.add(Dense(100))
model.add(Activation('relu'))
model.add(Dense(46, activation = 'softmax'))
model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
return model
model = reuters_model()
model.summary()
model.fit(X_train, y_train, epochs = 10, batch_size = 256, verbose = 1)
result = model.evaluate(X_test, y_test)
print('Test Accuracy: ', result[1])
Test Accuracy: 0.7468671798706055