数据准备

在 http://mng.bz/0tIo 下载IMDB数据集（57.9MB）并解压。在 \acllmdb\train 路径下，\neg 文件夹中存有 12500 个对电影的负面评价txt， \pos 文件夹中存有 12500 个对电影的正面评价txt.

读取所有txt文件中的文本保存到变量 texts 中。并匹配对应的标签 labels，0代表负面，1代表正面。

import numpy as np
import tensorflow as tf
import tensorflow.keras as keras
import os
origin_dir = 'D:\\Python Projects\\Neural Network\\RNN\\IMDB movie reviews'
train_dir = origin_dir + '\\aclImdb\\train'
test_dir  = origin_dir + '\\aclImdb\\test'
texts = []
labels = []
for fname in os.listdir(train_dir+'\\neg'):
    with open(train_dir+'\\neg\\'+fname,'r',encoding='utf8') as f:
        texts.append(f.read())
    labels.append(0)
for fname in os.listdir(train_dir+'\\pos'):
    with open(train_dir+'\\pos\\'+fname,'r',encoding='utf8') as f:
        texts.append(f.read())
    labels.append(1)

数据预处理

分词与编码：用 keras.preprocessing.text.Tokenizer 匹配文本 texts. 再用 tokenizer 将文本列表转化为数字列表 sequences（列表中的每个元素都是由整数构成的列表）。

word_index 是将单词对应到整数的字典：{‘the’:1, ‘and’:2, ‘a’:3, … }

再用 keras.preprocessing.pad_sequences 将列表中每个元素都变成长为 500 的整数列表。返回值是 ndarray，形状为 (25000,500)

maxlen = 500   # 每条评论至多看前 500 个词
max_words = 10000 # 所有文本只考虑出现频率最多的 10000 个单词。

tokenizer = keras.preprocessing.text.Tokenizer(num_words = max_words)
tokenizer.fit_on_texts(texts)
sequences = tokenizer.texts_to_sequences(texts)
word_index = tokenizer.word_index
data = keras.preprocessing.sequence.pad_sequences(sequences,maxlen = maxlen)

打乱：由于原本的数据中所有neg在前面，pos在后面，并不随机，故需要随机打乱。

indices = np.arange(data.shape[0])
np.random.shuffle(indices)
data = data[indices]
labels = np.array(labels)[indices]

模型训练

简单地建立一个Sequential模型，由三层构成：Embedding, LSTM, Dense.

模型输入参数形状是 (batch_size, 500)，表示由 batch_size 个长度500的整数向量构成。
Embedding层的设定参数是(10000,50)，表示将10000个整数映射到10000个长度为50的向量。
LSTM层表示输出50个值。
Dense全连接层用sigmoid激活函数输出一个值，将其二分类。

训练模型10个epoch，batch_size=32，用时882.9s

model = keras.models.Sequential()
model.add(keras.layers.Embedding(max_words,50))
model.add(keras.layers.LSTM(50))
model.add(keras.layers.Dense(1,activation='sigmoid'))
model.summary()

model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['acc'])

history = model.fit(data,labels,epochs=10,batch_size=32,validation_split=0.2)

model.save(origin_dir + '\\IMDB movie review predictor 1.h5')

训练效果

训练效果可视化

利用 model.fit 的返回值的 history 来获得训练过程的acc和loss。

import matplotlib.pyplot as plt
acc = history.history['acc']
val_acc = history.history['val_acc']
plt.plot(np.arange(1,len(acc)+1), acc)
plt.plot(np.arange(1,len(acc)+1), val_acc)
plt.legend(['acc','val_acc'])
plt.title('Accuracy (Non-pretrained embedding & LSTM)')
plt.figure()

loss = history.history['loss']
val_loss = history.history['val_loss']
plt.plot(np.arange(1,len(loss)+1), loss)
plt.plot(np.arange(1,len(val_loss)+1), val_loss)
plt.legend(['loss','val_loss'])
plt.title('Loss (Non-pretrained embedding & LSTM)')
plt.show()

在这里插入图片描述

可见在训练集上准确率几乎不断上升，接近98%，且损失值几乎不断下降。但在验证集上几乎相反，最终的验证集准确率约83%。

实验测试

如下在 test 中选取了一段正面评价和一段负面评价，交给模型预测。（为了不让一行代码太长，已经对文本手工分行处理。）

同上对文本进行数字编码（tokenizer.texts_to_sequences）和选取前500个词（keras.preprocessing.sequence.pad_sequences）操作，

再将处理完的结果交给 model.predict

sample = ["""My boyfriend and I went to watch The Guardian.At first I didn't want to watch it, but I loved the movie-
 It was definitely the best movie I have seen in sometime.They portrayed the USCG very well, 
 it really showed me what they do and I think they should really be appreciated more. 
 Not only did it teach but it was a really good movie. The movie shows what the really 
 do and how hard the job is.I think being a USCG would be challenging and very scary. 
 It was a great movie all around. I would suggest this movie for anyone to see.
 The ending broke my heart but I know why he did it. The storyline was great I give it 2 thumbs up.
  I cried it was very emotional, I would give it a 20 if I could!""",

"""I was completely bored with this film, melodramatic for no apparent reason.
   Every thing just becomes so serious and people are swearing with really dumb expressions.
   Then there is a serial Killer who apparently can Kill one person to get the title of serial Killer.
   Well the serial Killer likes butterflies and is illustrated by sound effects you might hear in the dream sequence of most modern films;<br /><br />why oh why? 
   I nave no idea. It really really wants to be scary, but I think in this universe scary equals talk a whole bunch and 
   add dark ambient noises.Just for the record, this is in no way is a horror film, its most definitely a thriller (barely).
    Really movie makers nowadays need to do their homework before making "horror" films or 
    at least calling a movie a "horror" film. it makes me say (in too may words ironically) "acolytes,
    you take forever to say nothing."""]

sample_conversion = keras.preprocessing.sequence.pad_sequences(tokenizer.texts_to_sequences(sample),maxlen = maxlen)
print(model.predict(sample_conversion))

输出结果为：
[[0.97874266]
[0.08455005]]
第一个结果接近1，说明预测为正面评价；第二个结果接近0，说明预测为负面评价，与实际相符。

Embedding可视化

提取模型第一层（Embedding层）的权重。注意单词表 word_index 中第一个单词’the’对应的整数为1，所以权重矩阵第零行（不对应任何实际单词）是要舍弃的。

将舍弃第零行的权重矩阵（9999*50）写入一个 tsv文件中。再在另一个tsv文件中写入前9999个单词。

embedding_weights = model.layers[0].get_weights()
print(embedding_weights[0].shape) # (10000,50)
with open('D:\\Python Projects\\Neural Network\\RNN\\IMDB movie reviews\\embedding_words1.tsv','w',encoding='utf-8') as f:
    for word in list(word_index.keys())[:max_words-1]:
        f.write(word+'\n')
with open('D:\\Python Projects\\Neural Network\\RNN\\IMDB movie reviews\\embedding_vectors1.tsv','w',encoding='utf-8') as f:
    for line in embedding_weights[0][1:]:
        f.write('\t'.join(str(x) for x in line) + '\n')