|
focal loss参考
https://blog.csdn.net/u011583927/article/details/90716942
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("/media/liao/My Passport/MNIST/", one_hot=True)
def multi_category_focal_loss1(y_true, y_pred):
epsilon = 1.e-7
gamma = 1.0
alpha = tf.constant([[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]], dtype=tf.float32)
y_true = tf.cast(y_true, tf.float32)
y_pred = tf.clip_by_value(y_pred, epsilon, 1. - epsilon)
y_t = tf.multiply(y_true, y_pred) + tf.multiply(1-y_true, 1-y_pred)
ce = -tf.log(y_t)
weight = tf.pow(tf.subtract(1., y_t), gamma)
fl = tf.matmul(tf.multiply(weight, ce), alpha)
loss = tf.reduce_mean(fl)
return loss
x = tf.placeholder(tf.float32, [None, 784])
y_ = tf.placeholder(tf.float32, [None, 10])
weight = tf.Variable(tf.zeros([784, 10]))
bias = tf.Variable(tf.zeros([10]))
y = tf.matmul(x, weight) + bias
y_softmax=tf.nn.softmax(y)
focal_loss=multi_category_focal_loss1(y_,y_softmax)
cross_entropy=focal_loss
train_op = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
with tf.Session() as sess:
tf.global_variables_initializer().run()
validate_feed = {x: mnist.validation.images, y_: mnist.validation.labels}
test_feed = {x: mnist.test.images, y_: mnist.test.labels}
for i in range(30000):
if i % 1000 == 0:
validate_accuracy = sess.run(accuracy, feed_dict=validate_feed)
print("After %d trainging step(s) ,validation accuracy"
"using average model is %g%%" % (i, validate_accuracy * 100))
f_loss=sess.run(focal_loss,feed_dict=validate_feed)
print(f_loss,f_loss.shape)
xs, ys = mnist.train.next_batch(100)
sess.run(train_op, feed_dict={x: xs, y_: ys})
test_accuracy = sess.run(accuracy, feed_dict=test_feed)
print("After 30000 trainging step(s) ,test accuracy using average"
" model is %g%%" % (test_accuracy * 100))
After 0 trainging step(s) ,validation accuracyusing average model is 9.14%
(2.1671617, ())
After 1000 trainging step(s) ,validation accuracyusing average model is 59.38%
(7.243082, ())
After 2000 trainging step(s) ,validation accuracyusing average model is 57.86%
(7.5029922, ())
After 3000 trainging step(s) ,validation accuracyusing average model is 62.8%
(7.0280437, ())
After 4000 trainging step(s) ,validation accuracyusing average model is 61.74%
(6.5108142, ())
After 5000 trainging step(s) ,validation accuracyusing average model is 58.18%
(7.4595795, ())
After 6000 trainging step(s) ,validation accuracyusing average model is 59.26%
(7.498939, ())
After 7000 trainging step(s) ,validation accuracyusing average model is 59.78%
(7.1583071, ())
After 8000 trainging step(s) ,validation accuracyusing average model is 60.56%
(7.7969065, ())
After 9000 trainging step(s) ,validation accuracyusing average model is 59.78%
(8.9209671, ())
After 10000 trainging step(s) ,validation accuracyusing average model is 59.32%
(7.4151506, ())
After 11000 trainging step(s) ,validation accuracyusing average model is 50.9%
(10.775988, ())
After 12000 trainging step(s) ,validation accuracyusing average model is 62.74%
(7.0292954, ())
After 13000 trainging step(s) ,validation accuracyusing average model is 57.54%
(7.7753515, ())
After 14000 trainging step(s) ,validation accuracyusing average model is 62.86%
(6.4984117, ())
After 15000 trainging step(s) ,validation accuracyusing average model is 56.66%
(9.2939491, ())
After 16000 trainging step(s) ,validation accuracyusing average model is 56.52%
(7.5239344, ())
After 17000 trainging step(s) ,validation accuracyusing average model is 60.24%
(7.5586343, ())
After 18000 trainging step(s) ,validation accuracyusing average model is 61.98%
(6.5576291, ())
After 19000 trainging step(s) ,validation accuracyusing average model is 61.1%
(7.1839633, ())
After 20000 trainging step(s) ,validation accuracyusing average model is 61.98%
(7.9907374, ())
After 21000 trainging step(s) ,validation accuracyusing average model is 62.2%
(6.2956591, ())
After 22000 trainging step(s) ,validation accuracyusing average model is 58.12%
(6.9647093, ())
After 23000 trainging step(s) ,validation accuracyusing average model is 58.6%
(7.16395, ())
After 24000 trainging step(s) ,validation accuracyusing average model is 55.66%
(7.9839039, ())
After 25000 trainging step(s) ,validation accuracyusing average model is 59.2%
(8.3046751, ())
After 26000 trainging step(s) ,validation accuracyusing average model is 61.86%
(6.7330093, ())
After 27000 trainging step(s) ,validation accuracyusing average model is 62.24%
(6.7317719, ())
After 28000 trainging step(s) ,validation accuracyusing average model is 59.66%
(8.54006, ())
After 29000 trainging step(s) ,validation accuracyusing average model is 61.54%
(6.5660267, ())
After 30000 trainging step(s) ,test accuracy using average model is 60.31%
好像不如原本使用softmax交叉商损失函数的情况，如下
After 0 trainging step(s) ,validation accuracyusing average model is 9.14%
After 1000 trainging step(s) ,validation accuracyusing average model is 77.88%
After 2000 trainging step(s) ,validation accuracyusing average model is 81.52%
After 3000 trainging step(s) ,validation accuracyusing average model is 82.76%
After 4000 trainging step(s) ,validation accuracyusing average model is 79.52%
After 5000 trainging step(s) ,validation accuracyusing average model is 73.92%
After 6000 trainging step(s) ,validation accuracyusing average model is 81.64%
After 7000 trainging step(s) ,validation accuracyusing average model is 83.04%
After 8000 trainging step(s) ,validation accuracyusing average model is 76.26%
After 9000 trainging step(s) ,validation accuracyusing average model is 76.14%
After 10000 trainging step(s) ,validation accuracyusing average model is 73.8%
After 11000 trainging step(s) ,validation accuracyusing average model is 84.16%
After 12000 trainging step(s) ,validation accuracyusing average model is 81.18%
After 13000 trainging step(s) ,validation accuracyusing average model is 82.68%
After 14000 trainging step(s) ,validation accuracyusing average model is 78.26%
After 15000 trainging step(s) ,validation accuracyusing average model is 80.88%
After 16000 trainging step(s) ,validation accuracyusing average model is 76.52%
After 17000 trainging step(s) ,validation accuracyusing average model is 84.26%
After 18000 trainging step(s) ,validation accuracyusing average model is 77.52%
After 19000 trainging step(s) ,validation accuracyusing average model is 71.3%
After 20000 trainging step(s) ,validation accuracyusing average model is 77.08%
After 21000 trainging step(s) ,validation accuracyusing average model is 78.76%
After 22000 trainging step(s) ,validation accuracyusing average model is 84.44%
After 23000 trainging step(s) ,validation accuracyusing average model is 82.98%
After 24000 trainging step(s) ,validation accuracyusing average model is 83.28%
After 25000 trainging step(s) ,validation accuracyusing average model is 80.84%
After 26000 trainging step(s) ,validation accuracyusing average model is 81.56%
After 27000 trainging step(s) ,validation accuracyusing average model is 69.84%
After 28000 trainging step(s) ,validation accuracyusing average model is 83.28%
After 29000 trainging step(s) ,validation accuracyusing average model is 77.96%
After 30000 trainging step(s) ,test accuracy using average model is 80.59%
Process finished with exit code 0
|