python - 为什么 TensorFlow matmul() 比 NumPy multiply() 慢得多？

Question

在下面的python代码中，为什么numpy乘法的时间比tensorflow小很多？

import tensorflow as tf
import numpy as np
import time
size=10000
x = tf.placeholder(tf.float32, shape=(size, size))
y = tf.matmul(x, x)

with tf.Session() as sess:
  rand_array = np.random.rand(size, size)

  start_time = time.time()
  np.multiply(rand_array,rand_array)
  print("--- %s seconds numpy multiply ---" % (time.time() - start_time))

  start_time = time.time()
  sess.run(y, feed_dict={x: rand_array})
  print("--- %s seconds tensorflow---" % (time.time() - start_time))

输出是

--- 0.22089099884 seconds numpy multiply ---
--- 34.3198359013 seconds tensorflow---

Answer 1

嗯，引用文档:

numpy.multiply(x1, x2[, out]) = Multiply arguments element-wise.

和

tf.matmul(a, b, transpose_a=False, transpose_b=False, a_is_sparse=False, b_is_sparse=False, name=None)

Multiplies matrix a by matrix b, producing a * b.

The inputs must be two-dimensional matrices, with matching inner dimensions, possibly after transposition.

建议您比较不同的操作:O(n^2) 逐点乘法和 O(n^3) 矩阵乘法。我将测试更正为在两种情况下都使用矩阵乘法 2 次:

import tensorflow as tf
import numpy as np
import time
size=2000
x = tf.placeholder(tf.float32, shape=(size, size))
y = tf.matmul(x, x)
z = tf.matmul(y, x)

with tf.Session() as sess:
  rand_array = np.random.rand(size, size)

  start_time = time.time()
  for _ in xrange(10):
      np.dot(np.dot(rand_array,rand_array), rand_array)
  print("--- %s seconds numpy multiply ---" % (time.time() - start_time))

  start_time = time.time()
  for _ in xrange(10):
      sess.run(z, feed_dict={x: rand_array})
  print("--- %s seconds tensorflow---" % (time.time() - start_time))

得到结果:

--- 2.92911195755 seconds numpy multiply ---
--- 0.32932305336 seconds tensorflow---

使用快速 GPU (gtx 1070)。