python - 为什么在 GPU 上创建 Theano 共享变量会影响 numpy 的随机流？

Question

我刚开始玩 Theano，想知道为什么在 gpu 上第一次创建共享变量似乎会影响 numpy 的随机数生成器。有时，这种最初的创造似乎插入了随机数生成器。

我在这段代码中探索了以下测试用例:

import numpy

import theano
from theano.compile.sharedvalue import shared
import theano.sandbox.cuda as tcn

def make_cpu_shared():
    #Create, but don't return/use shared variable on cpu
    shared(theano._asarray(numpy.asarray([.67]), dtype='float32'), 'cpu_shared')
    return None

def make_gpu_shared():
    #Create, but don't return/use shared variable on gpu
    tcn.shared_constructor(theano._asarray(numpy.asarray([.67]), dtype='float32'), 'gpu_shared')
    return None


def rand_test0():
    #Match - Sanity check - Ensure numpy.random.seed creates repeatable random streams
    numpy.random.seed(666) #Note: 666 seems to be seed used in Theano test suite
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

    numpy.random.seed(666)
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

def rand_test1():
    #Match - Show creation of shared variable on cpu has no effect on random streams
    numpy.random.seed(666)
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

    numpy.random.seed(666)
    make_cpu_shared()
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

def rand_test2():
    #No Match - Show creation of shared variable on gpu effects random streams
    numpy.random.seed(666)
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

    numpy.random.seed(666)
    make_gpu_shared()
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

def rand_test3():
    #Match - Show effect is only for initial creation of shared gpu variable
    make_gpu_shared()

    numpy.random.seed(666)
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

    numpy.random.seed(666)
    make_gpu_shared()
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

def rand_test4():
    #No Match - Show initial creation of shared gpu variable effecting random streams
    numpy.random.seed(666)
    make_gpu_shared()    
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

    numpy.random.seed(666)
    make_gpu_shared()
    temp = numpy.random.rand(1)    
    print "temp[0]=",temp[0]

1. rand_test0 - 显示我可以使用 numpy.random.seed 重置随机流的健全性检查
2. rand_test1 - 显示在 cpu 上创建共享变量没有意外
3. rand_test2 - 显示在 gpu 上创建共享变量确实有意想不到的效果
4. rand_test3 - 表明它只是 gpu 上共享变量的初始创建，具有意想不到的效果
5. rand_test4 - rand_test3 的验证

我得到的结果如下:

(Test 0 - Sanity Check)
me@Bedrock1:~/Projects/Theano/packageTests$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_rand_shared import *
>>> rand_test0()
temp[0]= 0.700437121858
temp[0]= 0.700437121858
>>> 

(Test 1 - Shared on CPU OK)
me@Bedrock1:~/Projects/Theano/packageTests$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_rand_shared import *
>>> rand_test1()
temp[0]= 0.700437121858
temp[0]= 0.700437121858
>>> 

(Test 2 - Shared on GPU effects random stream)
me@Bedrock1:~/Projects/Theano/packageTests$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_rand_shared import *
>>> rand_test2()
temp[0]= 0.700437121858
Using gpu device 0: GeForce GTX 670MX
temp[0]= 0.859992279406
>>> 

(Test 3 - Only initial creation of shared variable on GPU effects random stream)
me@Bedrock1:~/Projects/Theano/packageTests$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_rand_shared import *
>>> rand_test3()
Using gpu device 0: GeForce GTX 670MX
temp[0]= 0.700437121858
temp[0]= 0.700437121858
>>> 

(Test 4 - Variation on Test 3)
me@Bedrock1:~/Projects/Theano/packageTests$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from test_rand_shared import *
>>> rand_test4()
Using gpu device 0: GeForce GTX 670MX
temp[0]= 0.859992279406
temp[0]= 0.700437121858
>>> 

这对任何人都有意义吗？它是 Theano 神器吗？它是 CUDA 工件吗，由我最初访问 GPU 引起(即我正在玩共享变量的事实只是我所看到的附带的)。或者，我是不是误会了什么？

Answer 1

Theano 的 documentation讨论播种随机变量的困难以及为什么他们用自己的随机数生成器为每个图实例播种。

Sharing a random number generator between different {{{RandomOp}}} instances makes it difficult to producing the same stream regardless of other ops in graph, and to keep {{{RandomOps}}} isolated. Therefore, each {{{RandomOp}}} instance in a graph will have its very own random number generator. That random number generator is an input to the function. In typical usage, we will use the new features of function inputs ({{{value}}}, {{{update}}}) to pass and update the rng for each {{{RandomOp}}}. By passing RNGs as inputs, it is possible to use the normal methods of accessing function inputs to access each {{{RandomOp}}}’s rng. In this approach it there is no pre-existing mechanism to work with the combined random number state of an entire graph. So the proposal is to provide the missing functionality (the last three requirements) via auxiliary functions: {{{seed, getstate, setstate}}}.

他们还提供examples关于如何为所有随机数生成器播种。

You can also seed all of the random variables allocated by a RandomStreams object by that object’s seed method. This seed will be used to seed a temporary random number generator, that will in turn generate seeds for each of the random variables.

>>> srng.seed(902340)  # seeds rv_u and rv_n with different seeds each

尝试使用 Theano 的种子功能而不是 numpy 的功能来为随机变量设定种子。