526互联

强化学习中子进程调用atari游戏是否受父进程中设置的随机种子影响

发布时间 2023-09-11 14:42:45作者: Angry_Panda

相关：

python中numpy.random.seed设置随机种子是否影响子进程

 

 

============================================

 

代码：

from ale_python_interface import ALEInterface
import numpy as np
np.random.seed(1111)
import cv2
import time

filename = "atari_roms" + "/" + "pong" + ".bin"

ale_int = ALEInterface()
ale_int.setInt(b"random_seed", 1111)
ale_int.setFloat(b"repeat_action_probability", 0.0)
ale_int.setInt(b"frame_skip", 1)
ale_int.setBool(b"color_averaging", False)
ale_int.loadROM(str.encode(filename))
num_actions = len(ale_int.getMinimalActionSet())
legal_actions = ale_int.getMinimalActionSet()
h, w = ale_int.getScreenDims()
gray_screen = np.zeros((h, w, 1), dtype=np.uint8)


ale_int.reset_game()

pre_screen = None
for i in range(30):
    ale_int.act(legal_actions[0])
    ale_int.getScreenGrayscale(gray_screen)

for i in range(1000):
    pre_screen = np.copy(gray_screen)
    ale_int.act(legal_actions[np.random.randint(len(legal_actions))])
    ale_int.getScreenGrayscale(gray_screen)
    # cv2.imshow("Example Image", gray_screen)
    # time.sleep(0.01)

    # print(gray_screen)
    det = np.sum(pre_screen - gray_screen)
    print(det, ale_int.game_over())

 

运行结果：

 

 

=============================

 

对比代码：

from multiprocessing import Process

from ale_python_interface import ALEInterface
import numpy as np
np.random.seed(1111)
# import cv2
# import time

filename = "atari_roms" + "/" + "pong" + ".bin"

ale_int = ALEInterface()
ale_int.setInt(b"random_seed", 1111)
ale_int.setFloat(b"repeat_action_probability", 0.0)
ale_int.setInt(b"frame_skip", 1)
ale_int.setBool(b"color_averaging", False)
ale_int.loadROM(str.encode(filename))
num_actions = len(ale_int.getMinimalActionSet())
legal_actions = ale_int.getMinimalActionSet()
h, w = ale_int.getScreenDims()
gray_screen = np.zeros((h, w, 1), dtype=np.uint8)

ale_int.reset_game()


class NN(Process):
    def __init__(self, id, ale):
        super(NN, self).__init__()
        self.id = id
        self.ale = ale
    def run(self):
        super(NN, self).run()

        ale_int = self.ale


        num_actions = len(ale_int.getMinimalActionSet())
        legal_actions = ale_int.getMinimalActionSet()
        h, w = ale_int.getScreenDims()
        gray_screen = np.zeros((h, w, 1), dtype=np.uint8)

        pre_screen = None
        for i in range(30):
            ale_int.act(legal_actions[0])
            ale_int.getScreenGrayscale(gray_screen)

        for i in range(1000):
            pre_screen = np.copy(gray_screen)
            ale_int.act(legal_actions[np.random.randint(len(legal_actions))])
            ale_int.getScreenGrayscale(gray_screen)
            # cv2.imshow("Example Image", gray_screen)
            # time.sleep(0.01)

            # print(gray_screen)
            det = np.sum(pre_screen - gray_screen)
            print(det, ale_int.game_over())


ps = [NN(i, ale_int) for i in range(1)]
for p in ps:
    p.start()

for p in ps:
    p.join()

 

运行结果：

 

 

PS:

可以看到，在python中子进程生成时会copy父进程中的对象，哪怕是atari游戏这种调用C语言扩展模块的对象也会被copy状态给子进程，这个和其他python中对象一样；这个特点和python中numpy.random.seed设置随机种子是否影响子进程相一致。

 

 

 

注意，上面代码中设置numpy和atari游戏的随机种子状态的代码为：

 

 

 

 

 

============================================