rewardGym + PsychoPy

The toolbox is build in such a way, that it can also be used from a standalone installation of PsychoPy (tested only at v2023.2.3 and higher).

Showing stimuli with PsychoPy is recommended for data collection for multiple reasons.

  1. The installation of PsychoPy is often times easier than setting up a new Python environment, especially on different stimulus computers.

  2. PsychoPy has been shown to provide a reasonable and accurate report of stimulus and response timings.

  3. PsychoPy is probably better known in the cognitive and psychology communities.

Adding stimulus representations

The main purpose of providing support for PsychoPy is that rewardGym should also be used for data collection.

Before writing your own loop and functions, I recommend checking out the rewardgym_psychopy.py file in the root directory.

Adding stimuli works in principle in the same way as in the pygame rendering example by creating an info_dict, which has entries for every node in the graph. However, this time using “psychopy” as the key. Same as before, we are also using a list to collate different stimulus representations.

info_dict = {0: {"pychopy": []}}

rewardGym has some of pre-specified stimulus classes (see the API documentation for more details).

We are going to use four classes for our example from before:

  1. BaseStimulus()

  2. TextStimulus()

  3. ActionStimulus()

  4. FeedBackStimulus()

The BaseStimulus() is simply a flip / update of the window, which could for example be used to clear the screen again. TextStimulus() and FeedBackStimulus() are text stimuli, where the latter is specially designed to be used for reward feedback. ActionStimulus(), finally indicates that the program should wait for a user action.

Full example

You can find the full worked example under notebooks/psychopy_example.py.

To run it in PsychoPy copy the file to the root directory (it won’t work otherwise!).

  1"""
  2Example code for using psychopy + rewardGym.
  3As a set up create a copy of this code and move it into the root directory of
  4the rewardGym folder.
  5Then open the file in PsychoPy coder. Note, that it requires a recent
  6PsychoPy version (tested on at least v2023.2.3).
  7"""
  8
  9from psychopy import core, event, visual
 10
 11from rewardgym.environments import PsychopyEnv
 12from rewardgym.psychopy_render.logger import MinimalLogger
 13from rewardgym.psychopy_render.stimuli import (
 14    ActionStimulus,
 15    BaseStimulus,
 16    FeedBackStimulus,
 17    TextStimulus,
 18)
 19from rewardgym.reward_classes import BaseReward
 20
 21# As in the pygame example, we create the graph, rewards and their locations:
 22env_graph = {0: [1, 2], 1: [], 2: []}
 23
 24reward1 = BaseReward(reward=[0, 1], p=[0.2, 0.8], seed=2222)
 25reward2 = BaseReward(reward=[0, 1], p=[0.5, 0.5], seed=3333)
 26
 27# This creates the reward dictionary necessary for the environment.
 28reward_locs = {1: reward1, 2: reward2}
 29
 30# We then create the environment. Note that we use a PsychopyEnv.
 31env = PsychopyEnv(environment_graph=env_graph, reward_locations=reward_locs)
 32
 33# We then creat the stimuli we want to display. Here we use a very minimal example
 34# as in the pygame example:
 35
 36# This is just a flip of the screen (clear everything that is on there)
 37flip_screen = BaseStimulus(duration=0)
 38# Then we put a fixatoin cross on the screen for 0.2 s (note the different time units for pygame (ms) and psychopy (s))
 39fixation = TextStimulus(duration=0.2, text="+", position=(0, 0), name="text1")
 40# Another TextStimulus to show the choic
 41selection = TextStimulus(duration=0.001, text="A or B", position=(0, 0), name="text2")
 42# Finally, we require an action, which is done using the ActionStimulus.
 43action = ActionStimulus(duration=10, key_dict={"left": 0, "right": 1})
 44
 45# In the end we create some reward feedback.
 46reward = FeedBackStimulus(
 47    duration=1.0, text="You gain: {0}", position=(0, 0), target="reward"
 48)
 49earnings = FeedBackStimulus(
 50    duration=0.5, text="You have gained: {0}", position=(0, 0), target="total_reward"
 51)
 52
 53# We then augment the nodes in the info dict with the stimuli.
 54info_dict = {
 55    0: {"psychopy": [flip_screen, fixation, selection, action]},
 56    1: {"psychopy": [flip_screen, reward, earnings]},
 57    2: {"psychopy": [flip_screen, reward, earnings]},
 58}
 59
 60env.info_dict.update(info_dict)
 61
 62
 63# For psychopy we need to create a window to render the stimuli onto.
 64win = visual.Window(
 65    size=[1680, 1050],
 66    fullscr=False,
 67    color=[-0.5, -0.5, -0.5],
 68    units="pix",
 69)
 70
 71# As the stimuli require a logger object, we create on here. Note that this
 72# is the MinimalLogger class, that does not write anything to file, and is
 73# just used for compatibility with the stimulus classes (see the main psychopy
 74# code in the root dir for advanced logging).
 75Logger = MinimalLogger(global_clock=core.Clock())
 76Logger.create()
 77
 78# Finally, the stimuli need to be setup (associating and creating the psychopy
 79# objects that need a window object). The pyschopy environment has a method for that:
 80env.setup_render(window=win, logger=Logger)
 81
 82# We also show some instructions here:
 83instruction = visual.TextStim(
 84    win=win, text="Hi\nWellcome to the example", color=[1, 1, 1]
 85)
 86
 87instruction.draw()
 88win.flip()
 89event.waitKeys()
 90win.flip()
 91
 92# The main loop to display the experiment.
 93n_episodes = 5  # Change to a higher number, disabled here for rendering.
 94
 95for episode in range(n_episodes):
 96    obs, info = env.reset(agent_location=0)
 97    done = False
 98
 99    while not done:
100        # The environment stores the action (and the previous action)
101        next_obs, reward, terminated, truncated, info = env.step(env.action)
102
103        done = terminated or truncated
104        obs = next_obs
105
106# And finally closing everything.
107env.close()
108win.close()
109core.quit()