Factored action spaces?

[rdgain]

Reading a paper on this, and considering this as a future development.

We could additionally provide the option to get from the game state a list of action "shells", i.e. all the action types available with no variables assigned (if there are more than 1 option for variables in an action). Then, each action shell would provide:

• a list of valid options for each variable
• a way to correctly construct the action from one value per variable (generic, without need to know the types of the variables, but normal constructors still public as well)

Example: a "Move token" action would provide:

• options for variables:
  • token id: {57, 65, 80}
  • direction: {up, down, left, right}
• constructor with 2 integers representing indeces in the variable options arrays, and assigns the correct values
• can check if the resulting action is valid if it exists in the full list of actions provided normally by the game state

This could allow more granular/hierarchical planning.