Change size to 30 and agent is unable to reach the goal

[daniel-xion]

Hi, thanks for the code. I changed the grid size to 30 and it seems the agent is unable to learn to reach the goal:

It seems Q-learning is unable to handle large grids, and DQN is needed to solve it?

[qihongl]

thanks for the datapoint daniel!

I think the epoch number needs to be much much larger since the state space increased 36 times (30 x 30) / (5 x 5). q learning requires an agent with a random policy to bump into the target many times. This "target visitation probability" exponentially decreases as the state space gets larger.

[daniel-xion]

Thanks for the prompt reply! Do you suggest, for large grids, and to be scalable (including reasonable training episodes), do you think DQN or other reinforcement learning will work?

[daniel-xion]

FYI, I tried DQN from the following notebook and it is still not working. The agent keep bumping into the walls (even with the size of only 12):
DeepReinforcementLearning/DeepReinforcementLearningInAction#38

[qihongl]

Thanks for the datapoint -- that's really interesting. I'm not sure what's the next simplest thing to try...

[daniel-xion]

No problem. By the way, what do you think of using transformer in the RL to solve the maze game? for example, by using the transformer to predict a sequence of actions that leads to a sequence of high rewards?