policy_kwargs not documented in DQN

[pstahlhofen]

📚 Documentation

Currently, the documentation of DQN on the policy_kwargs parameter states the following:

policy_kwargs (Dict[str, Any] | None) – additional arguments to be passed to the policy on creation

A list of policy keyword arguments and their default settings would be very helpful here. If you think that this would take to much space, please at least add a hint to the possible policy classes and document the keyword arguments there. Spending some time reading the source code, I found the following keyword arguments for MlpPolicy:

• net_arch
• activation_fn
• featuers_extractor_class
• features_extractor_kwargs
• normalize_images
• optimizer_class
• optimizer_kwargs

For other people interested in customizing those parameters: In dqn/policies.py you will find the relevant code for DQN. This can be used as a starting point as long as the documentation is not complete.

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation

[araffin]

you mean linking https://stable-baselines3.readthedocs.io/en/master/modules/dqn.html#dqn-policies which is just below https://stable-baselines3.readthedocs.io/en/master/modules/dqn.html#parameters ?

[pstahlhofen]

Okay, I agree that it's not completely unfindable. But yes, I still think a link would be helpful here.

[araffin]

Could you do a PR that links the two ? (using rst command :ref: probably and for all algorithms)

[pstahlhofen]

I've never done anything with rst before. I think it would be better if someone could do it who has experience with that

[pstahlhofen]

I have never worked with rst before, so maybe someone more experienced could do it

[kplers]

Hi @pstahlhofen,

I worked with this since it was simple enough, so I would like to submit a PR for this.
Is it OK?
Thank you.

[pstahlhofen]

Yes, sure! Thank you :)