Add configurability to dropout in MultiHeadSelfAttention Module

[theissenhelen]

Is your feature request related to a problem? Please describe.

Currently, the MultiHeadSelfAttention module has a fixed dropout rate of 0.00, which limits the ability to tune this hyperparameter for different use cases. This lack of configurability can hinder model optimization and performance, especially in scenarios where overfitting may occur due to smaller datasets.

Describe the solution you'd like

I would like to see the addition of a configurable dropout parameter to the MultiHeadSelfAttention module. This parameter should allow users to specify the dropout rate when initialising the module, enabling better customisation and optimization of the model.

Describe alternatives you've considered

No response

Additional context

No response

Organisation

No response