Diversity in beam search

[ayseozgun]

Hello,

I want to create 5 different outputs by using flan-t5-xxl.
Here is my generate code:

"outputs = model.generate(input_ids, max_new_tokens=300,
num_beams=5,
num_return_sequences=5,
num_beam_groups=5,
diversity_penalty=5.0,
no_repeat_ngram_size=2)"
I read num_beam_groups and diversity_penalty parameters should affect the output diversity, but when I run this code, the outputs are still so similar each other. I also tried to change the values of the parameters (like num_beams=8 num_beam_groups=2, 4), still I have diversity problem.

Can you please help me?
Thanks in advance,