About the rich feedback model release

[srymaker]

Thanks for your great work!
Will the rich feedback model be released?I'd love to test and experience the model and apply it to my own tasks！

[leebird]

Hello, currently we don't have the plan to release the model.

[densechen]

@leebird Looking forward to the rich feedback model...

[udrs]

Looking forward

[leebird]

Thanks for all the interests in our work! Due to company policies (related to productization etc.) we could not open source the model. We have included details of how to reproduce the results in our paper. If you have further questions please email the corresponding authors, and we'd be happy to help you reproduce the results.

[srymaker]

Hello, can you tell me how you trained the reward model, like which layers were frozen and which tuning method was used?

[leebird]

Hi @srymaker , we finetuned all the layers in the model, including the ViT component. We tried freezing the ViT component but it didn't work well, especially for the heatmap tasks. Experiment details including hyperparameters and optimizer can be found in Section 9 in the paper.

[srymaker]

Thank you for your answer. Do all layers refer to the encoder and decoder in t5?

[leebird]

@srymaker yes, all the layers are from the ViT and T5 encoder/decoder. Note there is a pretraining stage for the ViT and T5 layers on multimodal data as they were originally pretrained on unimodal data only.

[Nieleilei]

@srymaker yes, all the layers are from the ViT and T5 encoder/decoder. Note there is a pretraining stage for the ViT and T5 layers on multimodal data as they were originally pretrained on unimodal data only.

Hello, For single-modal pretraining tasks, are only the natural image captioning tasks on the WebLI dataset used? What other tasks are included?