Multimodality Training Dataset

[amitbcp]

Hi,

Thanks for open-sourcing the work and paper. Wanted to understand the following -

1. For image modality and interleaved data which datasets did you use or have they been open sourced ?
2. Will the training script remain the same as mentioned in the readme ?

[kongds]

Thank you for your interest in our work.

For the first question, we use CIRR and FashionIQ. You can obtain the results by running the following command (which will download the dataset and evaluate it):

#Evaluate on COCO, Flickr30k, FashionIQ, and CIRR
accelerate launch --num_machines=1 --num_processes 8 --machine_rank 0 retrieval.py --use_e5v

For the second question, we use the same training script as outlined in the README.

[amitbcp]

Thanks for the help