pretrain and finetune

[github-l-zc]

When you‘re pre-training, shouldn’t you use the fusion of 3 datasets, then fine-tune on each dataset, but the code doesn‘t seem to be implemented that way

[RS2002]

Thank you for the attention to our work.

Your suggestion is a good direction to help improve the model generalization capacity. However, in our paper, we mainly focus on the data scarcity scenario so that we only pre-train on the single dataset by the MLM.

Best Regards