--freeze_backbone False?

[fyting]

Why does the file internvl_chat_v1_2_hermes2_yi34b_448_finetune.sh include --freeze_backbone False? Isn't the visual encoder supposed to be frozen during the pre-training phase?

[czczup]

Hello, this is the fine-tuning script. When we fine-tune, we open the entire model and train it.

[daiqing98]

Hello, this is the fine-tuning script. When we fine-tune, we open the entire model and train it.

May I ask if you do any ablation on this? Does open the vision module bring any notable benefits?

[czczup]

Yes, in my experiments, turning on the vision encoder was significantly better than freezing it, so in all recent experiments, I have turned on the vision encoder during the finetune phase.

For these hyperparameters, which modules are trained and which modules are frozen at each stage, you can find out on our blog: https://internvl.github.io/blog/