Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问为什么在训练llama的脚本中,预训练和微调所使用的conv不一样 #89

Open
shidingz opened this issue Apr 24, 2024 · 1 comment

Comments

@shidingz
Copy link

https://github.com/dvlab-research/MGM/blob/main/scripts/llama/train/stage_1_2_full_v7b_336_hr_768.sh
在这个脚本中pretrain用的--version plain
而finetune用的是--version v1
前后不一致模型不会混乱吗

@yanwei-li
Copy link
Member

Hi, for the LLaMA 7B and 13B, we follow the instruction format in LLaVA. In the pretraining stage, the main focus is image caption. So, it works well with plain style.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants