Some weights of the model checkpoint were not used when initializing MGMLlamaForCausalLM

[charlesCXK]

When loading the pretrained ckeckpoint after stage 2, I get the following warning:

Some weights of the model checkpoint at /train_logs/MGM/MGM-7B/mgm_v7b_336_hr_768_stage2 were not used when initializing MGMLlamaForCausalLM: ['model.vision_tower.vision_tower.vision_model.encoder.layers.9.mlp.fc2.bias', 'model.vision_tower.vision_tower.vision_model.encoder.layers.20.self_attn.out_proj.weight', 'model.vision_tower.vision_tower.vision_model.encoder.layers.9.layer_norm2.weight', 'model.vision_tower.vision_tower.vision_model.encoder.layers.7.self_attn.k_proj.bias', 'model.vision_tower_aux.vision_stages.1.blocks.1.norm.weight', 'model.vision_tower_aux.vision_stages.2.blocks.4.mlp.fc1.weight', 'model.vision_tower.vision_tower.vision_model.encoder.layers.11.layer_norm2.bias'

However, when loading the ckeckpoint provided in the github, there is no warning.
Could you give me some advices on this?

Thanks!

[yanwei-li]

Hi, please provide more details to locate the error, like the detailed script and settings for training.

[charlesCXK]

Hi, I am using the official script without modifications (https://github.com/dvlab-research/MGM/blob/main/scripts/llama/train/stage_1_2_full_v7b_336_hr_768.sh).