You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, Great work!
I was wondering what is the difference between different model files, like bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt, mp_rank_00_model_states.pt, pytorch_model.bin which is obtained after the zero_to_fp32.py, and the file obtained after merge_lora_weights_and_save_hf_model.py.
Which of them is the lora weight? or they are just different formats of lora weight?
Thanks!
The text was updated successfully, but these errors were encountered:
I have met the same problem.
If you train with deepspeed, save the model and run python zero_to_fp32.py, you will get the pytorch_model.bin. The pytorch_model.bin is the full model weight which means there is no need to merge.
Hi, Great work!
I was wondering what is the difference between different model files, like
bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt
,mp_rank_00_model_states.pt
,pytorch_model.bin
which is obtained after thezero_to_fp32.py
, and the file obtained aftermerge_lora_weights_and_save_hf_model.py
.Which of them is the lora weight? or they are just different formats of lora weight?
Thanks!
The text was updated successfully, but these errors were encountered: