Adding test cases for the bugs found, local_rank None and output dir … #343
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes 2 bugs introduced on #220:
Created new tests to reproduce these errors and fixed them
Testing
Running unit tests and running finetuning for an epoch:
Tests:
Finetuning command:
python -m llama_recipes.finetuning --use_peft --peft_method lora --quantization --model_name ../llama/models_hf/7B/ --output_dir ../llama/models_ft/Llama-2-7b-peft-samsum-1 --batch_size_training 2 --gradient_accumulation_steps 2 --num_epochs 1 --save_model --save_metrics --dataset samsum_dataset
Before submitting
Pull Request section?
to it if that's the case.