Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Partially frozen model support #11

Open
1 task
scenarios opened this issue Aug 1, 2023 · 2 comments
Open
1 task

[Feature] Partially frozen model support #11

scenarios opened this issue Aug 1, 2023 · 2 comments
Assignees
@SolenoidWGT
Labels
enhancement New feature or request

Comments

@scenarios
Copy link

scenarios commented Aug 1, 2023 •
edited

Describe the feature

Current implementation does not support partially training, i.e., part of the model parameters are frozen.
If I understand it correctly, the assertion around line 584 in hybrid_zero_optim.py suggests that all parameters should be involved in the training.
image

Very looking forward this feature being implemented as there are many scenarios where users only want to finetune parts of the model.

Is there any plan on this feature?

Will you implement it?

  • I would like to implement this feature and create a PR!
@scenarios scenarios added the enhancement New feature or request label Aug 1, 2023
@SolenoidWGT
Copy link
Collaborator

Sure, we will support freeze tuning ASAP.

@scenarios
Copy link
Author

Sure, we will support freeze tuning ASAP.

Hi, any updates here?

@hellock hellock transferred this issue from InternLM/InternLM Jan 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SolenoidWGT SolenoidWGT

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@scenarios @SolenoidWGT