Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update base_llm.py #176

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update base_llm.py #176

wants to merge 1 commit into from

Conversation

seanxuu
Copy link
Contributor

@seanxuu seanxuu commented Mar 21, 2024

Increase code flexibility and backward compatibility

Increase code flexibility and backward compatibility
@Harold-lkk
Copy link
Collaborator

We do not want to support **kwargs in BaseModel, for it wont raise error when wrong init parameter

@seanxuu
Copy link
Contributor Author

seanxuu commented Mar 21, 2024

We do not want to support **kwargs in BaseModel, for it wont raise error when wrong init parameter

In fact, I ran into an issue where if I tried to pass in device_map='auto', it reported an error. But this is a useful parameter in AutoModelForCausalLM.from_pretrained. If there any other solution?

@Harold-lkk
Copy link
Collaborator

we have model_kwargs for HFTransformer model, which will be passed to AutoModelForCausalLM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants