-
Notifications
You must be signed in to change notification settings - Fork 56.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
optional max_tokens #4401
base: main
Are you sure you want to change the base?
optional max_tokens #4401
Conversation
@Algorithm5838 is attempting to deploy a commit to the NextChat Team on Vercel. A member of the Team first needs to authorize it. |
Your build has completed! |
@Algorithm5838 Just letting you know, there is a bug related to the attach messages feature due to the |
You are correct. I encountered it before and solved it by commenting out this part:
The issue with the logic is that they assumed max_tokens is input + output, where it is actually output only. |
I figured that out a few weeks ago when trying to implement support for |
Use a checkbox to optionally enable the use of max_tokens instead of having it disabled. This feature is useful for OpenAI models, as well as models from OpenRouter and other platforms.
I've set the default to
2048
for smaller context models (4k); however,4096
is the preferred setting for newer models from OpenAI and Anthropic. Despite these models supporting much larger contexts, their output is capped at4096
.