error when i modify the max_tokens of the custom Maximum chunk length from 1000 to 2000

[rudadasir]

Self Checks

• This is only for bug report, if you would like to ask a quesion, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

Dify version

0.6.6

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

i have modify the max_tokens to 2000 in my localhost，but error happens below:

✔️ Expected Behavior

to modify the custom fo Maximum chunk length from 1000 to 2000

❌ Actual Behavior

nothing changes,still 1000 ,the Maximum chunk length

[dosubot[bot]]

To address the issue of modifying the max_tokens parameter to allow a maximum chunk length of 2000, you'll need to update the code that enforces the maximum limit. Specifically, locate the validation logic in the BaseIndexProcessor class that restricts max_tokens to a maximum of 1000 and adjust it to your desired limit of 2000. Here's the snippet you need to change:

if segmentation["max_tokens"] < 50 or segmentation["max_tokens"] > 1000:
    raise ValueError("Custom segment length should be between 50 and 1000.")

Modify it to:

if segmentation["max_tokens"] < 50 or segmentation["max_tokens"] > 2000:
    raise ValueError("Custom segment length should be between 50 and 2000.")

After making this change, rebuild your Docker image to include the updated code and redeploy your Dify environment using the updated Docker image. This adjustment should allow the system to accept a max_tokens value up to 2000.

Be mindful of the potential performance or system stability implications when increasing the max_tokens parameter. Higher values can require more computational resources, which might affect performance, especially under heavy load or with limited system resources.

Additionally, ensure that other parts of the Dify codebase that enforce or rely on the max_tokens parameter are also updated to accommodate your new maximum. This includes methods that calculate remaining tokens, recalculate max_tokens, and configuration files that set the max_tokens parameter.

Lastly, consider that tests in the Dify codebase validating the max_tokens parameter might fail or produce unintended results if the max_tokens value is increased without corresponding adjustments in the test expectations or setup.

Details

Sources

• how to modify the maxtokens of custom segment
• api/core/rag/index_processor/index_processor_base.py
• api/core/model_runtime/model_providers/tongyi/llm/qwen-max-longcontext.yaml
• api/core/app/apps/base_app_runner.py
• api/core/model_runtime/model_providers/nvidia/llm/llama3-8b.yaml
• api/tests/unit_tests/core/prompt/test_prompt_transform.py
• api/tests/integration_tests/model_runtime/spark/test_llm.py
• api/core/model_runtime/model_providers/openai_api_compatible/llm/llm.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[rudadasir]

tks,you are so smart@dosu