Handling 429 error from Azure OpenAI

[cnkang]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing?

Yes, I am experiencing a challenge with the dify tool when using the Azure OpenAI Text Embedding 3 Large API. Specifically, I encountered a 429 error, which suggests that the tool is not handling rate limit responses (HTTP 429) from Azure OpenAI properly.

2. Describe the feature you'd like to see

I would like to see enhanced error handling in dify for HTTP 429 (Too Many Requests) responses from the Azure OpenAI API. Ideally, the tool should implement a retry mechanism that respects the Retry-After header provided by Azure. This feature should manage the request rate or pause the requests for the duration specified in the Retry-After header before retrying.

3. How will this feature improve your workflow or experience?

Implementing robust error handling for rate limit responses will significantly improve the reliability and usability of the tool when interacting with Azure OpenAI services. It will prevent the tool from stopping unexpectedly due to unhandled errors and ensure smoother, more consistent performance during high-frequency API interactions. This improvement will enable users like myself to maintain productivity without manually handling these interruptions.

4. Additional context or comments

No response

5. Can you help us with this feature?

• I am interested in contributing to this feature.

[cnkang]

The screenshot of the error

[cnkang]

我是一次处理了很大量的文本文件到数据库中，触发了429，如果能有遇到429错误等待然后再重试的机制，会对导入大量的文件有帮助。

[crazywoola]

We will introduce the retry failed task later.

[ciekawy]

same for embeedings/reranking APIs