Support custom reranker #3779
Replies: 3 comments 1 reply
-
Hey there, @soulteary! Great to see you back with another insightful suggestion. Hope you're doing well! Thank you for your feature request. It seems you're interested in having the ability to set local reranker model calls in Dify. This is indeed a valuable feature that could facilitate the migration of existing pipelines to Dify. I found a similar issue in the repository: 可否增加 bge-reranker 或者接入 local ranker 的功能. This issue suggests adding the bge-reranker or integrating a local ranker, which aligns with your request. However, it's currently closed and the last update was on January 24, 2024. As for implementing this feature, one possible approach could be to extend the functionality of the existing Here's an example of how you might update the from pydantic import BaseModel
from typing import Callable, Optional
class RerankDocument(BaseModel):
"""
Model class for rerank document.
"""
index: int
text: str
score: float
class RerankResult(BaseModel):
"""
Model class for rerank result.
"""
model: str
docs: list[RerankDocument]
local_reranker_call: Optional[Callable[[list[RerankDocument]], list[RerankDocument]]] = None
def set_local_reranker(self, reranker_function: Callable[[list[RerankDocument]], list[RerankDocument]]):
"""
Sets the local reranker model call.
"""
self.local_reranker_call = reranker_function
def rerank(self):
"""
Applies the local reranker model call if set.
"""
if self.local_reranker_call is not None:
self.docs = self.local_reranker_call(self.docs) In this updated version of The method This approach allows for the flexible integration of local reranker models into existing pipelines, facilitating the migration process by enabling the use of custom reranking logic without requiring significant changes to the existing codebase. Please note that this is just a suggestion and the actual implementation may vary based on the specific requirements and constraints of your project. If you have any further questions or need more clarification, feel free to ask. Sources
|
Beta Was this translation helpful? Give feedback.
-
You can use xinference to integrate with the rerank model deployed locally |
Beta Was this translation helpful? Give feedback.
-
Hi, @soulteary, I'm helping the Dify team manage their backlog and am marking this issue as stale. From what I understand, the issue you raised requests support for setting local reranker model calls to handle RAG scenarios more effectively. I provided a detailed response suggesting a possible approach to implementing this feature, including code examples and references to relevant files in the repository. Additionally, takatost suggested using xinference to integrate with the rerank model deployed locally. The issue has been marked as resolved based on my detailed response, which provides a potential resolution to the issue. Could you please confirm if this issue is still relevant to the latest version of the Dify repository? If it is, please let the Dify team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days. Thank you! |
Beta Was this translation helpful? Give feedback.
-
Self Checks
1. Is this request related to a challenge you're experiencing?
RAG is a scenario that often needs to be dealt with, and the reranker model currently only supports two solutions.
2. Describe the feature you'd like to see
Is it possible to support setting local reranker model calls?
3. How will this feature improve your workflow or experience?
Existing pipelines can be painlessly migrated to Dify.
4. Additional context or comments
No response
5. Can you help us with this feature?
Beta Was this translation helpful? Give feedback.
All reactions