bug: Parent run not found when using Callbacks with langchain EnsembleRetriever

[nathan-vo810]

Describe the bug

When trying to pass Langfuse callbacks into EnsembleRetriever, Exception was raised.

To reproduce

from langfuse.callback import CallbackHandler
from langchain.retrievers import BM25Retriever, EnsembleRetriever

question = ""
    
vector_db_client_provider = PGVectorClient()
vectorstore = vector_db_client_provider.get_client("user")

docs = vectorstore.similarity_search(query=question, k=1)
bm25_retriever = BM25Retriever.from_documents(documents=docs)
vector_retriever = vectorstore.as_retriever()

ensemble_retriever = EnsembleRetriever(retrievers=[vector_retriever, bm25_retriever], weights=[0.5, 0.5])

relevant_docs = ensemble_retriever.get_relevant_documents(question, callbacks=[CallbackHandler()])

Additional information

No response

[marcklingen]

Thanks for sharing, do you use this retriever standalone or within a larger chain/agent? We designed the hadnler to work best when being passed to invoke/call/... methods which then also include the EnsembleRetriever. I'll try to look into solution on how to fix this issue though as this seems to be a very valid use case

[nathan-vo810]

I use that as a standalone for my RAG pipeline. Rather than incorporating with Langchain, I built separated components and finally link them together.

An additional note:

• When I was using the ConversationRetrievalChain, Langfuse callbacks worked perfectly (show traces for both retrievers).
• But when coding a function and use observe, traces do not appear for the retrievers (only the function trace)

[marcklingen]

I use that as a standalone for my RAG pipeline. Rather than incorporating with Langchain, I built separated components and finally link them together.

Thanks for confirming

An additional note:

• When I was using the ConversationRetrievalChain, Langfuse callbacks worked perfectly (show traces for both retrievers).
• But when coding a function and use observe, traces do not appear for the retrievers (only the function trace)

This is expected, unless you pass a callback handler to the langchain pieces of your function as shown here: https://langfuse.com/docs/sdk/python/decorators#langchain

[nathan-vo810]

Ah yes, I did pass langfuse_context.get_current_langchain_handler() to my EnsembleRetriever. But the result is as above exception.

Thanks for providing such a great tool and I look forward to the solution!

[marcklingen]

Ah yes, I did pass langfuse_context.get_current_langchain_handler() to my EnsembleRetriever. But the result is as above exception.

Thanks for providing such a great tool and I look forward to the solution!

👍 awesome, I was worried that the docs were not clear enough on how this works

probably decorating a function that wraps the get_relevant_documents and just passing all kwargs is the best workaround until this is fixed. thereby you at least can track the inputs/outputs/timings of this within your larger context.

[marcklingen]

Merging this into #1927 to collect all instances of this error