How to communicate trace-id between backend and frontend ? (langchain+langserve integration)

[Sinnaeve]

I use langchain+langserve as backend. I wanted to allow frontend to send user feedbacks as score to langfuse.
But it's unclear to me how the frontend can get the correct trace-id.

Is there a proper way to have backend and frontend use the same trace-id when using langchain+langserve ?
I feel like either the backend have to send the trace-id to the front in the endpoint output, or the front must be able to send the trace-id to be used to the backend. But I'm not sure how to do that correclty.
Any help would be much appreciated !

---

Here are my research:
In: https://langfuse.com/docs/sdk/typescript/guide-web
it is stated:
While integrating Langfuse, it is important to either include the Langfuse Ids in the response to the frontend or to use an own id as the trace id which is available in both backend and frontend.

In discord someone managed it be re-writing the CallbackHandler it seems.
https://discord.com/channels/1111061815649124414/1211795284665831464/1223303914816929933

for my part:
I thought I might be able to use RunnableParallel from langchain to get current trace id as output:

final_chain= RunnableParallel(passed=chain, traceid=lambda x: langfuse_handler.get_trace_id())

But I'm not sure if my method would work with concurrent calls to the endpoint as I would have a single langfuse_handler object for mutliple calls.

[marcklingen]

Hi there, can you pass a separate langchain callback handler for every trace when using langserve?

[Sinnaeve]

I try using per request config modifier like this:

async def per_req_config_modifier(config: Dict, request: Request) -> Dict:
    """Modify the config for each request."""
    langfuse_session_id = config["configurable"]["langfuse_session_id"]
    langfuse_trace_id = config["configurable"]["langfuse_trace_id"]
    trace = langfuse_.trace(
            id=langfuse_trace_id,
            session_id=langfuse_session_id
        )
    if "callbacks" not in config:
        config["callbacks"] = []
    trace_handler = trace.get_langchain_handler()
    config["callbacks"].extend([trace_handler])
    return config

async def per_req_config_modifier(config: Dict, request: Request) -> Dict:
    """Modify the config for each request."""
    langfuse_session_id = config["configurable"]["langfuse_session_id"]
    langfuse_trace_id = config["configurable"]["langfuse_trace_id"]
    trace = langfuse_.trace(
            id=langfuse_trace_id,
            session_id=langfuse_session_id
        )
    if "callbacks" not in config:
        config["callbacks"] = []
    trace_handler = trace.get_langchain_handler()
    config["callbacks"].extend([trace_handler])
    return config

add_routes(app, agent, path="/agent",
           per_req_config_modifier=per_req_config_modifier)

But now I don't have trace name, input or output. I have the correct data in my 'generation' inside the trace. And I have output but not input in the Span contained in the trace that contains the generation.

Before I still didn't had input but I had output and trace name, using something like:

agent = agent.with_config( 
             config=RunnableConfig(
                 callbacks=[langfuse_handler],
             )
         )

---

I'm not to last langfuse version, so I will try to upgarde langchain/langserve/lanfuse to last version.

[Sinnaeve]

Ok, now I have updated to more recent versions:
langfuse: 2.28.2
langchain: 0.1.17
langserve: 0.1.1

I still don't have input and output for the trace (also the input and output are the good ones inside the Generation which is inside the trace.

I could manually add input when creating the trace with the following code, but that doesn't feel like it'sthe right way:

langfuse_= Langfuse(
    secret_key=os.environ['LANGFUSE_SECRET_KEY'],
    public_key=os.environ['LANGFUSE_PUBLIC_KEY'],
  host=os.environ['LANGFUSE_HOST'],
)

async def per_req_config_modifier(config: Dict, request: Request) -> Dict:

    """Modify the config for each request."""
    user = await get_current_active_user_from_request(request)
    langfuse_session_id = config["configurable"]["langfuse_session_id"]
    langfuse_trace_id = config["configurable"]["langfuse_trace_id"]
    trace = langfuse_.trace(
        id=langfuse_trace_id,
        session_id=langfuse_session_id,
        name='agent name',
        user_id=user.username,
        release=gitsha,
        input=request_data['input']['input'],
    )
    if "callbacks" not in config:
        config["callbacks"] = []
    trace_handler = trace.get_langchain_handler()
    config["callbacks"].extend([trace_handler])
    return config

add_routes(app, agent, path="/agent",
           per_req_config_modifier=per_req_config_modifier)

[marcklingen]

handlers created this way (trace_handler = trace.get_langchain_handler()) do not update the trace for which they are created as this is often undesirable when adding multiple runs to the same trace. We'll add an additional optional toggle for this this week