bug: prompts with JinJa2 templates get their format changed

[ajram23]

Describe the bug

get_langchain_prompt() removes "{{" in the Jinja2 template not just the variables!

To reproduce

Use this jinja2 template {{ handle_text(payload['Name'], "Name is") }}

Additional information

No response

[marcklingen]

Thanks for reporting. As a temporary workaround you can use prompt.prompt directly and add variables yourself

[hassiebp]

Thanks for your report, @ajram23 ! Our application logic is treating handling all variables in double curly braces as langfuse variables and converts them to single curly braced variables when using get_langchain_prompt.

If you have Jinja2 templates in there as well (which happen to be demarcated by double curly braces as well), you can escape this behavior by adding an additional curly brace: {{{ handle_text(payload['Name'], "Name is") }}}

get_langchain_prompt will return {{ handle_text(payload['Name'], "Name is") }} that is the correct Jinha2 string.

Please be careful though that when you use this escaping and not call get_langchain_prompt on it, then the template will have 3 curly braces that might lead to issues with Jinja2.

[ajram23]

@hassiebp While this is the work around I am using, expecting all of your users to do this is weird. Adds an additional step to move prompts in and out of your system.

[hassiebp]

@ajram23 Could you please share a full prompt with Langfuse variables + Jinja2 templates and the workflow you are using? Maybe I am missing the point here :)

[ajram23]

Will DM you the information. Give me sometime, have a lot to dig into given OpenAI announcements.

[marcklingen]

We just discussed this and consider changing the behavior of get_langchain_prompt() to ignore all {{ string }} where string does not match the naming convention for LangFuse prompts (alphabetical characters or underscores).

Does this solve the issue for you or do you have some templates which are only alphabetical characters or underscores? This'd be a quick fix.

[ajram23]

@marcklingen might make sense. Meanwhile not sure what you mean by this "Does this solve the issue for you or do you have some templates which are only alphabetical characters or underscores? "

[marcklingen]

Meanwhile not sure what you mean by this "Does this solve the issue for you or do you have some templates which are only alphabetical characters or underscores? "

Do the parts of your Jinja templates that you want Langfuse to leave unchanged all include spaces, numbers, or special characters (such as the example {{ handle_text(payload['Name'], "Name is") }})? In that case the proposed change should fix this issue.

[hassiebp]

Hi @ajram23 - I've opened a PR to skip the langchain variable transformation for all double curly braces variables that are not valid Langfuse variables. This should hopefully fix the issue for your use-case 👍🏾

[ajram23]

@hassiebp thank you!