suggestions for chatflow

[badbye]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing?

I was trying to implement the chat-langchain project using chatflow.
It rewrite the user's question based on the memory(chat history), then let the LLM answer the rewrited question.
However, in chatflow the LLM node always treat sys.query as the last user message.

2. Describe the feature you'd like to see

1. Set memory as a system variable so that every node can read it.
2. Allow us customize user message(set to any variable) in LLM node.

3. How will this feature improve your workflow or experience?

Well, if those two features are added, we can

1. Cut the memory by ourself, or even rewrite the memory.
2. Use the memory to rewrite user's query.
3. We can even customize the whole prompt using the template(jinja2) node, then send to LLM.
   The flexibility comes back.

4. Additional context or comments

No response

5. Can you help us with this feature?

• I am interested in contributing to this feature.

[crazywoola]

You can add another node to modify this query and use it in second llm.

[badbye]

I am saying "chatflow", you were showing a screenshot of workflow.
The LLM settings in chatflow is different.

[badbye]

https://docs.dify.ai/v/zh-hans/guides/workflow/key_concept#chatflow-he-workflow-de-qu-bie

[VincePotato]

This feedback is reasonable. Users insert a processed user query into the final user message. We will optimize this issue as soon as possible.

[badbye]

Thanks. However, it is not enough if we can not read the memory.

Suppose we want to rewrite user's query based on the memory. Here is a case:

from openai import AzureOpenAI
client = AzureOpenAI(...)
def completion(messages):
    response = client.chat.completions.create(
        model="gpt-4",
        messages=messages
    )
    print(response.choices[0].message.content)

memory = [
    {"role": "user", "content": "李白的妻子是谁？"},
    {"role": "assistant", "content": "李白的妻子是余氏。"},
    {"role": "user", "content": "她死的时候多少岁？"}, 
    {"role": "assistant", "content": "关于李白妻子余氏的生平事迹，历史上并没有明确的记载，包括她的年龄以及去世的年龄等信息，都无从得知。"},
]
query = {"role": "user", "content": "用英文讲她的名字"}

system_prompt = """Given the following conversation and a follow up input, rephrase the follow up input to be a standalone input, in its original language.
The standalone input should be detailed, accurate, unambiguous, understandable, while still being concise, keep the same meaning with the original input.
In the following cases, the standalone input just repeat the Follow Up Input without any character changes:
- the follow up input is already a standalone input
- the follow up input is just some polite phrases without asking any detail questions
- the conversation is empty"""

If we use memory as multiple records, the model will forget the instruction and answer the query directly

prompt = [{"role": "system", "content": system_prompt}] + memory + [query]
completion(prompt)
# Her name is Lady Yu in English.

If we can read memory and reformat it, it works as expect:

memory_context = "<conversation>\n%s\n</conversation>" % '\n'.join([f"{i['role']}: {i['content']}" for i in memory])
question = f"Follow Up Input: {query['content']} Standalone input: "
prompt = '\n'.join([system_prompt, memory_context, question])
completion(prompt)
# 请用英文告诉我李白妻子的名字。

[badbye]

You may doubt that the system_prompt was written for the reformatted memory.
Actually, I've tried editing it to make it work with raw memory, not succeeded yet.

[evan-wu]

• + 1 for referencing the Memory as a Variable.

[hsp8712]

+1 I need this one