[FEATURE] keep SystemMessage first in ChatMemory

[McFoggy]

Is your feature request related to a problem? Please describe.

The behavior of ChatMemory implementations looks a bit weird in the position handling of SystemMessage.

• When SystemMessage is added first it is kept as first item
• When a SystemMessage message is added, either after UserMessage or after a first SystemMessage, it is placed at last position
  • during evictions, the SystemMessage position will go up in the Memory until it reaches the first place and remains there.

Describe the solution you'd like

I'd like to standardize and simplify the position handling of the SystemMessage so that it is always kept as first message in the list.

Describe alternatives you've considered

I could reimplement (add my own Implementation) or proxify existing implementations: MessageWindowChatMemory and TokenWindowChatMemory

Additional context

based on current state of langchain4j:0.30.0

May I propose something to solve this?

[langchain4j]

Hi, there is some logic behind the current behaviour (I can explain a bit later). But do you see the problem with it? Do you see the LLM behaves not as you want? Thanks

[McFoggy]

Using the following sample, where I used a simple ChatMemory impl allowing me to really control the order of messages (to demonstrate what can occure with existing impl)

public class ChatMemoryExamples {
    public static class SimpleChatMemory implements ChatMemory {
        private List<ChatMessage> messages = new ArrayList<>();

        @Override
        public Object id() {
            return "simple";
        }

        @Override
        public void add(ChatMessage message) {
            messages.add(message);
        }

        @Override
        public List<ChatMessage> messages() {
            return new ArrayList<>(messages);
        }

        @Override
        public void clear() {
            messages.clear();
        }
    }

    public static class BadPositionningSystemMessage_Example {
        public static void main(String[] args) throws IOException {
            ChatLanguageModel model = OpenAiChatModel.withApiKey(ApiKeys.OPENAI_API_KEY);
            SystemMessage systemMessage = new SystemMessage("provide answer to user questions even if it is not real time data. format the result of user question using asciidoc format");
            UserMessage userFormatMessage = new UserMessage("format the result of the following questions using xml format");
            UserMessage userQuestion = new UserMessage("provide me the list of the 10 biggest cities in term on inhabitants in the USA");
            AiMessage response;

            ChatMemory memory = new SimpleChatMemory();
            memory.add(systemMessage);
            memory.add(userFormatMessage);
            memory.add(userQuestion);

            response = model.generate(memory.messages()).content();
            System.out.println("With ordered messages:");
            System.out.println(memory.messages());

            System.out.println("Answer:");
            System.out.println(response.text());

            memory.clear();
            memory.add(userFormatMessage);
            memory.add(systemMessage);
            memory.add(userQuestion);

            response = model.generate(memory.messages()).content();
            System.out.println("With ordered messages:");
            System.out.println(memory.messages());

            System.out.println("Answer:");
            System.out.println(response.text());
        }
    }
}

the answer is as following

    With ordered messages:
    [SystemMessage { text = "provide answer to user questions even if it is not real time data. format the result of user question using asciidoc format" }, UserMessage { name = null contents = [TextContent { text = "format the result of the following questions using xml format" }] }, UserMessage { name = null contents = [TextContent { text = "provide me the list of the 10 biggest cities in term on inhabitants in the USA" }] }]
    Answer:
    Below is the list of the 10 biggest cities in the USA in terms of population:
    
    ```xml
    <cities>
        <city>
            <name>New York City</name>
            <population>8,336,817</population>
        </city>
        <city>
            <name>Los Angeles</name>
            <population>3,979,576</population>
        </city>
        <city>
            <name>Chicago</name>
            <population>2,693,976</population>
        </city>
        <city>
            <name>Houston</name>
            <population>2,320,268</population>
        </city>
        <city>
            <name>Phoenix</name>
            <population>1,680,992</population>
        </city>
        <city>
            <name>Philadelphia</name>
            <population>1,584,064</population>
        </city>
        <city>
            <name>San Antonio</name>
            <population>1,532,233</population>
        </city>
        <city>
            <name>San Diego</name>
            <population>1,425,976</population>
        </city>
        <city>
            <name>Dallas</name>
            <population>1,345,047</population>
        </city>
        <city>
            <name>San Jose</name>
            <population>1,030,119</population>
        </city>
    </cities>
    ```
    
    With ordered messages:
    [UserMessage { name = null contents = [TextContent { text = "format the result of the following questions using xml format" }] }, SystemMessage { text = "provide answer to user questions even if it is not real time data. format the result of user question using asciidoc format" }, UserMessage { name = null contents = [TextContent { text = "provide me the list of the 10 biggest cities in term on inhabitants in the USA" }] }]
    Answer:
    I'm sorry, but I cannot provide real-time data. However, here is an example of how you can format the result using AsciiDoc:
    
    ```
    == List of the 10 Biggest Cities in the USA by Population
    
    1. New York City
    2. Los Angeles
    3. Chicago
    4. Houston
    5. Phoenix
    6. Philadelphia
    7. San Antonio
    8. San Diego
    9. Dallas
    10. San Jose
    ```

Thus of course to me it looks obvious that ensuring a proper order is important.
The above behavior could occure with the different xxxWindowChatMemory impl if we let the system to reset/update the SystemMessage.

IMO it would be more clear to always ensure a standard behavior. That's why I propose to keep the SystemMessage as first when one is present.
I am just starting with LC4J so you might have other opinion/view on the issue.

[langchain4j]

Hi, I am not sure I understand your point from this example. In the first case, LLM ignored system instructions regarding formatting. In the second case, LLM ignored system instructions about real time data. In my understanding, both cases demonstrate the weakness of this particular LLM, but do not prove anything regarding the order of messages.

Now, why the current logic: AI Service can have multiple methods with different system messages, but still share the same memory. Let's imagine 2 methods with the following system messages:

• @SystemMessage("always answer in english") String answerInEnglish(String userMessage);
• @SystemMessage("always answer in german") String answerInGerman(String userMessage);

Now let's imagine that there were a few interactions in English, but now user wants to switch to German. If we put new system message on the first place in the memory, before all user/AI messages in English, with a high probability it will ignore it and will keep answering in English (aka few-shot examples).

[langchain4j]

Consider adding "always keep system message on index 0" as a configurable property on ChatMemory implementations