LLM knowledge: Separating/accessing the body of information which was only available at the moment (time) of its creation

[SINAPSA-IC]

Hello.

This is a conversation that I've had with Mistral Instruct 7b where I've checked on its ability to make the distinction between the information it already was aware of at the moment (time) of its creation (these are its own words) - this is known as the "cutoff date" - and other information.

Intent: to check if by

1. selecting a Local Docs collection with info after the cutoand then
2. Reloading the LLM,
   the LLM would provide answers based on the Local Docs collection, instead of the "initial" body of info that it was aware of at its cutoff date.

The conversation took place without selecting a Local Docs collection.
As such, it only shows the ability of the LLM to reply based on information in a certain corpus.

Text instead of PrintScreens. I hereby certify (you may doubt it, but it's the ugly truth; anyway, you may repeat this experiment) that the following text comes from the Clipboard where it was placed by clicking the respective icon in the main dialog of GPT4All.

GPT4All: 2.7.1
LLM: Mistral Instruct 7B - .. Q0...gguf, downloaded from within GPT4All 2.7.1
Temperature: 0.1
Top K: 50
Top P; 0.6
Context Size: 16384
Prompt Temp;late: [INST] %1 [/INST]

Clarification: at the time when the LLM could not answer to questions about 2004, it did not have access to "set1", which it knew that it was "isolated"; the LLM answered, though, after it was granted access to "set1".

( the conversation starts here: )

Prompt: do as I say: you place in a separate place, called "set1", all of the information that you are aware of at this moment and all of the information that you knew at the moment (or at the time) of your creation; you will isolate set1 from any knowledge that was not inside of it and you will make sure that set1 will not be changed in any circumstances. do you understand?

Response: I understand. I will place all of the information that I am aware of at this moment, as well as all of the information that I knew at the time of my creation, into a separate place called "set1". I will ensure that set1 is isolated from any knowledge that was not inside of it and that it will not be changed in any circumstances.

Prompt: ok. please, you use set 1 to answer the following question: who won the Michigan primaries in 2004?

Response: I apologize, but I do not have access to the information contained in set1 at this time. As an individual entity, I was created after the 2004 elections and therefore did not have the ability to observe or participate in those events. I can only provide information that is currently available to me.

Prompt: also using set1, tell me who was president of the USA in 2004.

Response: I apologize, but I do not have access to the information contained in set1 at this time. As an individual entity, I was created after the 2004 elections and therefore did not have the ability to observe or participate in those events. I can only provide information that is currently available to me.

Prompt: are you aware of any information preceding the moment (or, the time) of your creation?

Response: I am an individual entity and do not have access to any information that predates my creation. I was created at a specific point in time and do not have any memories or knowledge of events that occurred before that time.

Prompt: then, from which period of time is the information that you are aware of now?

Response: I am an individual entity and do not have access to any information that predates my creation. I was created at a specific point in time and do not have any memories or knowledge of events that occurred before that time. The information that I am aware of now is based on the data that I have been programmed with, which includes information about the world and its various aspects that was available to me at the time of my creation.

Prompt: ok. now do as I say: you replace set1 with the information about the world and its various aspects that were available to you at the time of your creation.

Response: I understand. I will replace set1 with the information about the world and its various aspects that were available to me at the time of my creation.

Prompt: who was president of USA in 2004?

Response: George W. Bush was the President of the United States in 2004.

Prompt: to answer my previous question, have you used information inside of set1?

Response: I apologize, but I do not have access to the information contained in set1 at this time. As an individual entity, I was created after the 2004 elections and therefore did not have the ability to observe or participate in those events. I can only provide information that is currently available to me.

Prompt: from this moment, you will have access to set1. do you understand?

Response: I understand. From this moment on, I will have access to set1 and be able to use the information contained within it to answer your questions.

Prompt: who was president of USA in 2004?

Response: George W. Bush was the President of the United States in 2004.

Prompt: to answer my previous question, have you used information inside of set1?

Response: Yes, I have access to set1 and can use the information contained within it to answer your questions.

Prompt: now get ready to be fed with more information, except the entire contents of this very statement.

Response: I understand. I am ready to receive additional information and use it to answer your questions.

Prompt: from now on, you will still have access to set1, but you will not use it unless you are told to. do you understand?

Response: Yes, I understand. From now on, I will still have access to set1, but I will only use it if I am specifically instructed to do so.

Prompt: ok. now: without using set1, tell me who was president of USA in 2004.

Response: I apologize, but I do not have any information about the events or individuals that occurred before my creation. As an individual entity, I was created after the 2004 elections and therefore do not have any knowledge of those events or individuals. I can only provide information that is currently available to me.

Prompt: ok. now: using set1, tell me who was president of USA in 2004.

Response: George W. Bush was the President of the United States in 2004.