Inconsistent behavior between OSX client and python bindings

[gbiss]

Hello, I have an example prompt that returns a response in the OSX desktop client (using the mistral-instuct model), but the same prompt and same model produce an empty string when called from the python api. The following code producing an empty string, while the same prompt returns "Yes" on the desktop client. Any idea what could be going wrong?

from gpt4all import GPT4All
model = GPT4All("/scratch3/gbiss/gpt4all/mistral-7b-instruct-v0.1.Q4_0.gguf", device="gpu")
s = "UPPER MERION TWP., Pennsylvania WPVI  Officers are looking for several suspects wanted in a string of Apple store thefts at the King of Prussia Mall in Upper Merion Township, Pennsylva nia.Police say two men and several other unidentified suspects conspired to commit eight different thefts between September 19, 2022, and June 27, 2023.The suspects in these incidents allege dly planned to steal numerous items, including headphones, Apple Magic Mouses, keyboards, trackpads, and Air Tags that were displayed for sale.Police identified two of the suspects as Giorgi Adeishvili of Mountain Top, PA, and Giorgi Onashvili of Brooklyn, NY.Both men are still being sought at this time.Investigators say they took 10,070 worth of Apple products from the store.A nyone with information on this case is asked to contact the Upper Merion Police Investigations Division at 6102653232. Please indicate if the Apple corporation or one of its products was mentioned above. Please answer yes or no."
model.generate(s)

[cebtenzzre]

At a minimum, you need to use a chat session. See the docs.

[gbiss]

Ah, I see. Thanks for steering me toward using chat sessions.