Output encoding problem?

[tomsnunes]

Hey guys! When I run the binary compiled using code from the master branch on Windows 11, during interaction with the model, in situations involving Latin characters such as á, ã, é as a response, the characters are displayed incorrectly.
I got the same kind of behavior when trying to interact with the model in languages like Portuguese and Spanish. Characters from languages such as Japanese, Chinese, and Korean are also displayed incorrectly.
I believe it's not a problem with my terminal's encode configuration, as the characters are displayed correctly when inputting data to the model, but the output from model brings invalid characters.

My system information:
CPU: Ryzen 9 5900x
OS: Windows 11 Pro x64
cmake version 3.26.0
Python 3.10.10
Compiler: Visual Studio Community 2022 using compilers for 17.5.2

Example:

Is this behaviour expected?
Thanks!

[j-f1]

That output suggests to me that your terminal is not using the UTF-8 encoding. Make sure it’s configured to do that — LLaMA was trained to output UTF-8 text.

[tomsnunes]

Looks like you're right. After some further research, I had to turn my regional settings on to use the beta feature that allows the system locale (language for non-Unicode programs) to UTF-8 like that:

Thanks for the help!