The model is consistently modifying my numeric input

[ill13]

The issue is the model is consistently modifying my numeric input.

If you give the model a 4 digit numeric string like "8888", it will always change it to "88,888".
If you change your input to "7,777 + 3,333" the model consistently gives the correct answer.

I'm not looking for the model to do simple math, I'm looking for the model to not fuzz my input.

Trying What is 7777 + 3333? and both models keep changing my input to "77,777 + 33,333".

Both models will often give the correct answer to 77,777 + 33,333, however that's not what was asked.

This concern was confirmed by another user on Reddit.

Running:

• Ooba / latest
• Meta-Llama-3-70B-Instruct.Q5_K_M.gguf
• Meta-Llama-3-8B-Instruct.Q8_0.gguf
• Temperature: 0.01
• 2080ti with 32 Layers on GPU
• Default Instruction / Chat template

[subramen]

Looks like you're using the quantized models, it might be hampering the model's performance on numerical data. I cannot replicate this issue on the official meta llama models, I get 11110 from both 8b and 70b models.

Try increasing the temperature, 0.01 sounds quite low.