Replies: 3 comments 3 replies
-
Could you make and upload to github SD.made_with_Intel_compiler_AVX2.exe, please? |
Beta Was this translation helpful? Give feedback.
-
|
It's a Linux binary, not a Windows executable. Is it Ok? |
Beta Was this translation helpful? Give feedback.
-
Not my cup of tea. I don't use Linux. |
Beta Was this translation helpful? Give feedback.
-
|
Is here benchmark with gguf? gguf with q8_0 |
Beta Was this translation helpful? Give feedback.
-
|
Quantization doesn't make it faster unless the device get instructions. |
Beta Was this translation helpful? Give feedback.
-
|
It does for llama.cpp and whisper.cpp. For example, 8-bit quantization is twice as fast as the original 16-bit model. And 4-bit - almost 4x times. Of course, SD is much more sensitive to quantization, which degrades the image quality a lot, but at least it must be faster. Otherwise, there is no sense at all to use quantization. It's worse and not faster. |
Beta Was this translation helpful? Give feedback.
-
SD 1.5 512x512
Q8 is the fastest, but not significantly. Overall, there is barely any difference between precisions, which is kind of a surprize. To me it looks like code is always using F32. Used memory is about the same too: 2815, 2325, 2084, 1971 MB. Intel compiler is a little bit faster than GCC. OpenBLAS slows down a lot, wouldn't recommend it at all.
Beta Was this translation helpful? Give feedback.
All reactions