Support for SD2.x has been added, welcome to try it out

[leejet]

./bin/sd -m ../models/v2-1_512-ema-pruned-ggml-model-f16.bin -p "a lovely cat" -v

[juniofaathir]

Your newest windows ready version, is that mean the image generation now can be accelerated by nvidia gpu?

[leejet]

Not yet, the convolution operation can only run on the cpu for the time being

[juniofaathir]

So, what does 'add CUDA RNG' mean?

[leejet]

Imitiating torch cuda randn on CPU. This results in the same image as stable-diffusion-webui GPU RNG

[Green-Sky]

should we make the stabe-diffusion-webui one the default?

[Green-Sky]

$ bin/sd -m ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin -p "night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)"

same speed as 1.x models (~10s per sampling step for 512x512)

[Green-Sky]

@leejet id did not acually finish, it crashed very late.

Details

$ bin/sd -m ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin -p "night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)" -H 1024 -W 1024 -v
Option:
    n_threads:       12
    mode:            txt2img
    model_path:      ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin
    output_path:     output.png
    init_img:
    prompt:          night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)
    negative_prompt:
    cfg_scale:       7.00
    width:           1024
    height:          1024
    sample_method:   eular a
    sample_steps:    20
    strength:        0.75
    rng:             std_default
    seed:            42
System Info:
    BLAS = 0
    SSE3 = 1
    AVX = 1
    AVX2 = 1
    AVX512 = 0
    AVX512_VBMI = 0
    AVX512_VNNI = 0
    FMA = 1
    NEON = 0
    ARM_FMA = 0
    F16C = 1
    FP16_VA = 0
    WASM_SIMD = 0
    VSX = 0
[INFO]  stable-diffusion.cpp:2793 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin'
[DEBUG] stable-diffusion.cpp:2801 - verifying magic
[DEBUG] stable-diffusion.cpp:2812 - loading hparams
[INFO]  stable-diffusion.cpp:2821 - model type: SD2.x
[INFO]  stable-diffusion.cpp:2829 - ftype: q8_0
[DEBUG] stable-diffusion.cpp:2835 - loading vocab
[DEBUG] stable-diffusion.cpp:2863 - ggml tensor size = 288 bytes
[DEBUG] stable-diffusion.cpp:2868 - clip params ctx size =  359.99 MB
[DEBUG] stable-diffusion.cpp:2887 - unet params ctx size =  1406.40 MB
[DEBUG] stable-diffusion.cpp:2908 - vae params ctx size =  95.51 MB
[DEBUG] stable-diffusion.cpp:2929 - preparing memory for the weights
[DEBUG] stable-diffusion.cpp:2945 - loading weights
[DEBUG] stable-diffusion.cpp:3048 - model size = 1858.32MB
[INFO]  stable-diffusion.cpp:3053 - total params size = 1858.69MB (clip 358.69MB, unet 1405.49MB, vae 94.51MB)
[INFO]  stable-diffusion.cpp:3059 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin' completed, taking 1.33s
[DEBUG] stable-diffusion.cpp:3129 - diffusion context need 10.91MB static memory, with work_size needing 0.08MB
[INFO]  stable-diffusion.cpp:3188 - check is_using_v_parameterization_for_sd2 completed, taking 0.12s
[DEBUG] stable-diffusion.cpp:3189 - diffusion graph use 12.65MB runtime memory: static 10.91MB, dynamic 1.75MB
[DEBUG] stable-diffusion.cpp:3193 - 1024 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3082 - running in v-prediction mode
[DEBUG] stable-diffusion.cpp:813  - parse 'night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, (blue eyes:0.8), cute young face, 18 yo, soft volumetric lights, (backlit:1.3), (cinematic:1.3), intricate details, (ArtStation:1.2)' to [['night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, ', 1], ['blue eyes', 0.8], [', cute young face, 18 yo, soft volumetric lights, ', 1], ['backlit', 1.3], [', ', 1], ['cinematic', 1.3], [', intricate details, ', 1], ['ArtStation', 1.2], ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "night scene, close up 85mm photo of a beautiful cute girl, posing, look at a camera and smile, pink ponytail hair, " to tokens ["night</w>", "scene</w>", ",</w>", "close</w>", "up</w>", "8</w>", "5</w>", "mm</w>", "photo</w>", "of</w>", "a</w>", "beautiful</w>", "cute</w>", "girl</w>", ",</w>", "posing</w>", ",</w>", "look</w>", "at</w>", "a</w>", "camera</w>", "and</w>", "smile</w>", ",</w>", "pink</w>", "ponytail</w>", "hair</w>", ",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "blue eyes" to tokens ["blue</w>", "eyes</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt ", cute young face, 18 yo, soft volumetric lights, " to tokens [",</w>", "cute</w>", "young</w>", "face</w>", ",</w>", "1</w>", "8</w>", "yo</w>", ",</w>", "soft</w>", "<|endoftext|>", "lights</w>", ",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "backlit" to tokens ["<|endoftext|>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt ", " to tokens [",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "cinematic" to tokens ["cinematic</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt ", intricate details, " to tokens [",</w>", "intricate</w>", "details</w>", ",</w>", ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "ArtStation" to tokens ["<|endoftext|>", ]
[DEBUG] stable-diffusion.cpp:3230 - condition context need 10.70MB static memory, with work_size needing 0.32MB
[DEBUG] stable-diffusion.cpp:3255 - building condition graph completed: 1205 nodes, 397 leafs
[DEBUG] stable-diffusion.cpp:3263 - computing condition graph completed, taking 0.25s
[INFO]  stable-diffusion.cpp:3309 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[DEBUG] stable-diffusion.cpp:3318 - 316416 bytes of dynamic memory has not been released yet
[DEBUG] stable-diffusion.cpp:813  - parse '' to [['', 1], ]
[DEBUG] stable-diffusion.cpp:375  - split prompt "" to tokens []
[DEBUG] stable-diffusion.cpp:3230 - condition context need 10.70MB static memory, with work_size needing 0.32MB
[DEBUG] stable-diffusion.cpp:3255 - building condition graph completed: 1205 nodes, 397 leafs
[DEBUG] stable-diffusion.cpp:3263 - computing condition graph completed, taking 0.23s
[INFO]  stable-diffusion.cpp:3309 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[DEBUG] stable-diffusion.cpp:3318 - 316416 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3854 - get_learned_condition completed, taking 0.49s
[INFO]  stable-diffusion.cpp:3870 - start sampling
[DEBUG] stable-diffusion.cpp:3367 - diffusion context need 32.64MB static memory, with work_size needing 21.25MB
[INFO]  stable-diffusion.cpp:3496 - step 1 sampling completed, taking 61.77s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 2 sampling completed, taking 60.59s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 3 sampling completed, taking 68.61s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 4 sampling completed, taking 57.71s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 5 sampling completed, taking 58.78s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 6 sampling completed, taking 58.51s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 7 sampling completed, taking 59.81s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 8 sampling completed, taking 59.15s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 9 sampling completed, taking 57.99s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 10 sampling completed, taking 62.49s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 11 sampling completed, taking 58.17s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 12 sampling completed, taking 58.04s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 13 sampling completed, taking 58.53s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 14 sampling completed, taking 58.87s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 15 sampling completed, taking 57.54s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 16 sampling completed, taking 58.88s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 17 sampling completed, taking 58.62s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 18 sampling completed, taking 60.46s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 19 sampling completed, taking 59.86s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3496 - step 20 sampling completed, taking 57.64s
[DEBUG] stable-diffusion.cpp:3497 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3501 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3572 - diffusion graph use 6705.54MB of memory: params 1405.49MB, runtime 5300.04MB (static 32.64MB, dynamic 5267.40MB)
[DEBUG] stable-diffusion.cpp:3580 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3875 - sampling completed, taking 1192.06s
[DEBUG] stable-diffusion.cpp:3743 - vae context need 10.14MB static memory, with work_size needing 0.00MB
Segmentation fault (core dumped)

[Green-Sky]

it also immediately crashes with --steps 1

[leejet]

All the issues has been fixed. Also, I have set the default rng to cuda. You can pull the latest code and give it a try. Don't forget to update the submodule as well.

git pull origin master
git submodule update

[Green-Sky]

Don't forget to update the submodule as well.

yes, always :)

tested and works. will run a more steps later, since large images take forever and my cpu is occupied :)

Details

 bin/sd -m ../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin -p "a lovely cat" --steps 1 -t 10 -W 1024 -H 1024
[INFO]  stable-diffusion.cpp:2802 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin'
[INFO]  stable-diffusion.cpp:2830 - model type: SD2.x
[INFO]  stable-diffusion.cpp:2838 - ftype: q8_0
[INFO]  stable-diffusion.cpp:3066 - total params size = 1858.69MB (clip 358.69MB, unet 1405.49MB, vae 94.51MB)
[INFO]  stable-diffusion.cpp:3068 - loading model from '../models/artiusV21_v21_nsfw-ggml-model-q8_0.bin' completed, taking 1.86s
[INFO]  stable-diffusion.cpp:3197 - check is_using_v_parameterization_for_sd2 completed, taking 0.24s
[INFO]  stable-diffusion.cpp:3091 - running in v-prediction mode
[INFO]  stable-diffusion.cpp:3325 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[INFO]  stable-diffusion.cpp:3325 - condition graph use 373.30MB of memory: params 358.69MB, runtime 14.61MB (static 10.70MB, dynamic 3.91MB)
[INFO]  stable-diffusion.cpp:3864 - get_learned_condition completed, taking 0.66s
[INFO]  stable-diffusion.cpp:3880 - start sampling
[INFO]  stable-diffusion.cpp:3505 - step 1 sampling completed, taking 111.65s
[INFO]  stable-diffusion.cpp:3588 - diffusion graph use 6705.54MB of memory: params 1405.49MB, runtime 5300.04MB (static 32.64MB, dynamic 5267.40MB)
[INFO]  stable-diffusion.cpp:3885 - sampling completed, taking 111.70s
[INFO]  stable-diffusion.cpp:3805 - vae graph use 6760.65MB of memory: params 94.51MB, runtime 6666.14MB (static 10.14MB, dynamic 6656.00MB)
[INFO]  stable-diffusion.cpp:3898 - decode_first_stage completed, taking 169.61s
[INFO]  stable-diffusion.cpp:3912 - txt2img completed in 281.97s, use 6800.05MB of memory: peak params memory 1858.69MB, peak runtime memory 6666.14MB
save result image to 'output.png'

[Green-Sky]

1024x1024 15steps

...
[DEBUG] stable-diffusion.cpp:3509 - diffusion graph use 5300.04MB runtime memory: static 32.64MB, dynamic 5267.40MB
[DEBUG] stable-diffusion.cpp:3510 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3588 - diffusion graph use 6705.54MB of memory: params 1405.49MB, runtime 5300.04MB (static 32.64MB, dynamic 5267.40MB)
[DEBUG] stable-diffusion.cpp:3589 - 263168 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3885 - sampling completed, taking 837.42s
[DEBUG] stable-diffusion.cpp:3754 - vae context need 10.14MB static memory, with work_size needing 0.00MB
[DEBUG] stable-diffusion.cpp:3782 - computing vae graph completed, taking 82.86s
[INFO]  stable-diffusion.cpp:3805 - vae graph use 6760.65MB of memory: params 94.51MB, runtime 6666.14MB (static 10.14MB, dynamic 6656.00MB)
[DEBUG] stable-diffusion.cpp:3806 - 12583936 bytes of dynamic memory has not been released yet
[INFO]  stable-diffusion.cpp:3898 - decode_first_stage completed, taking 82.95s
[INFO]  stable-diffusion.cpp:3912 - txt2img completed in 920.75s, use 6800.05MB of memory: peak params memory 1858.69MB, peak runtime memory 6666.14MB

[bssrdf]

@Green-Sky, could you share this model (artiusV21_v21_nsfw-ggml-model-q8_0.bin) somewhere? Thanks.

[Green-Sky]

the q8_0.bin specifically? the safetensor can be aquired here: https://civitai.com/models/27739/artius-v21 and https://civitai.com/models/15975/artius-v21-nsfw

[bssrdf]

Never mind, I downloaded it from the link you provided. Thanks.