[Performance 3/6] Disable nan check by default

[huchenlei]

Description

According to lllyasviel/stable-diffusion-webui-forge#716 (comment) , nan check has ~20ms/it overhead. The overhead is large enough that option should only be used for debugging purpose.

Screenshots/videos:

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

[SLAPaper]

can the nan check only enable for VAE?

[wfjsw]

nan check is not great but disabling that has a lot of implications for example the VAE fallback will no longer work

[wfjsw]

long term wise it may be desirable to load VAE as bfloat16 instead

[AUTOMATIC1111]

As it is now it will break automatically switching to full precision VAE.

[AUTOMATIC1111]

Can we maybe just get the needed performance improvement by checking a single element instead of the whole tensor? Since there are batch norms, single values becoming NaN dooms the whole tensor to become all NaNs.

I pushed 547778b to dev with this change.

[AUTOMATIC1111]

Also what tool is being used here for those performance visualizations? I'd like that too.

Edit: it's torch's profiler visualized in chrome https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html

[huchenlei]

Can we maybe just get the needed performance improvement by checking a single element instead of the whole tensor? Since there are batch norms, single values becoming NaN dooms the whole tensor to become all NaNs.

I pushed 547778b to dev with this change.

I think checking only a single element is a better way to handle this. Thanks for doing that!