[Feature Request]:

[Rakile]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

The future would not load the Controlnet model, if the weight is 0.
Because I am developing the extension Deforumation that uses Deforum, I would like if the Controlnet model did not load if a user uses a weight schedule value of 0... A value of zero (to my knowledge means that the controlnet module doesn't influence the img2img render.

A simple solution that I tried is to just put these line in the function "create_cnu_dict" in "deforum_controlnet.py" (currently after row 302 (cnu['weight'] = getattr(CnSchKeys, f"cn_{model_num}_weight_schedule_series")[frame_idx]):
if cnu['weight'] == 0:
cnu['enabled'] = False

This will prevent the module to be loaded at all if the weight is 0, and also saves some time, as to not load all enabled controlnet modules. If I'm using 5 controlnet modules that I want to switch on/off realtime this really saves time.

I have tried it and it seems to work, well now with the latest fix, that you did and which closes the StableDiffusionProcessingTxt2Img pipeline.

Well met
//Rakila

Proposed workflow

Add the two line:
if cnu['weight'] == 0:
cnu['enabled'] = False

Additional information

No response

Are you going to help adding it?

I'll let you decide upon this.

[andyxr]

Users can schedule CN weights; my concern is that if we read a weight of 0 as meaning "disable this CN", then that CN will get switched off during a normal weight schedule (e.g 0:(1.1), 200:(0), 300:(1))

[Rakile]

Well, I’m running it like this (in my private version of deforumation <-> deforum), and it works just fine… it’s 2 rows of tweek if you want to test it yourself in deforum_controlnet.py together with a cn schedule as you yourself suggested above 🤗
It would make ”my” life a little bit easier if you could include it as I don’t really want to provide more than 2 modified files with versions of deforumation (I only edit render.py and animation.py)… So, summasumarum, it does not break by dissabling or enabling the cn module, whenever, where ever. Actually the initial image does not enable any cn, bit gets enabled in the following images (original code of deforum_controlnet.py)

[reallybigname]

It's why it has an enable and a weight. The weight decides the influence, the enable decides if it's enabled. I can't say it any simpler than that. If you want to cache 5 CN's at the beginning and not use any of the weights until frame 1000, you can do that.

[Rakile]

Not sure I agree on that. Controlnet was made for on and off for simple render, regardless of weight. Your telling me that they could just not bothered with on or of and just stick with the weight (0 == off?)? For a fact I know it takes much less time when using 5 cn's on 0 weight, compared to 5 cn's turned off.