Is there a way to use T2I adapters with an image2image pipeline?

[nivedwho]

I want to use T2I canny adapter along with additional image inputs passed along with strength value like image2image pipeline? Is there a SDXLAdapterImg2ImgPipline or something similar that achieves the same thing?

[tolgacangoz]

Aren't T2I-Adapters special for Text-2-Image 🤔? There is a multi-adapter usage that you can adjust conditioning levels.
Or, does Image2Image with IP-Adapters help?

[asomoza]

Not really, they're named like that but it's just a form of control. There isn't a reason why they can't be used with image-to-image, it's just that no one has created the community pipeline.

source	normal	canny t2i adapter

Since @nivedwho is asking for something similar, the more obvious one is the controlnet equivalent: StableDiffusionXLControlNetImg2ImgPipeline

[nivedwho]

Yes, I actually wanted to use the T2I adapter instead of the existing controlnet img2img pipe just to save some inference time.

[tolgacangoz]

@asomoza Hmm, thank you for your answer. I will look into this.

@nivedwho Faster diffusion on less beefy GPUs. You can also compile ControlNet.