Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instruct-pix2pix pipeline: add ability to pass cross_attention_kwargs in call method #7962

Closed
AlexeyZhuravlev opened this issue May 16, 2024 · 2 comments

Comments

@AlexeyZhuravlev
Copy link
Contributor

Is your feature request related to a problem? Please describe.
I have a use-case where I need to use custom adapter together with instruct-pix2pix model, which requires passing custom cross_attention_kwargs into unet.
However, having ability to pass this parameter would allow other use-cases as well, like LoRAs. Basically any use-case where this parameter is used for StableDiffusionPipeline should also be relevant for StableDiffusionInstructPix2PixPipeline as well.

Describe the solution you'd like.
Add a new cross_attention_kwargs parameter to __call__ method of StableDiffusionInstructPix2PixPipeline. Pass this parameter to unet call inside the method.

Describe alternatives you've considered.
Other hacky solutions with context manager, which manipulate UNet tensors might work, but they would overcomplicate the code. Also it's hard to make such solutions work properly with torch.compile

@AlexeyZhuravlev
Copy link
Contributor Author

I opened the PR which solves this issue: #7961

@yiyixuxu
Copy link
Collaborator

fixed with #7961!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants