Adding openpose controlnet and masks for faces and hands

[kinggeorge05]

Hey team of AnyDoor,
first I want to thank and congrats you with creating AnyDoor. It's amazing and it has so many potential for tons of creative solutions. I really have much appriciation for all the great work you have created.

Currently we are diving into the possibilities of AnyDoor for virtual try on. We are currently at a point where we would like to be able to have more control on the pose to keep it as close as possible to the input pose model. And we are trying to optimize faces and hands. At your projectpage you are mentioning

"Noticing that AnyDoor does not contain any specific design/tuning for tryon, we think it would be helpful to add skeleton infos or warped garment, and tune on tryon data to make it better :)"

Would it be possible to add a Openpose skeleton with ControlNet in the proces? Do you have any ideas where we should look? Or do you mean adding skeletons in the training proces?

The other question is about the faces. We can create a mask for the faces and hands. But do you have experiences or ideas how we should add this to the proces? What ways are there to add such steps within the current proces?

Again thank you so much for everyting. Would love to hear your ideas or tips on this

[XavierCHEN34]

An additional control net for pose would work but it might be heavy. Maybe a PoseGuider in Animate Anyone would be more effcicient.

[kinggeorge05]

Thank you for your quick Xi Chen, I really appreciatie it. There is a lot of things happening in the code. If we wanted to add control net pose or PoseGuider. Where should we start to look? Do you have any advice on which part we should dive into?