v3.9.0 #624
Replies: 7 comments 27 replies
-
Amazing work as always! 👍 |
Beta Was this translation helpful? Give feedback.
-
|
Finally. Got it to work, yay! 😃 Neither adding the cudart dll, nor pasting the pyd in the python library folder worked, same error. Since I'm not using miniconda, it additionaly needed the python311.lib in the fused folder and in coiffeur\Lib\site-packages\torch\lib. Now it works but phew it does take its time to process with a ref image - maybe less iterations would work too? |
Beta Was this translation helpful? Give feedback.
-
|
🎆 👍
Yeah, it's not something you can run on your average consumer pc, at least not for videos. Perhaps you noticed already, if you're using an image for the 2nd time it is much faster. The first time there is a lot of preparation going on, aligned and cropping the faces, calculating the latents etc. but for every face the different things get saved into subfolders in testimages. That is already an improvement to the main branch which cached nothing and had no GUI. But it's still just some sort of prototype to have fun with at the moment. You once wrote you own a VR Headset, weren't you? You're not by any chance coding for it? Because currently all my coding spare time is used up for a Quest 3 XR Project using Unreal Engine 5. It's working well so far BUT like many other people I can't get UE accept my installed Android SDK for some reason. THIS IS DRIVING ME NUTS! It's working great in the Simulator but I can't package it for the real hardware. And my google foo fail me in this case, it seems lots of people have this issue but no solution so far. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah videos would be overkill for a consumer pc. But it's ok for photos and it definitely is FUN 😄 Yes I've seen there are a lot of steps involved, but the results are really good IMO. Yes I do, but it's an Oculus GO from the dinosaur era ^^ And no, coding drives ME nuts in general, damned typos and such make me wanna hit the chair with the little toe 🤣 I like to troubleshoot and figure stuff out - like the coiffeur stuff, was a nice exercise - but coding myself? Not so much. BUT I do have a friend who's into UE coding. I'll visit him sometime this week, maybe he knows something. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, meanwhile I figured it out. Android must be installed into the engine version(!!!) in addition to setting it as target in the project. Took me a almost a week and multiple Android SDK installs! This is very much still in its infancy but now I can have a virtual music visualizer in my living room 😄
That's stone age but not dinosaur era. I started my VR adventures with the Oculus Rift DK2. Later I also wanted to have an Oculus Go but bought the Samsung Gear VR instead because it was compatible with some hacks and I already had the necessary cellphone. The Gear VR would still give most current headsets a run for the money because it had OLED and pretty good resolution. I watched so many movies with it back then. Meanwhile all has matured and AR is of course the next big thing. I miss the old times though, there were so many relaxing and free prototypes to try out the magic of VR illusion. My favourite was flying in an air balloon with a roof above in the middle of a rain storm, a scrolling satellite google map below and a radio you could turn on. Nowadays everything looks similar Unitysomething and wants to be the next Beat Saber. But I disgress... 😃 |
Beta Was this translation helpful? Give feedback.
-
|
Great! I was just asking myself if I would've figured that out. Probably not 😆
Would be very much interested in this 😃You think that would run on the Go? I still use the Winamp visualization plug-ins sometimes - Yes, I'm robot-dialing-into-the-internet-old 😜
Indeed, there was lots of free, simple, relaxing VR stuff. Used the headset far more for that and watching movies, than games and other more "demanding" things. |
Beta Was this translation helpful? Give feedback.
-
|
good morning, can you help me solve this problem? I've probably tried everything. I have Visual Studio installed, antivirus programs disabled. Launching App During handling of the above exception, another exception occurred: Traceback (most recent call last): Done! |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
It's the very same reason and please don't pollute this thread with non-issues but search through the existing ones. There's a very high possibility somebody else already had your problem, see above. |
Beta Was this translation helpful? Give feedback.
-
|
A guide on masking within the current interface would be great... it seems like we need to do keyframing of masks, but I am not sure how layers come into play (and how a single mask at some point affects a certain part of the timeline or the WHOLE timeline), etc. |
Beta Was this translation helpful? Give feedback.
-
|
Was wondering how the new masking works. I am still relatively confused about how Xseg works on how it masks on the program and I was not able to get it to work properly. |
Beta Was this translation helpful? Give feedback.
-
|
I think a lot of people overlooked that the manual masking only works for images, not videos. |
Beta Was this translation helpful? Give feedback.
-
|
TBH I am not entirely sure how xseg works since I am relatively new to the whole thing. Cuz I saw what Xseg is doing and basically highlighting the portion of the face in which it could be replaced but I do not know how to do that part. |
Beta Was this translation helpful? Give feedback.
-
|
It's an already pretrained model / mask. You just have to select it, thats all. If you want to create your own, follow the link in the first post. |
Beta Was this translation helpful? Give feedback.
-
|
Maybe I'm doing something wrong. I put Xseg masking on and saw the preview of the masking. However, when I try to switch, the face still clips as if the Xseg masking was turned off. |
Beta Was this translation helpful? Give feedback.
-
|
Wait, how does Xseg work? Noob here. Do I just change to Xseg masking and that's it? Or does the setting not matter unless choosing manual masking? Is it possible (or an idea at all?) to add facesets for target faces as well? Would that increase accuracy on the target face? Running CPU only fwiw. Oh and thank you so much for this repo, it works way better than I expected when I found it! Considering its relative ease of use. |
Beta Was this translation helpful? Give feedback.
-
Yes. It's a trained Net to detect and mask face occlusion automatically.
No and no. However it's planned to save the face selections into some collection for latter use, so that you don't need to redo the face selection if you process the same video/images again.
I'm very glad if that's your opinion because lately there are so many people opening issues about problems, where the solution mostly is right in front of their eyes and the real problem is sitting in front of a monitor 😄 |
Beta Was this translation helpful? Give feedback.
-
|
Does the occlusion masking work primarily on source or target? I.E: Perfectly visible source face to occluded target vs occluded source face to perfectly visible target. |
Beta Was this translation helpful? Give feedback.
-
|
Target only. The order is Source Face -> Target Face -> Mask occluded areas on target face -> Re-apply occluded target face areas to final image. That way a hand in front of the face won't be pasted over with parts of your fake face. |
Beta Was this translation helpful? Give feedback.
-
|
The last part is the one I am confused. I chose Xseg as the masking option and looked at the preview. I see where the outline is but every time I run the program it does not mask out properly as if Xseg did not work. I am still relatively new to the program so not entirely sure how it works when applying the end product. |
Beta Was this translation helpful? Give feedback.
-
|
my directml has tanked since the update and will only recognize CPU. "AMD always giving a headache >.<" before only when using masking it would only use CPU instead of Directml. Has anything been automatically implemented? I can't even get it to show up now. I tried to force some dml/directml in the settings but it automatically resorts to Cuda now. |
Beta Was this translation helpful? Give feedback.
-
|
If you're on windows and using the automatic installer to update it probably overwrote the onnxruntime with the cuda one. You would need to re-install the directml one. |
Beta Was this translation helpful? Give feedback.
-
|
You hit the nail on the head. I went into C:\Users\username\AppData\Local\Programs\Python I saw another python in there, not related to the one I have installed/path etc I deleted that one completely. I then went into C:\Users\username\AppData\Local\Programs\Python\Python310\Lib deleted any onnx inside. ran pip uninstall onnxruntime onnxruntime-directml just incase. then ran pip install onnxruntime-directml to update to the latest directml. (1.17.3 latest) works perfect now. thankyou C0untFloyd Also Xseg works with DML amd graphics. 😄 |
Beta Was this translation helpful? Give feedback.
-
Bugfix: Face detection bounding box corrupt values at weird angles
Rewrote mask previewing to work with every model
Switching mask engines toggles text interactivity
Clearing target files, resets face selection dropdown
Massive rewrite of swapping architecture, needed for xseg implementation
Added DFL Xseg Support for partial face occlusion
Face masking only runs when there is a face detected
Removed unnecessary toggle checkbox for text masking
Added ability to do multiple swap passes
This discussion was created from the release v3.9.0.
Some further infos:
Adding DFL XSeg support and the possibility of doing several passes with a fake face are in my opinion huge improvements. The XSeg masking works automatically and is very, very fast. If you're wondering what this is all about: https://www.deepfakevfx.com/tutorials/deepfacelab-2-0-xseg-tutorial/
I was thinking about having it enabled by default but chose against it. The model I'm using is a the official one from Deepface Lab, converted to ONNX. If you want to try your own, I used https://github.com/Bruc3Xu/XSegNet2onnx to convert between formats. After conversion you would place it into models "xseg.onnx". It should work right away.
Increasing the number of fake passes should be used sparingly. It sometimes affects the coloring of the face too much or gets too blurry. It's always hit and miss, some fakes looked a lot better when using 2-3 steps, most of them became too artificial looking. Experiment yourself!
Manual masking: I left it out of the regular batch processing because it works with images only. It's perfectly working but to save the result you need to save the image from the preview area.
I'm pretty satisfied with the state of roop unleashed now, the only thing missing is whole frame modification like coloring b/w images or upscaling and adding filters. With the rewritten architecture this should be pretty easy, I'm just unsure if this should be in the faceswap tab or separated into the extras tab. Also there still are many open issues and bugs with the gradio framework I'm using. Hopefully those will be fixed by them before I completely loose interest in this 😄
Anyway, I'm currently putting much work in another project fork I started some time ago:
Coiffeur
I have it running on my local machine but it's currently CUDA only and you need to be very tech savy to install it successfully. Check it out if you have an NVIDIA GPU and feel brave enough!
Beta Was this translation helpful? Give feedback.
All reactions