-
Notifications
You must be signed in to change notification settings - Fork 447
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Why is the loss of Diffusion model calculated between “RANDOM noise” and “model predicted noise”? #118
Comments
@egshkim I'm working with this repo too so i'll give my 2 cents. In the inference step
That function is getting called by
x_recon is actually correctly named.
the u-net ( |
Thanks for your kind and detailed explanation. : ) In my opinion, the actually added noise between step "t-1" and step "t" should be used for loss calculation. |
Yes - that comes from the original definition of diffusion processes in the first paper: https://arxiv.org/pdf/2006.11239.pdf (specifically equation 14): The real loss function (Equations 3 and 5) do relate The SR3 paper (https://arxiv.org/pdf/2104.07636.pdf) they also experiment with different loss norms (L1 vs L2) and find that L1 loss gives better results... I'm no mathematician so the derivation is a bit hard to follow, I also welcome further explanations :) |
As far as i know,the function 'default' means that return the actually noise if actually noise is exist,only return the random noise when actually noise is not exist.i hope my answear can help you. |
Thanks for your a lot contribution and hard work.
Why is the loss of Diffusion model calculated between “RANDOM noise” and “model predicted noise”?
Not between “Actual added noise” and “model predicted noise”?
The text was updated successfully, but these errors were encountered: