Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About performance #33

Open
layumi opened this issue May 8, 2023 · 2 comments
Open

About performance #33

layumi opened this issue May 8, 2023 · 2 comments

Comments

@layumi
Copy link

layumi commented May 8, 2023

Thanks a lot for the great work. I test the trained model "1222_mld_humanml3d_FID041.ckpt"

The performance is as follows, which seems a bit lower than the reported performance in the paper.

image

Is the performance correct?

@ChenFengYe
Copy link
Owner

ChenFengYe commented May 8, 2023

Hi, thanks for your attention!
Your testing on R_Precision is a bit lower, 0.756 (your testing) v.s. 0.772 (Tab.1 in Paper). However, it seems your FID is even better, 0.468(your testing) v.s. 0.473(Tab.1 in Paper). Yet, I believe your testing is quite close.

BTW, the FID of this model "1222_mld_humanml3d_FID041.ckpt" should be 0.41, like the name. The mean/std files could cause this downgrade. Please pay attention to the following:

If your demo results have a severe issue on foot sliding, please take a look to the below. It could happen when ``self.feats2joints`` (use mean and std for de-normalization) is broken.

@layumi
Copy link
Author

layumi commented May 9, 2023

Thanks a lot @ChenFengYe ^_^

I am checking the dataset.

Could I have your paper model with 47.3 to evaluate as well?

Cheers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants