t2m checkpoint model size error

[yhyu13]

I've finished setup with py39 and torch 1.12.1 and ran

python demo.py --cfg ./configs/config_mld_humanml3d.yaml --cfg_assets ./configs/assets.yaml --example ./demo/example.txt

But there is an error in loading the pre-trained weight

Length-50: a man kicks with something or someone with his left leg.
Length-100: A person is skipping rope.
Length-100: a person walks backward slowly.
Pointer Pointing at 0
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
/home/hangyu5/anaconda3/envs/t2m_mld/lib/python3.9/site-packages/torch/cuda/__init__.py:146: UserWarning: 
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Traceback (most recent call last):
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/demo.py", line 337, in <module>
    main()
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/demo.py", line 105, in main
    model = get_model(cfg, dataset)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/get_model.py", line 7, in get_model
    return get_module(cfg, datamodule)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/get_model.py", line 17, in get_module
    return Model(cfg=cfg, datamodule=datamodule)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/modeltype/mld.py", line 86, in __init__
    self._get_t2m_evaluator(cfg)
  File "/home/hangyu5/Documents/Git-repoMy/AIResearchVault/repo/AIGC/Text2Motion/motion-latent-diffusion/mld/models/modeltype/mld.py", line 175, in _get_t2m_evaluator
    self.t2m_moveencoder.load_state_dict(
  File "/home/hangyu5/anaconda3/envs/t2m_mld/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for MovementConvEncoder:
	size mismatch for main.0.weight: copying a param with shape torch.Size([512, 259, 4]) from checkpoint, the shape in current model is torch.Size([512, 247, 4]).

Is it an known issue?

[billl-jiang]

Hi @yhyu13

From the error log, it seems that the input size of evaluators is not correct which indicates the feature size of the dataset is not correct.

motion-latent-diffusion/mld/models/modeltype/mld.py

Lines 157 to 161 in c28a064

	self.t2m_moveencoder = t2m_motionenc.MovementConvEncoder(
	input_size=cfg.DATASET.NFEATS - 4,
	hidden_size=cfg.model.t2m_motionencoder.dim_move_hidden,
	output_size=cfg.model.t2m_motionencoder.dim_move_latent,
	)

Maybe the problem is caused by the incorrect dataset. The feature size of the HumanML3D data format is 263 and the feature size of the KIT-ML data format is 251. From the log it seems that your HumanML3d dataset feature size is 251 so maybe you are loading the KIT-ML data for the HumanML3D config.

1. First you can check whether the path of HumanML3D is correct.

   motion-latent-diffusion/configs/assets.yaml

   Lines 13 to 14 in c28a064

   	HUMANML3D:
   	ROOT: './datasets/humanml3d' # HumanML3D directory

2. Then you can use NumPy to load the npy files in the HumanML3D folder manually and check whether the shape is [motion_length, 263] which 263 is the feature size of HumanML3D data format.

You can refer to HumanML3D for more details about the HumanML3D dataset.