MTSN-Spot-ME-MaE

This is the code repository for the FME 2022 accepted paper:
MTSN: A Multi-Temporal Stream Network for Spotting Facial Macro- and Micro-Expression with Hard and Soft Pseudo-labels.

Network

Results

Test on MEGC 2022 unseen datasets

Step 1) Installation of packages using pip

pip install -r requirements.txt

Step 2) Download processed optical flow features from :

https://drive.google.com/file/d/1Cn4rux-Hwrt6E1LWO3VL3ddNqOwmgP71/view?usp=sharing

Step 3) Place the folder (megc2022-processed-data) accordingly:

├─megc2022-pretrained-weights
├─megc2021-ground-truth
├─megc2022-processed-data
├─......
├─test_main.py
└─......

Step 4) Evaluation for MEGC 2022 unseen datasets CAS(ME)³ and SAMM Challenge

python test_main.py

Train on MEGC 2021 datasets

Step 1) Installation of packages using pip

pip install -r requirements.txt

Step 2) Download processed optical flow features from :

https://drive.google.com/file/d/1UVnJtZoCZK5nmMbt1zhIhYkJYPW3Tc6o/view?usp=sharing

Step 3) Place the folder (megc2021-processed-data) accordingly:

├─megc2022-pretrained-weights
├─megc2021-ground-truth
├─megc2021-processed-data
├─......
├─train_main.py
└─......

Step 4) Train on MEGC 2021 dataset CAS(ME)² and SAMM Long Videos

python train_main.py --dataset_name CASME_sq

Note for parameter settings

  --dataset_name (CASME_sq or SAMMLV)

Additional Notes

If you have issue installing torch, run this:
pip install torch===1.5.0 torchvision===0.6.0 torchsummary==1.5.1 -f https://download.pytorch.org/whl/torch_stable.html

Link to research paper

If you find this work useful, please cite the paper: https://dl.acm.org/doi/abs/10.1145/3552465.3555040

@inproceedings{liong2022mtsn,
title={MTSN: A Multi-Temporal Stream Network for Spotting Facial Macro-and Micro-Expression with Hard and Soft Pseudo-labels},
author={Liong, Gen Bing and Liong, Sze-Teng and See, John and Chan, Chee-Seng},
booktitle={Proceedings of the 2nd Workshop on Facial Micro-Expression: Advanced Techniques for Multi-Modal Facial Expression Analysis},
pages={3--10},
year={2022}
}

Please email me at genbing67@gmail.com if you have any inquiries or issues.