Video Frame Interpolation Rankings
and Video Deblurring Rankings

Researchers! Please develope joint video deblurring and frame interpolation models, incorporate InterpAny-Clearer in your models and train at least one of your models on Style loss, also called Gram matrix loss (the best perceptual loss function):

Source: FILM - Loss Functions Ablation https://film-net.github.io/

List of Rankings

Each ranking includes only the best model for one method.

The rankings exclude all event-based and spike-guided models.

Joint Video Deblurring and Frame Interpolation Rankings

1. 👑 RBI with real motion blur✔️: LPIPS😍 (no data)
   This will be the King of all rankings. We look forward to ambitious researchers.
2. RBI with real motion blur✔️: PSNR😞>=28.5dB
3. Adobe240 (640×352) with synthetic motion blur✖️: LPIPS😍 (no data)
4. Adobe240 (640×352) with synthetic motion blur✖️: PSNR😞>=33.3dB
5. Adobe240 (5:8) with synthetic motion blur✖️: LPIPS😍 (no data)
6. Adobe240 (5:8) with synthetic motion blur✖️: PSNR😞>=25dB

Video Deblurring Rankings

• (to do)

Video Frame Interpolation Rankings

1. Vimeo-90K triplet: LPIPS😍(SqueezeNet)<=0.014
2. Vimeo-90K triplet: LPIPS😍<=0.018
3. Vimeo-90K triplet: PSNR😞>=36dB
4. Vimeo-90K septuplet: LPIPS😍<=0.032
5. Vimeo-90K septuplet: PSNR😞>=36dB

Appendices

• Appendix 1: Rules for qualifying models for the rankings (to do)
• Appendix 2: Metrics selection for the rankings
• Appendix 3: List of all research papers from the above rankings

---

RBI with real motion blur✔️: PSNR😞>=28.5dB

RK	Model	PSNR ↑    {Input fr.}	Training dataset	Official   repository	Practical model	VapourSynth
1	Pre-BiT++	31.32 {3}	Pretraining: Adobe240 Training: RBI			-
2	DeMFI-Netrb(5,3)	29.03 {4}	RBI		-	-
3	PRF4 -Large ENH:	28.55 {5}	RBI		-	-

Adobe240 (640×352) with synthetic motion blur✖️: PSNR😞>=33.3dB

RK	Model	PSNR ↑    {Input fr.}	Originally announced or Training dataset	Official   repository	Practical model	VapourSynth
1	BVFI	35.43 {4}	Adobe240	-	-	-
2	BiT++	34.97 {3}	Adobe240			-
3	DeMFI-Netrb(5,3)	34.34 {4}	Adobe240		-	-
4	ALANET	33.34dB 1	August 2020 1		-	-
5	PRF4 -Large	33.32dB 2	February 2020 3		-	-

Adobe240 (5:8) with synthetic motion blur✖️: PSNR😞>=25dB

RK	Model	PSNR ↑	Originally announced	Official   repository	Practical model	VapourSynth
1	VIDUE	28.74dB 4	March 2023 4		-	-
2	FLAVR	27.23dB 4	December 2020 5		-	-
3	UTI-VFI	26.69dB 4	December 2020 6		-	-
4	DeMFI	25.71dB 4	November 2021 7		-	-

Vimeo-90K triplet: LPIPS😍(SqueezeNet)<=0.014

RK	Model	LPIPS ↓	Originally announced	Official   repository	Practical model	VapourSynth
1	CDFI w/ adaP/U	0.008 8	March 2021 9		-	-
2	EDSC_s-𝓛F	0.010 9	June 2020 10		EDSC_s-𝓛F	-
3	DRVI	0.013 11	August 2021 11	-	-	-

Vimeo-90K triplet: LPIPS😍<=0.018

RK	Model	LPIPS ↓    {Input fr.}	Training dataset	Official   repository	Practical model	VapourSynth
1	EAFI-𝓛ecp	0.012 {2}	Vimeo-90K triplet	-	EAFI-𝓛ecp	-
2	UGFI 𝓛S	0.0126 {2}	Vimeo-90K triplet	-	UGFI 𝓛S	-
3	SoftSplat - 𝓛F	0.013 {2}	Vimeo-90K triplet		SoftSplat - 𝓛F	-
4	FILM-𝓛S	0.0132 {2}	Vimeo-90K triplet		FILM-𝓛S	-
5	EDSC_s-𝓛F	0.016 {2}	Vimeo-90K triplet		EDSC_s-𝓛F	-
6	CtxSyn - 𝓛F	0.017 {2}	proprietary	-	CtxSyn - 𝓛F	-
7	PerVFI	0.018 {2}	Vimeo-90K triplet		PerVFI	-

Vimeo-90K triplet: PSNR😞>=36dB

RK	Model	PSNR ↑    {Input fr.}	Originally announced or Training dataset	Official   repository	Practical model	VapourSynth
1	MA-GCSPA triplet-trained	36.76dB 12	March 2022 12		-	-
2	VFIformer + HRFFM ENH:	36.69 {2}	Vimeo-90K triplet	ENH: -	-	-
3	LADDER-L	36.65 {2}	Vimeo-90K triplet	-	-	-
4	EMA-VFI	36.64dB 13	March 2023 13		-	-
5	DQBC-Aug	36.57dB 14	April 2023 14		-	-
6	TTVFI	36.54dB 15	July 2022 15		-	-
7	AMT-G	36.53dB 16	April 2023 16		-	-
8	AdaFNIO	36.50dB 17	November 2022 17		-	-
9	FGDCN-L	36.46dB 18	November 2022 18		-	-
10	VFIFT	36.43 {2}	Vimeo-90K triplet	-	-	-
11	UPR-Net LARGE	36.42dB 19	November 2022 19		-	-
12	EAFI-𝓛ecc	36.38dB 20	July 2022 20	-	EAFI-𝓛ecp	-
13	H-VFI-Large	36.37dB 21	November 2022 21	-	-	-
14	UGFI 𝓛1	36.34 {2}	Vimeo-90K triplet	-	UGFI 𝓛S	-
15	VFIT-B	36.33 {2}	?		-	-
16	SoftSplat - 𝓛Lap with ensemble	36.28dB 22	March 2020 23		SoftSplat - 𝓛F	-
17	NCM-Large	36.22dB 24	July 2022 24	-	-	-
18-19	IFRNet large	36.20dB 25	May 2022 25		-	-
18-19	RAFT-M2M++ ENH:	36.20 {2}	Vimeo-90K triplet		-	-
20-21	EBME-H*	36.19dB 26	June 2022 26		-	-
20-21	RIFE-Large	36.19 {2}	Vimeo-90K triplet		RIFE v4.17
22-23	ABME	36.18dB 27	August 2021 27		-	-
22-23	ProBoost-Net	36.18 {2}	?	-	-	-
24	TDPNetnv w/o MRTM	36.069 {2}	Vimeo-90K triplet	-	TDPNet	-
25	FILM-𝓛1	36.06 {2}	Vimeo-90K triplet		FILM-𝓛S	-

Vimeo-90K septuplet: LPIPS😍<=0.032

RK	Model	LPIPS ↓	Originally announced	Official   repository	Practical model	VapourSynth
1	RIFE	0.0233 28	November 2020 29		RIFE v4.17
2	IFRNet	0.0274 28	May 2022 25		-	-
3	VFIT-B	0.0304 28	November 2021 30		-	-
4	ABME	0.0309 28	August 2021 27		-	-

Vimeo-90K septuplet: PSNR😞>=36dB

RK	Model	PSNR ↑    {Input fr.}	Originally announced or Training dataset	Official   repository	Practical model	VapourSynth
1	JNMR	37.19dB 31	June 2022 31		-	-
2	VFIT-B	36.96 {4}	Vimeo-90K septuplet		-	-
3	VRT	36.53dB 32	June 2022 (VFI) 32		-	-
4	ST-MFNet	36.507dB 33	November 2021 34		-	-
5	MA-GCSPA septuplet-trained	36.50dB 12	March 2022 12		-	-
6	EDENVFI PVT(15,15)	36.387dB 33	July 2023 33	-	-	-
7	IFRNet	36.37 {2}	Vimeo-90K septuplet		-	-
8	RN-VFI	36.33 {4}	Vimeo-90K septuplet	-	-	-
9	FLAVR	36.25dB 5	December 2020 5		-	-
10	DBVI	36.17dB 35	October 2022 35		-	-
11	EDC	36.14dB 31	February 2022 36		-	-

Appendix 2: Metrics selection for the rankings

Currently, the most commonly used metrics in the existing works on video frame interpolation and video deblurring are: PSNR, SSIM and LPIPS. Exactly in that order.

The main purpose of creating my rankings is to look for the best perceptually-oriented model for practical applications - hence the primary metric in my rankings will be the most common perceptual image quality metric in scientific papers: LPIPS.

At the time of writing these words, in October 2023, in relation to VFI, I have only found another perceptual image quality metric - DISTS in one paper: and also in one paper I found a bespoke VFI metric - FloLPIPS [arXiv]. Unfortunately, both of these papers omit to evaluate the best performing models based on the LPIPS metric. If, in the future, some researcher will evaluate LPIPS top-performing models using alternative, better perceptual metrics, I would of course be happy to add rankings based on those metrics.

I would like to use only one metric - LPIPS. Unfortunately still many of the best VFI and video deblurring methods are only evaluated using PSNR or PSNR and SSIM. For this reason, I will additionally present rankings based on PSNR, which will show the models that can, after perceptually-oriented training, be the best for practical applications, as well as providing a source of knowledge for building even better practical models in the future.

I have decided to completely abandon rankings based on the SSIM metric. Below are the main reasons for this decision, ranked from the most important to the less important.

• The main reason is the following quote, which I found in a paper by researchers at Adobe Research: ²². In the quote they refer to a paper by researchers at NVIDIA: [arXiv].

  We limit the evaluation herein to the PSNR metric since SSIM [57] is subject to unexpected and unintuitive results [39].

• The second reason is, more and more papers are appearing where PSNR scores are given, but without SSIM: ³³ and A model from such a paper appearing only in the PSNR-based ranking and at the same time not appearing in the SSIM-based ranking may give the misleading impression that the SSIM score is so poor that it does not exceed the ranking eligibility threshold, while there is simply no SSIM score in a paper.

• The third reason is, that often the SSIM scores of individual models are very close to each other or identical. This is the case in the SNU-FILM Easy test, as shown in Table 3: [CVPR 2023], where as many as 6 models achieve the same score of 0.991 and as many as 5 models achieve the same score of 0.990. In the same test, PSNR makes it easier to determine the order of the ranking, with the same number of significant digits.

• The fourth reason is that PSNR-based rankings are only ancillary when a model does not have an LPIPS score. For this reason, SSIM rankings do not add value to my repository and only reduce its readability.

• The fifth reason is that I want to encourage researchers who want to use only two metrics in their paper to use LPIPS and PSNR instead of PSNR and SSIM.

• The sixth reason is that the time saved by dropping the SSIM-based rankings will allow me to add new rankings based on other test data, which will be more useful and valuable.

Appendix 3: List of all research papers from the above rankings

📝 Note: Temporarily, the following list contains full descriptions of those methods that have been removed from the footnotes or not included in the footnotes at all due to the new layout of the tables.

Method	Paper	Venue
ABME
AdaFNIO
ALANET
AMT
BIN	Blurry Video Frame Interpolation
BiT	Blur Interpolation Transformer for Real-World Motion from Blur
BVFI	Three-Stage Cascade Framework for Blurry Video Frame Interpolation
CDFI
CtxSyn	Context-aware Synthesis for Video Frame Interpolation
DBVI
DeMFI	DeMFI: Deep Joint Deblurring and Multi-Frame Interpolation with Flow-Guided Attentive Correlation and Recursive Boosting
DQBC
DRVI
EAFI	Error-Aware Spatial Ensembles for Video Frame Interpolation
EBME
EDC
EDENVFI
EDSC	Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution
EMA-VFI
FGDCN
FILM	FILM: Frame Interpolation for Large Motion
FLAVR
HRFFM	Video Frame Interpolation with Region-Distinguishable Priors from SAM
H-VFI
IFRNet	IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation
JNMR
LADDER	LADDER: An Efficient Framework for Video Frame Interpolation
M2M	Many-to-many Splatting for Efficient Video Frame Interpolation
MA-GCSPA
NCM
PerVFI	Perceptual-Oriented Video Frame Interpolation Via Asymmetric Synergistic Blending
PRF	Video Frame Interpolation and Enhancement via Pyramid Recurrent Framework
ProBoost-Net	Progressive Motion Boosting for Video Frame Interpolation
RIFE	Real-Time Intermediate Flow Estimation for Video Frame Interpolation
RN-VFI	Range-nullspace Video Frame Interpolation with Focalized Motion Estimation
SoftSplat	Softmax Splatting for Video Frame Interpolation
SSR	Video Frame Interpolation with Many-to-many Splatting and Spatial Selective Refinement
ST-MFNet
TDPNet	Textural Detail Preservation Network for Video Frame Interpolation
TTVFI
UGFI	Frame Interpolation Transformer and Uncertainty Guidance
UPR-Net
UTI-VFI
VFIformer	Video Frame Interpolation with Transformer
VFIFT	Video Frame Interpolation with Flow Transformer
VFIT	Video Frame Interpolation Transformer
VIDUE
VRT

Footnotes

1. ALANET: Adaptive Latent Attention Network for Joint Video Deblurring and Interpolation [MM 2020] [arXiv] ↩ ↩²

2. Video Frame Interpolation and Enhancement via Pyramid Recurrent Framework [TIP 2020] ↩

3. Blurry Video Frame Interpolation [CVPR 2020] [arXiv] ↩

4. Joint Video Multi-Frame Interpolation and Deblurring under Unknown Exposure Time [CVPR 2023] [arXiv] ↩ ↩² ↩³ ↩⁴ ↩⁵

5. FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation [WACV 2023] [arXiv] ↩ ↩² ↩³

6. Video Frame Interpolation without Temporal Priors [NeurIPS 2020] [arXiv] ↩

7. DeMFI: Deep Joint Deblurring and Multi-Frame Interpolation with Flow-Guided Attentive Correlation and Recursive Boosting [ECCV 2022] [arXiv] ↩

8. AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling [TIP 2022] [arXiv] ↩

9. CDFI: Compression-Driven Network Design for Frame Interpolation [CVPR 2021] [arXiv] ↩ ↩²

10. Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolution [TPAMI 2021] [arXiv] ↩

11. DRVI: Dual Refinement for Video Interpolation [Access 2021] ↩ ↩²

12. Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolation [CVPR 2023] [arXiv] ↩ ↩² ↩³ ↩⁴

13. Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation [CVPR 2023] [arXiv] ↩ ↩²

14. Video Frame Interpolation with Densely Queried Bilateral Correlation [IJCAI 2023] [arXiv] ↩ ↩²

15. TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation [TIP 2023] [arXiv] ↩ ↩²

16. AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation [CVPR 2023] [arXiv] ↩ ↩²

17. AdaFNIO: Adaptive Fourier Neural Interpolation Operator for video frame interpolation [arXiv] ↩ ↩²

18. Flow Guidance Deformable Compensation Network for Video Frame Interpolation [TMM 2023] [arXiv] ↩ ↩²

19. A Unified Pyramid Recurrent Network for Video Frame Interpolation [CVPR 2023] [arXiv] ↩ ↩²

20. Error-Aware Spatial Ensembles for Video Frame Interpolation [arXiv] ↩ ↩²

21. H-VFI: Hierarchical Frame Interpolation for Videos with Large Motions [arXiv] ↩ ↩²

22. Revisiting Adaptive Convolutions for Video Frame Interpolation [WACV 2021] [arXiv] ↩ ↩²

23. Softmax Splatting for Video Frame Interpolation [CVPR 2020] [arXiv] ↩

24. Neighbor Correspondence Matching for Flow-based Video Frame Synthesis [MM 2022] [arXiv] ↩ ↩²

25. IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation [CVPR 2022] [arXiv] ↩ ↩² ↩³

26. Enhanced Bi-directional Motion Estimation for Video Frame Interpolation [WACV 2023] [arXiv] ↩ ↩²

27. Asymmetric Bilateral Motion Estimation for Video Frame Interpolation [ICCV 2021] [arXiv] ↩ ↩² ↩³

28. Exploring Discontinuity for Video Frame Interpolation [CVPR 2023] [arXiv] ↩ ↩² ↩³ ↩⁴

29. Real-Time Intermediate Flow Estimation for Video Frame Interpolation [ECCV 2022] [arXiv] ↩

30. Video Frame Interpolation Transformer [CVPR 2022] [arXiv] ↩

31. JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation [TIP 2023] [arXiv] ↩ ↩² ↩³

32. VRT: A Video Restoration Transformer [arXiv] ↩ ↩²

33. Efficient Convolution and Transformer-Based Network for Video Frame Interpolation [ICIP 2023] [arXiv] ↩ ↩² ↩³ ↩⁴

34. ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolation [CVPR 2022] [arXiv] ↩

35. Deep Bayesian Video Frame Interpolation [ECCV 2022] ↩ ↩²

36. Enhancing Deformable Convolution based Video Frame Interpolation with Coarse-to-fine 3D CNN [ICIP 2022] [arXiv] ↩