The parameter grad_norm appears to be inf and then nan when input resolution is 112*112 during the pre-training on VIT-Small backbone

[DragonWang-cell]

Hello, thank you very much for your significant contribution to the computer vision community! When I set my input resolution to 112*112 and do the pre-training on VIT-Small backbone the parameter grad_norm appears to be inf and then nan and then back to normal, is this normal or abnormal? If the training is abnormal what should I do to avoid this, looking forward and thanking you for your answer!

[congee524]

I think the training is normal. How's it going back then?