Exported HF models contain SoftmaxCrossEntropyLoss node

[yuanyao-nv]

I notice that using the torchscript exporter, ie using the --torchscript-onnx flag, all the exported HF models contain a SoftmaxCrossEntropyLoss node at the end that compares the model output with the true labels. Why is training-related ops showing up in the exported model and is there a way to disable such ops?

The export command I'm using is:

python pytorch/benchmarks/dynamo/huggingface.py --performance --amp -dcuda --output=/workspace/output/dynamo-onnx_huggingface_amp_inference_cuda_performance.csv --inference --use-eval-mode -n1 --torchscript-onnx --no-skip --dashboard -k <model_name> 

[titaiwangms]

I think this is why https://github.com/pytorch/pytorch/blob/8184cd85fcfe663019edb3c1e502e03dcbaba4f0/benchmarks/dynamo/common.py#L3785-L3787C14

[titaiwangms]

Maybe you would want to hack around here to make it eval mode: https://github.com/pytorch/pytorch/blob/01694eaa56adb343f5d3d15b53d2962615dafe17/benchmarks/dynamo/huggingface.py#L513-L520