redundant memory allocation maybe the root cause of OOMs

[jinsong-mao]

Hi @xuzhao9 ,

during the investigation of LLAMA_7b OOM issue, we found that there are many redundant memory allocation. maybe it's not necessary for test.
1, there is deepcopy for maybe_cast() and deepcopy_and_maybe_cast(). which would duplicate the memory on GPU allocated for this model.
https://github.com/pytorch/benchmark/blob/main/userbenchmark/dynamo/dynamobench/common.py#L2400
https://github.com/pytorch/benchmark/blob/main/userbenchmark/dynamo/dynamobench/common.py#L2403

looks we need to check more strictly on deepcopy.

2, there is deepcopy in validate_model() too.
https://github.com/pytorch/benchmark/blob/main/userbenchmark/dynamo/dynamobench/common.py#L1918

we can run the LLAMA_7b model(which has OOM issue previously #2051 ) with one A100 40G after commenting out the unnecessary deepcopy().

hope this information can help on fixing the OOM issues in this repo.

Thanks

[xuzhao9]

dynamobench is owned by the PT2 team. In my understanding, it is used for accuracy check because some models are stateful.
cc @desertfire is there a way to turn off deepcopy in dynamobench?