Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

torch.load non backwards compatible on Transformer between 1.8.1 and 1.9.0 #60165

Open
jhoareau opened this issue Jun 17, 2021 · 7 comments 路 May be fixed by #126568
Open

torch.load non backwards compatible on Transformer between 1.8.1 and 1.9.0 #60165

jhoareau opened this issue Jun 17, 2021 · 7 comments 路 May be fixed by #126568
Labels
module: serialization Issues related to serialization (e.g., via pickle, or otherwise) of PyTorch objects triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@jhoareau
Copy link

jhoareau commented Jun 17, 2021

馃悰 Bug

A Transformer that I used torch.save on with Pytorch 1.8.1 cannot load on Pytorch 1.9.0.

To Reproduce

Steps to reproduce the behavior:

On Pytorch 1.8.1:

import torch
from torch.nn import Transformer
torch.save(Transformer(), 'transformer.pt')

Then update Pytorch to 1.9.0

import torch
torch.load('transformer.pt')

The following stacktrace will show:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../python3.8/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File ".../python3.8/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File ".../python3.8/site-packages/torch/serialization.py", line 875, in find_class
    return super().find_class(mod_name, name)
AttributeError: Can't get attribute '_LinearWithBias' on <module 'torch.nn.modules.linear' from '.../python3.8/site-packages/torch/nn/modules/linear.py'>

Expected behavior

torch.save() on Pytorch X should be able to load modules that are saved on previous versions (#6801 (comment) says that torch.load is supposed to be backwards-compatible)

In this specific case, redefining the _LinearWithBias module in torch.nn.modules.linear would solve the issue.

Environment

  • PyTorch Version (e.g., 1.0): 1.9.0
  • OS (e.g., Linux): macOS (but it also happens on Linux with a model loaded & saved there)
  • How you installed PyTorch (conda, pip, source): pip
  • Python version: 3.8.7
  • CUDA/cuDNN version: None
  • GPU models and configuration: No

cc @mruberry

@jhoareau
Copy link
Author

@bhosmer It looks like the breaking of backwards compatibility was introduced in your PR here #58727

@jhoareau
Copy link
Author

And @gchanan #52537 (comment) - this might also be related to this issue.

@gchanan
Copy link
Contributor

gchanan commented Jun 17, 2021

see https://pytorch.org/docs/stable/notes/serialization.html#id4 -- you should serialize the state_dict, not the module.

@zou3519 zou3519 added module: serialization Issues related to serialization (e.g., via pickle, or otherwise) of PyTorch objects triage review triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Jun 17, 2021
@zou3519
Copy link
Contributor

zou3519 commented Jun 17, 2021

Marking for triage review because I think we should really do something about https://pytorch.org/docs/stable/notes/serialization.html#id4 -- this is not the first time this has happened

@mruberry
Copy link
Collaborator

I think the doc is correct, though, right? It's just undiscoverable?

@jhoareau
Copy link
Author

jhoareau commented Jun 18, 2021

https://pytorch.org/tutorials/beginner/saving_loading_models.html#save-load-entire-model is mentioning the risk in refactoring our own application but does not warn about the risk of Pytorch cross-version incompatibilities for PyTorch-provided model architectures.

@mruberry
Copy link
Collaborator

cc @brianjo for the tutorial update

@z-a-f z-a-f linked a pull request May 17, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: serialization Issues related to serialization (e.g., via pickle, or otherwise) of PyTorch objects triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants