-
Notifications
You must be signed in to change notification settings - Fork 302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error when pruning a group with GLU Layer #366
Comments
class CustomGLU(nn.Module):
def __init__(self, dim=1):
super(CustomGLU, self).__init__()
self.dim = dim
def forward(self, x):
first_half, second_half = torch.split(x, x.size(self.dim)//2, dim=self.dim)
return first_half * torch.sigmoid(second_half) If you don't use it extensively, the performance degradation shouldn't be significant. |
Hi @janthmueller, thanks for the workaround, I tried that but the network comes back with only the last conv layer pruned. No dep group with first conv layer is being returned for pruning. |
After running the To accommodate this scenario, it's crucial to apply a targeted adjustment before gathering the To implement this adjustment, insert the following code snippet before collecting for dep, _ in group:
if isinstance(dep.target.module, ops._SplitOp):
n_pruned = n_pruned // 2
break By incorporating this adjustment, the pruning mechanism can appropriately handle scenarios involving the custom glu operation, ensuring accurate pruning outcomes. I think it might be best to fix this for all possible scenarios including a split, maybe similar to |
Great, thanks for the workaround and the explanation! It would be great to have this merged such that the lib works directly on GLU! |
Pruning a model with GLU results in an error when finding importance. GLU does not have any params but halves the input (in the given dimension). This is not accounted for during tracing, assigning indices, and finding importance.
Here's a minimal example with a simple model
I then prune using GroupNormPruner
This gives an index out of bounds error
Note that this error is raised when returning the group, so setting
interative=True
in pruner step does not help.The text was updated successfully, but these errors were encountered: