For A B matrix in LoRA, if A is zero and B is Gaussian initialized, if this will be different? #135
Unanswered
Henry-Xing
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
https://github.com/microsoft/LoRA/blob/3f5c193f431c8a09448f0184f6f883ad393f22d0/loralib/layers.py#L59C40-L59C40
in this code, the commence said A will be gaussian and B will be zero initialize but it seems to be opposite, will this matters? And what if using another way to inltialize A and B?
Beta Was this translation helpful? Give feedback.
All reactions