Revert unresolved changes in Bias initialization

it should be zeros_ or parameterized in future properly.
2022-10-27 14:44:53 +09:00 · 2022-10-27 14:44:53 +09:00 · 029d7c7543
commit 029d7c7543
parent cc56df996e
1 changed files with 1 additions and 1 deletions
--- a/modules/hypernetworks/hypernetwork.py
+++ b/modules/hypernetworks/hypernetwork.py
@ -75,7 +75,7 @@ class HypernetworkModule(torch.nn.Module):
                    w, b = layer.weight.data, layer.bias.data
                    if weight_init == "Normal" or type(layer) == torch.nn.LayerNorm:
                        normal_(w, mean=0.0, std=0.01)
-                        normal_(b, mean=0.0, std=0.005)
+                        normal_(b, mean=0.0, std=0)
                    elif weight_init == 'XavierUniform':
                        xavier_uniform_(w)
                        zeros_(b)