“讓Keras更酷一些！”：隨意的輸出和靈活的歸一化

By蘇劍林|2019-01-27

下面是利用這個思路實現的譜歸一化（Spectral Normalization）：

class SpectralNormalization:
    """層的一個包裝，用來加上SN。
    """

    def __init__(self, layer):
        self.layer = layer

    def spectral_norm(self, w, r=5):
        w_shape = K.int_shape(w)
        in_dim 
 
= np.prod(w_shape[:-1]).astype(int)
        out_dim = w_shape[-1]
        w = K.reshape(w, (in_dim, out_dim))
        u = K.ones((1, in_dim))
        for i in range(r):
            v = K.l2_normalize(K.dot(u, w))
            u = K.l2_normalize(K.dot(v, K.transpose(w)))
        return K.sum(K.dot(K.dot(u, w), K.transpose(v)))

    
 
def spectral_normalization(self, w):
        return w / self.spectral_norm(w)

    def __call__(self, inputs):
        with K.name_scope(self.layer.name):
            if not self.layer.built:
                input_shape = K.int_shape(inputs)
                self.layer.build(input_shape)
                self.layer.built 
 
= True
                if self.layer._initial_weights is not None:
                    self.layer.set_weights(self.layer._initial_weights)
        if not hasattr(self.layer, 'spectral_normalization'):
            if hasattr(self.layer, 'kernel'):
                self.layer.kernel = self.spectral_normalization(self.layer.kernel)
            if hasattr(self.layer, 'gamma'):
                self.layer.gamma = self.spectral_normalization(self.layer.gamma)
            self.layer.spectral_normalization = True
        return self.layer(inputs)

使用方法為

x = SpectralNormalization(Dense(100, activation='relu'))(x)

也就是定義完層之後加個SpectralNormalization修改一下就行了。至於原理，我們只需要觀察__call__部分，首先新建立的層是built=False的，然後我們自己手動執行build方法，然後對原來的權重進行歸一化，並賦值覆蓋原來的權重，即self.layer.kernel=self.spectral_normalization(self.layer.kernel)這一句。