投稿
  2. 程式人生 >實用技巧 >其它 >Tensorflow2.0：（三）神經網路搭建八股

Tensorflow2.0：（三）神經網路搭建八股

阿新 • 來源：網路 • 發佈：2021-05-06

第三章 神經網路搭建八股

本次Tensorflow2.0學習筆記參考北京大學曹健老師《人工智慧實踐：Tensorflow筆記》課程。
課程連結：https://www.icourse163.org/course/PKU-1002536002
B站連結：https://www.bilibili.com/video/BV1B7411L7Qt?p=1
課程程式碼及自制資料集下載：https://github.com/jlff/tf2_notes
該課程適合無基礎的初學者入門，在保證主線完整的前提下，各章節略有增刪改簡。

學習目的

  • 神經網路搭建八股
  • iris資料集識別鳶尾花程式碼復現
  • MNIST資料集識別手寫數字程式碼復現
  • FASHION資料集識別衣褲鞋包程式碼復現

一、六步法

  1. keras簡介
  • 什麼是keras？keras 是一個用 Python 編寫的高階神經網路 API，它能夠以 TensorFlow, CNTK, 或者 Theano 作為後端執行。它的開發由Google支援，支援GPU和CPU，主要包括Models API、Layers API、Callbacks API、Data preprocessing、Optimizers、Metrics、Losses、Built-in small datasets、Keras Applications、Utilities等10個模組包，參見文件：https://keras.io/api/
  • keras vs. tf.keras? TensorFlow 2.0釋出後， keras 正式成為 TensorFlow 的官方高階 API。2019年隨著 keras 2.3.0 的釋出，keras的建立者和首席維護者Francois Chollet 宣告： keras v2.3.0 是首個與 tf.keras 同步的版本，也將是最後一個支援除 TensorFlow 以外的後端（即 Theano，CNTK 等）的最終版本。
  1. keras搭建神經網路六步法
  • 第一步：import
    import相關模組，如import tensorflow as tf
  • 第二步：train，test
    指定輸入網路的訓練集和測試集
  • 第三步：model = tf.keras.models.Sequential / class MyModel(Model) model=MyModel
    逐層搭建網路結構
model = tf.keras.models.Sequential([網路結構]) #描述各層網路
"""
例如：
拉直層：tf.keras.layers.Flatten()
全連線層：tf.keras.layers.Dense(神經元個數, activation="**函式“ ,kernel_regularizer=哪種正則化)
activation（字串給出）可選: relu、softmax、sigmoid 、tanh
kernel_regularizer可選:tf.keras.regularizers.l1()、tf.keras.regularizers.l2()
卷積層：tf.keras.layers.Conv2D(filters = 卷積核個數, kernel_size= 卷積核尺寸,
strides = 卷積步長，padding = " valid" or "same")
LSTM層：tf.keras.layers.LSTM()
"""

使用Sequential可以快速搭建網路結構，但是如果網路包含跳連等其他複雜網路結構，Sequential就無法表示了。這時就需要使用class來自定義網路結構。

class MyModel(Model):
	def __init__(self):
	# 定義網路結構塊
		super(MyModel, self).__init__()
	def call(self, x):
	# 呼叫網路結構塊，實現前向傳播
	return y
	
model= MyModel()
  • 第四步：model.compile
    配置訓練方法：優化器、損失函式和最終評價指標。優化器、損失函式在第二章中有介紹。
model.compile(optimizer = 優化器, loss = 損失函式, metrics = [“準確率”])

"""
Optimizer可選:
‘sgd’ or tf.keras.optimizers.SGD(lr=學習率,momentum=動量引數)
‘adagrad’ or tf.keras.optimizers.Adagrad(lr=學習率,decay=學習率衰減率)
‘adadelta’ or tf.keras.optimizers.Adadelta(lr=學習率,decay=學習率衰減率)
‘adam’ or tf.keras.optimizers.Adam(lr=學習率, beta_1=0.9, beta_2=0.999)
loss可選:
‘mse’ or tf.keras.losses.MeanSquaredError()
‘sparse_categorical_crossentropy’ or tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
Metrics可選:
‘accuracy’ ：y_和y都是數值，如y_=[1] y=[1]
‘categorical_accuracy’ ：y_和y都是獨熱碼(概率分佈)，如y_=[0,1,0] y=[0.256,0.695,0.048]
‘sparse_categorical_accuracy’ ：y_是數值，y是獨熱碼(概率分佈),如y_=[1] y=[0.256,0.695,0.048]
"""
  • 第五步：model.fit
    執行訓練過程
model.fit(訓練集的輸入特徵, 訓練集的標籤, batch_size= , epochs= ,
validation_data=(測試集的輸入特徵，測試集的標籤),
validation_split=從訓練集劃分多少比例給測試集，validation_freq= 多少次epoch測試一次)
  • 第六步：model.summary
    列印網路結構，統計引數數目
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 3)                 15        
=================================================================
Total params: 15
Trainable params: 15
Non-trainable params: 0
_________________________________________________________________

二、iris資料集識別鳶尾花

使用Sequential

# 第一步import
import tensorflow as tf
from sklearn import datasets
import numpy as np

# 第二步train test
x_train = datasets.load_iris().data # 測試集的輸入特徵x_test和標籤y_test可以像x_train和y_train一樣直接從資料集獲取，也可以如上述在fit中按比例從訓練集中劃分，本例選擇從訓練集中劃分，所以只需載入x_train，y_train即可
y_train = datasets.load_iris().target

np.random.seed(116)
np.random.shuffle(x_train) # 將資料集亂序
np.random.seed(116)
np.random.shuffle(y_train)
tf.random.set_seed(116)

# 第三步models.Sequential
model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(3, activation='softmax', kernel_regularizer=tf.keras.regularizers.l2())
]) # 使用單層全連線網路，第一個引數表示神經元個數，第二個引數表示網路所使用的**函式，第三個引數表示選用的正則化方法

# 第四步model.compile
model.compile(optimizer=tf.keras.optimizers.SGD(lr=0.1),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy']) # 使用SGD優化器，並將學習率設定為0.1，選擇SparseCategoricalCrossentrop作為損失函式，輸出為概率分佈，所以metrics需要設定為sparse_categorical_accuracy

# 第五步model.fit
model.fit(x_train, y_train, batch_size=32, epochs=500, validation_split=0.2, validation_freq=20) # batch_size表示神經網路進行一次訓練樣本數，epochs表示所有樣本進行迭代的次數validation_split表示資料集中驗證集的劃分比例，validation_freq表示每迭代20次在測試集上測試一次準確率。

# 第六步model.summary()
model.summary()

...
Epoch 499/500
4/4 [==============================] - 0s 2ms/step - loss: 0.3691 - sparse_categorical_accuracy: 0.9306
Epoch 500/500
4/4 [==============================] - 0s 14ms/step - loss: 0.3634 - sparse_categorical_accuracy: 0.9304 - val_loss: 0.3516 - val_sparse_categorical_accuracy: 0.8667
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 3)                 15        
=================================================================
Total params: 15
Trainable params: 15
Non-trainable params: 0
_________________________________________________________________

使用自定義Class IrisModel(Model)

# 第一步import
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras import Model
from sklearn import datasets
import numpy as np

# 第二步train test
x_train = datasets.load_iris().data
y_train = datasets.load_iris().target

np.random.seed(116)
np.random.shuffle(x_train)
np.random.seed(116)
np.random.shuffle(y_train)
tf.random.set_seed(116)

# 第三步class IrisModel
class IrisModel(Model):
    def __init__(self):
        super(IrisModel, self).__init__()
        self.d1 = Dense(3, activation='sigmoid', kernel_regularizer=tf.keras.regularizers.l2())

    def call(self, x):
        y = self.d1(x)
        return y


model = IrisModel()

# 第四步model.compile
model.compile(optimizer=tf.keras.optimizers.SGD(lr=0.1),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

# 第五步model.fit
model.fit(x_train, y_train, batch_size=32, epochs=500, validation_split=0.2, validation_freq=20)

# 第六步model.summary()
model.summary()

...
Epoch 499/500
4/4 [==============================] - 0s 2ms/step - loss: 0.3691 - sparse_categorical_accuracy: 0.9306
Epoch 500/500
4/4 [==============================] - 0s 16ms/step - loss: 0.3634 - sparse_categorical_accuracy: 0.9304 - val_loss: 0.3516 - val_sparse_categorical_accuracy: 0.8667
Model: "iris_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              multiple                  15        
=================================================================
Total params: 15
Trainable params: 15
Non-trainable params: 0
_________________________________________________________________

三、MNIST資料集識別手寫數字

檢視資料格式

import tensorflow as tf
from matplotlib import pyplot as plt

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()

# 視覺化訓練集輸入特徵的第一個元素
plt.imshow(x_train[0], cmap='gray')  # 繪製灰度圖
plt.show()

# 打印出訓練集輸入特徵的第一個元素
print("x_train[0]:\n", x_train[0])
# 打印出訓練集標籤的第一個元素
print("y_train[0]:\n", y_train[0])

# 打印出整個訓練集輸入特徵形狀
print("x_train.shape:\n", x_train.shape)
# 打印出整個訓練集標籤的形狀
print("y_train.shape:\n", y_train.shape)
# 打印出整個測試集輸入特徵的形狀
print("x_test.shape:\n", x_test.shape)
# 打印出整個測試集標籤的形狀
print("y_test.shape:\n", y_test.shape)

Tensorflow2.0：（三）神經網路搭建八股

x_train[0]:
 [[  0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   3  18  18  18 126 136 175 26  166 255 247 127   0   0   0   0]
 [   0   0   0   0   0   0   0   0  30  36  94 154 170 253 253 253 253 253 225 172 253 242 195  64   0   0   0   0]
 [   0   0   0   0   0   0   0  49 238 253 253 253 253 253 253 253 253 251 93  82  82  56   39   0   0   0   0   0]
 [   0   0   0   0   0   0   0  18 219 253 253 253 253 253 198 182 247 241   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0  80 156 107 253 253 205  11   0  43 154   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0  14   1 154 253  90   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0 139 253 190   2   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0  11 190 253  70   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0  35 241 225 160 108   1   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0  81 240 253 253 119  25   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0  45 186 253 253 150  27   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  16  93 252 253 187   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 249 253 249  64   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0  46 130 183 253 253 207   2   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0  39 148 229 253 253 253 250 182   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0  24 114 221 253 253 253 253 201  78   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0  23  66 213 253 253 253 253 198  81   2   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0  18 171 219 253 253 253 253 195  80   9   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0  55 172 226 253 253 253 253 244 133  11   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0 136 253 253 253 212 135 132  16   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]
 [   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]]
y_train[0]:
 5
x_train.shape:
 (60000, 28, 28)
y_train.shape:
 (60000,)
x_test.shape:
 (10000, 28, 28)
y_test.shape:
 (10000,)

使用Sequential訓練模型

import tensorflow as tf

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0 # 將輸入特徵的灰度值歸一化到[0,1]區間，這可以使網路更快收斂

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(), # 輸入全連線網路時需要先將資料拉直為一維陣列，把784個畫素點的灰度值作為輸入特徵輸入神經網路
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()

...
Epoch 4/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.0554 - sparse_categorical_accuracy: 0.9830 - val_loss: 0.0746 - val_sparse_categorical_accuracy: 0.9769
Epoch 5/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0402 - sparse_categorical_accuracy: 0.9885 - val_loss: 0.0766 - val_sparse_categorical_accuracy: 0.9780
Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_1 (Flatten)          (None, 784)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 128)               100480    
_________________________________________________________________
dense_3 (Dense)              (None, 10)                1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

使用自定義Class MnistModel(Model)

import tensorflow as tf
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras import Model

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0


class MnistModel(Model):
    def __init__(self):
        super(MnistModel, self).__init__()
        self.flatten = Flatten()
        self.d1 = Dense(128, activation='relu')
        self.d2 = Dense(10, activation='softmax')

    def call(self, x):
        x = self.flatten(x)
        x = self.d1(x)
        y = self.d2(x)
        return y


model = MnistModel()

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()

...
Epoch 4/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9827 - val_loss: 0.0786 - val_sparse_categorical_accuracy: 0.9763
Epoch 5/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0443 - sparse_categorical_accuracy: 0.9869 - val_loss: 0.0774 - val_sparse_categorical_accuracy: 0.9755
Model: "mnist_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_2 (Flatten)          multiple                  0         
_________________________________________________________________
dense_4 (Dense)              multiple                  100480    
_________________________________________________________________
dense_5 (Dense)              multiple                  1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

四、FASHION資料集識別衣褲鞋包

使用Sequential訓練模型

import tensorflow as tf

fashion = tf.keras.datasets.fashion_mnist
(x_train, y_train),(x_test, y_test) = fashion.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()

...
Epoch 4/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0559 - sparse_categorical_accuracy: 0.9827 - val_loss: 0.0786 - val_sparse_categorical_accuracy: 0.9763
Epoch 5/5
1875/1875 [==============================] - 2s 1ms/step - loss: 0.0443 - sparse_categorical_accuracy: 0.9869 - val_loss: 0.0774 - val_sparse_categorical_accuracy: 0.9755
Model: "mnist_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_2 (Flatten)          multiple                  0         
_________________________________________________________________
dense_4 (Dense)              multiple                  100480    
_________________________________________________________________
dense_5 (Dense)              multiple                  1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

使用自定義Class MnistModel(Model)

import tensorflow as tf
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras import Model

fashion = tf.keras.datasets.fashion_mnist
(x_train, y_train),(x_test, y_test) = fashion.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0


class MnistModel(Model):
    def __init__(self):
        super(MnistModel, self).__init__()
        self.flatten = Flatten()
        self.d1 = Dense(128, activation='relu')
        self.d2 = Dense(10, activation='softmax')

    def call(self, x):
        x = self.flatten(x)
        x = self.d1(x)
        y = self.d2(x)
        return y


model = MnistModel()

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=5, validation_data=(x_test, y_test), validation_freq=1)
model.summary()

...
Epoch 4/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.3151 - sparse_categorical_accuracy: 0.8848 - val_loss: 0.3678 - val_sparse_categorical_accuracy: 0.8659
Epoch 5/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2976 - sparse_categorical_accuracy: 0.8902 - val_loss: 0.3643 - val_sparse_categorical_accuracy: 0.8717
Model: "mnist_model_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_4 (Flatten)          multiple                  0         
_________________________________________________________________
dense_8 (Dense)              multiple                  100480    
_________________________________________________________________
dense_9 (Dense)              multiple                  1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________

五、參考

【1】keras官方文件：https://keras.io/zh/
【2】tf.keras官方文件：https://tensorflow.google.cn/api_docs/python/tf/keras
【3】keras vs. tf.keras：https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/

深度學習神經網路
« 上一篇
下一篇 »

相關推薦