模型構建之整合模型

構建RF GBDT XDBoost LightGBM這四個模型，並對每一個模型使用準確率和AUC評分。在上次任務中使用了LR SVM DecisionTree這三個簡單的模型對樣本進行了預測和評價，請參照https://blog.csdn.net/wxq_1993/article/details/85703936。

#1.匯入要使用的模組
import numpy as np
import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.ensemble import
 
 RandomForestClassifier
from sklearn.ensemble import GradientBoostingClassifier
from lightgbm import LGBMClassifier
from xgboost import XGBClassifier
from sklearn.metrics import accuracy_score,roc_auc_score
import time
import warnings
warnings.filterwarnings('ignore')

# 2.劃分X和y並簡單分析資料
data_original=
 
pd.read_csv("data_all.csv")
data_original.head(5)
data_original.describe() 
#data_original.info()

8 rows × 85 columns

y=data_original['status'].copy()
X=data_original.drop(['status'],axis=1).copy()
print("the X shape is:", X.shape)
print("the X shape is:" ,y.shape)
print("the nums of label 1 in y are",len(y[y==1]))
print("the nums of label 0 in y are",len(y[y==0]))
df_ret=pd.DataFrame(columns=('Model','Accuracy','AUC','Time'))
row=0

the X shape is: (4754, 84)
the X shape is: (4754,)
the nums of label 1 in y are 1193
the nums of label 0 in y are 3561

一共有4754組資料，每組資料中有84個特徵；標籤值中為1的有1193個，為0的有3561個;正樣例與負樣例數量差別較大，在後續處理應當考慮。

#3.資料集的三七劃分
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.3,random_state=2018)
print('the proportition of label 1 in y_test: %.2f%%'%(len(y_test[y_test==1])/len(y_test)*100))

the proportition of label 1 in y_test: 25.16%

# 4.定義一個評價函式
def evaluate(y_pre,y):
    acc=accuracy_score(y,y_pre)
    auc=roc_auc_score(y,y_pre)
    return acc,auc

由於在第一次作業中頻繁呼叫accuracy_score()和f1_score(),在第二次作業中，將其定義成一個評價函式方便呼叫

問題來了，我從官方文件上直接複製RF GBDT XGBoost Lightgbm這四個分類器的預設引數，執行後竟然報錯，提示有中文字元或者空格，只好如下這麼簡單輸入了

# 5.構建模型進行預測
#分別採用 RF GBDT XGBoost Lightgbm,由於對模型不熟悉，故全部採用預設值

rf_model=RandomForestClassifier(n_estimators=100,max_depth=None,criterion='gini')
gbdt_model=GradientBoostingClassifier(n_estimators=100,max_depth=3,learning_rate=0.1)
xgb_model=XGBClassifier(n_estimators=100,learning_rate=0.1,max_depth=3)
lgbm_model=LGBMClassifier(n_estimators=100,learning_rate=0.1,max_depth=-1)

# 6.訓練模型
models=[('RF',rf_model),('gbdt',gbdt_model),('xgb',xgb_model),('lgbm',lgbm_model)]
for name,model in models:
    print(name,'start training.....')
    startTime=time.clock()
    model.fit(X_train,y_train)
    y_pred=model.predict(X_test)
    endTime=time.clock()
    print(name,'using time is %.4f'%(endTime-startTime))
    acc,auc=evaluate(y_pred,y_test)
    print(name,'accuracy_score:',round(acc,4),'auc_score: ',round(auc,4))
    df_ret.loc[row]=[name,acc,auc,(endTime-startTime)]
    row+=1
    print('\n')
print(df_ret)

RF start training.....
RF using time is 1.3224
RF accuracy_score: 0.7849 auc_score:  0.6076


gbdt start training.....
gbdt using time is 1.3351
gbdt accuracy_score: 0.78 auc_score:  0.6376


xgb start training.....
xgb using time is 0.7749
xgb accuracy_score: 0.7856 auc_score:  0.6432


lgbm start training.....
lgbm using time is 0.7061
lgbm accuracy_score: 0.7701 auc_score:  0.631


  Model  Accuracy       AUC      Time
0    RF  0.784863  0.607558  1.322362
1  gbdt  0.779958  0.637566  1.335071
2   xgb  0.785564  0.643161  0.774934
3  lgbm  0.770147  0.631012  0.706147

根據結果可知，整合學習的這四種模型明顯好於第一次使用的三種模型，**其中XGBoost表現最好，LGBM速度最快；**由於複製預設引數報錯，導致訓練過程中只是用了三個引數，在後續的訓練中繼續改進。另外在面試過程中XGBoost和GBDT模型是經常被提問的，應當重點掌握。

參考資料：

1.整合模型
2.XGBoost:
3.RandomForest:
4.GradientBoostingClassifier:
5.xgboost的安裝:
6.https://zhuanlan.zhihu.com/p/54042675