一级目录

二级目录

三级目录

ngboost分类实战

常用参数介绍

• n_estimators: 学习器数量
• learning_rate: 学习率
• minibatch_frac: 每次迭代的样本比例
• col_sample: 每次迭代的特征比例
• Dist’: Bernoulli, 预测目标的分布
• verbose: 日志输出
• Score: 相当于损失函数
• Base: 基学习器
• random_state: 随机种子

python">from ngboost.distns import Bernoulli
from ngboost.scores import LogScore

ngb_params = {
    'n_estimators': 1000,
    'learning_rate': 0.01,
    'minibatch_frac': 0.5,
    'col_sample': 0.5,
    'Dist': Bernoulli,  # k_categorical(2),
    'verbose': 100,
    'Score': LogScore,
    'Base': base,
    'random_state': RANDOM_SEED,
}

基学习器定义

ngboost中的基学习器只能使用回归模型

python">from sklearn.tree import DecisionTreeRegressor

base = DecisionTreeRegressor(
    criterion="friedman_mse",
    min_samples_split=5,
    min_samples_leaf=5,
    min_weight_fraction_leaf=0.0,
    splitter="best",
    random_state=RANDOM_SEED,
)

定义训练函数

训练参数

• X: 训练集特征
• Y: 训练集标签
• X_val=None: 验证集特征
• Y_val=None: 验证集标签
• sample_weight=None: 类别不平衡时每个样本的采样比例,长度与训练集大小一致
• val_sample_weight=None: 类别不平衡时每个样本的采样比例,长度与训练集大小一致
• train_loss_monitor=None: 类似于评价指标
• val_loss_monitor=None: 类似于评价指标
• early_stopping_rounds=None: 早停次数

python">from ngboost import NGBClassifier
from sklearn.utils.class_weight import compute_class_weight


weight = compute_class_weight('balanced', classes=[0, 1], y=y_train)
weights = dict(zip([0, 1], weight))


def train_single_ngb(x_train, y_train, x_valid, y_valid, fea_cols, params):
    """_summary_
    单折NGB模型训练
    Args:
        x_train (datafram): 训练集特征
        y_train (datafram): 训练集标签
        x_valid (datafram): 验证集特征
        y_valid (datafram): 验证集标签
        fea_cols (list): 使用的特征
        params (dict): NGB模型参数

    Returns:
        model: 训练完成的NBG模型
        dataframe: 特征重要性df
    """
    print("{0:=^70}".format(f'TRAIN LGB MODEL WITH {len(fea_cols)} FEARURES'))
    ngb = NGBClassifier(**params)
    ngb.fit(x_train[fea_cols].fillna(0),
            y_train.astype(int),
            sample_weight = y_train.map(weights).values,
            X_val=x_valid[fea_cols].fillna(0),
            Y_val=y_valid.astype(int),
            early_stopping_rounds=50,
           )    
    
    # 特征重要性
    imp_df = pd.DataFrame(zip(fea_cols, ngb.feature_importances_[0]))
    
    return ngb, imp_df
    
ngb_clf, imp = train_single_ngb(x_train, y_train, x_valid, y_valid, USE_COLS, ngb_params)