[人工智能]玩转Kaggle：Classify Leaves（叶子分类）——模型设计与训练

??趁假期的最后一天填上kaggle叶子分类的坑，本来想把这个比赛的成绩刷到95%以上再写这篇Blog记录的，但是后面学习工作比较多，刷比赛可能并不是很划算。这里基于Charlesyyun提供的baseline代码进行了模型设计和训练。主体思路其实没什么多说的，只是通过比赛可以将之前学到的东西融合整理起来，这能够较大地提高动手的能力。

7th: ResNeSt+ResNeXt+DenseNet (0.98840)

数据分析请参考前一篇Blog: 玩转Kaggle：Classify Leaves（叶子分类）——数据分析篇

一、数据加载

import torch.utils.data as Data 
from torchvision import transforms
import torchvision
from PIL import Image 
import os 
import pandas as pd 
import numpy as np 
import matplotlib.pyplot as plt
from sklearn.preprocessing import LabelEncoder,OneHotEncoder
import torch 
from torch import nn 
import d2l.torch as d2l 
from sklearn.model_selection import train_test_split,KFold
from torch.optim.lr_scheduler import CosineAnnealingLR
from tqdm import tqdm
import ttach as tta 

1. 首先构建自己的数据加载器，基于pytorch的dataset类进行复写是个不错的方法
2. 由于训练数据的标签是用得字符表示，在训练的时候我们需要将其映射到对应的数值上，使用labelencoder = LabelEncoder()可以得到这样一个映射表
3. 由于我们的训练数据集太小，需要适当地对数据进行数据增强。由于我是在我个人电脑上进行跑模型，GPU算力有限，所以为了保证内存不溢出，我将图片裁减到了128*128大小，这极大地影响了我最后结果地精度，如果有条件，请裁减到224*224比较合适

class LeavesSet(Data.Dataset):
    """
        construct the dataset
    """
    def __init__(self,images_path,images_label,transform=None,train=True):
        self.imgs = [os.path.join('/data/liufeng/project/MachineLearning/LiMu/data/classify-leaves/',"".join(image_path)) for image_path in images_path] 
        
        # if train dataset : get the appropriate label
        if train:
            self.train = True
            self.labels = images_label
        else:
            self.train = False
        
        # transform
        self.transform = transform
        
    def __getitem__(self,index):
        image_path = self.imgs[index]
        pil_img = Image.open(image_path)
        if self.transform:
            transform = self.transform
        else:
            # if not define the transform:default resize the figure(224,224) and ToTensor
            transform = transforms.Compose([
                transforms.Resize((224,224)),
                transforms.ToTensor()
                ])
        data = transform(pil_img)
        if self.train:
            image_label = self.labels[index]
            return data,image_label
        else:
            return data 
    
    def __len__(self):
        return len(self.imgs)
    
    
# load initial data to dataloader，and encode the label
def load_data_leaves(train_transform=None,test_transform=None):
    train_data = pd.read_csv('./data/classify-leaves/train.csv')
    test_data = pd.read_csv('./data/classify-leaves/test.csv')

    # encode the train label
    labelencoder = LabelEncoder()
    labelencoder.fit(train_data['label'])
    train_data['label'] = labelencoder.transform(train_data['label'])
    label_map = dict(zip(labelencoder.classes_,labelencoder.transform(labelencoder.classes_)))
    label_inv_map = {v:k for k,v in label_map.items()}
    
    # get the train data and transorm it as a batch 
    train_dataSet = LeavesSet(train_data['image'],train_data['label'],transform=train_transform,train=True)
    test_dataSet = LeavesSet(test_data['image'],images_label=0,transform=test_transform,train=False)
    
    return (
            train_dataSet,
            test_dataSet ,
            label_map,
            label_inv_map,
           )

定义一下训练数据和测试数据需要进行怎样的数据转换/增强。需要注意一点：一般情况下validation data 应该和test data 使用相同的transform，但是我这里发现的时候已经写好了后面的内容，偷懒不想更改了也就将错就错了。如果有人复写，可以自己考虑一下，其实也不难。

# define the transform
train_transform = transforms.Compose([
    # 随机裁剪图像，所得图像为原始面积的0.08到1之间，高宽比在3/4和4/3之间。
    # 然后，缩放图像以创建224 x 224的新图像
    transforms.RandomResizedCrop(128, scale=(0.08, 1.0), ratio=(3.0 / 4.0, 4.0 / 3.0)),
    transforms.RandomHorizontalFlip(),
    # 随机更改亮度，对比度和饱和度
    transforms.ColorJitter(brightness=0.4, contrast=0.4, saturation=0.4),
    # 添加随机噪声
    transforms.ToTensor(),
    # 标准化图像的每个通道
    transforms.Normalize([0.485, 0.456, 0.406],[0.229, 0.224, 0.225])])

test_transform = transforms.Compose([
    transforms.Resize(256),
    # 从图像中心裁切224x224大小的图片
    transforms.CenterCrop(128),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406],[0.229, 0.224, 0.225])])

train_dataset,test_dataset,label_map,label_inv_map = load_data_leaves(train_transform,test_transform)

二、模型构建与训练

1. resnet

1.1 ResNet-50模型微调+冻结

微调+冻结部分层：

# 是否要冻住模型的前面一些层
def set_parameter_requires_grad(model, feature_extracting):
    if feature_extracting:
        model = model
        for param in model.parameters():
            param.requires_grad = False

# ResNeSt模型
def resnet_model(num_classes, feature_extract = False):
    model_ft = torchvision.models.resnet50(pretrained=True)
    set_parameter_requires_grad(model_ft, feature_extract)
    num_ftrs = model_ft.fc.in_features
    model_ft.fc = nn.Sequential(nn.Linear(num_ftrs, num_classes))

    return model_ft

定义超参数

# Configuration options
k_folds = 5
num_epochs = 20
learning_rate = 1e-4
weight_decay = 1e-3
loss_function = nn.CrossEntropyLoss()
# For fold results
results = {}

# Set fixed random number seed
torch.manual_seed(1)

# devices
device = d2l.try_gpu()

# Define the K-fold Cross Validator
kfold = KFold(n_splits=k_folds, shuffle=True)

1.2 五折交叉验证

# Start print
print('--------------------------------------')

# K-fold Cross Validation model evaluation
for fold, (train_ids,valid_ids) in enumerate(kfold.split(train_dataset)):
    # Print
    print(f'FOLD {fold}')
    print('--------------------------------------')
    # Sample elements randomly from a given list of ids, no replacement.
    train_subsampler = torch.utils.data.SubsetRandomSampler(train_ids)
    valid_subsampler = torch.utils.data.SubsetRandomSampler(valid_ids)
    # Define data loaders for training and testing data in this fold
    trainloader = torch.utils.data.DataLoader(train_dataset,
                        batch_size=64, sampler=train_subsampler, num_workers=4)
    validloader = torch.utils.data.DataLoader(train_dataset,
                        batch_size=64, sampler=valid_subsampler, num_workers=4)
  
    # Initialize a model and put it on the device specified.
    model = resnet_model(176)
    model = model.to(device)
    model.device = device
  
    # Initialize optimizer
    optimizer = torch.optim.Adam(model.parameters(),lr=learning_rate,weight_decay= weight_decay)
    #   optimizer = SWA(our_optimizer, swa_start=5, swa_freq =5, swa_lr=0.05)
    scheduler = CosineAnnealingLR(optimizer,T_max=10)
    
    # Run the training loop for defined number of epochs
    for epoch in range(0,num_epochs):
        model.train()
        # Print epoch
        print(f'Starting epoch {epoch+1}')
        # These are used to record information in training
        train_losses = []
        train_accs = []
        # Iterate the training set by batches
        for batch in tqdm(trainloader):
            # Move images and labels to GPU
            imgs, labels = batch
            imgs = imgs.to(device)
            labels = labels.to(device)
            # Forward the data
            logits = model(imgs)
            # Calculate loss
            loss = loss_function(logits,labels)
            # Clear gradients in previous step
            optimizer.zero_grad()
            # Compute gradients for parameters
            loss.backward()
            # Update the parameters with computed gradients
            optimizer.step()
            # Compute the accuracy for current batch.
            acc = (logits.argmax(dim=-1) == labels).float().mean()
            # Record the loss and accuracy.
            train_losses.append(loss.item())
            train_accs.append(acc)
#         print("第%d个epoch的学习率：%f" % (epoch+1,optimizer.param_groups[0]['lr']))
        scheduler.step()
    # The average loss and accuracy of the training set is the average of the recorded values.
    train_loss = np.sum(train_losses) / len(train_losses)
    train_acc = np.sum(train_accs) / len(train_accs)
    # Print the information.
    print(f"[ Train | {epoch + 1:03d}/{num_epochs:03d} ] loss = {train_loss:.5f}, acc = {train_acc:.5f}")
#     print(f"[ Train | {epoch + 1:03d}/{num_epochs:03d} ] loss = {train_loss:.5f}")

    # Train process (all epochs) is complete
    print('Training process has finished. Saving trained model.')
    print('Starting validation')

    # Saving the model
    print('saving model with loss {:.3f}'.format(train_loss))
    save_path = f'./model/leaf/resnet-fold-{fold}.pth'
    torch.save(model.state_dict(),save_path)
  
    # Start Validation
    model.eval()
    valid_losses = []
    valid_accs = []
    with torch.no_grad():
        for batch in tqdm(validloader):
            imgs, labels = batch
            # No gradient in validation
            logits = model(imgs.to(device))
            loss = loss_function(logits,labels.to(device))
            acc = (logits.argmax(dim=-1) == labels.to(device)).float().mean()
            # Record loss and accuracy
            valid_losses.append(loss.item())        
            valid_accs.append(acc)
        # The average loss and accuracy
        valid_loss = np.sum(valid_losses)/len(valid_losses)
        valid_acc = np.sum(valid_accs)/len(valid_accs)
        print(f"[ Valid | {epoch + 1:03d}/{num_epochs:03d} ] loss = {valid_loss:.5f}, acc = {valid_acc:.5f}")
        print('Accuracy for fold %d: %d' % (fold, valid_acc))
        print('--------------------------------------')
        results[fold] = valid_acc
# Print fold results
print(f'K-FOLD CROSS VALIDATION RESULTS FOR {k_folds} FOLDS')
print('--------------------------------')
total_summation = 0.0
for key, value in results.items():
    print(f'Fold {key}: {value} ')
    total_summation += value
print(f'Average: {total_summation/len(results.items())} ')

--------------------------------------
FOLD 0
--------------------------------------
...
--------------------------------------
K-FOLD CROSS VALIDATION RESULTS FOR 5 FOLDS
--------------------------------
Fold 0: 0.7646294236183167 
Fold 1: 0.772324800491333 
Fold 2: 0.7688811421394348 
Fold 3: 0.7619268894195557 
Fold 4: 0.7501959204673767 
Average: 0.7635916471481323 

1.3 模型预测

使用训练得到的五个模型进行预测结果，并将结果分别保存好

testloader = torch.utils.data.DataLoader(test_dataset,batch_size=64, num_workers=4)

## predict
model = resnet_model(176)

# create model and load weights from checkpoint
model = model.to(device)
# load the all folds
for test_fold in range(k_folds):
    model_path = f'./model/leaf/model-fold-{test_fold}.pth'
    saveFileName = f'./kaggle_submission/leaf/submission-fold-{test_fold}.csv'
    model.load_state_dict(torch.load(model_path))
    
    # Make sure the model is in eval mode.
    # Some modules like Dropout or BatchNorm affect if the model is in training mode.
    model.eval()
    tta_model = tta.ClassificationTTAWrapper(model, tta.aliases.five_crop_transform(200,200)) # Test-Time Augmentation

    # Initialize a list to store the predictions.
    predictions = []
    # Iterate the testing set by batches.
    for batch in tqdm(testloader):
        imgs = batch
        with torch.no_grad():
            logits = tta_model(imgs.to(device))
            # Take the class with greatest logit as prediction and record it.
            predictions.extend(logits.argmax(dim=-1).cpu().numpy().tolist())
    preds = []
    for i in predictions:
        preds.append(label_inv_map[i])
        
    test_data = pd.read_csv('./data/classify-leaves/test.csv')
    test_data['label'] = pd.Series(preds)
    submission = pd.concat([test_data['image'], test_data['label']], axis=1)
    submission.to_csv(saveFileName, index=False)
    print("ResNeSt Model Results Done!!!!!!!!!!!!!!!!!!!!!!!!!!!")

100%|██████████| 138/138 [01:08<00:00,  2.02it/s]

ResNeSt Model Results Done!!!!!!!!!!!!!!!!!!!!!!!!!!!

1.4 利用K-Fold结果投票获取最终提交数据

加载预测结果

# load the result of 5-fold cross validation
df0 = pd.read_csv('./kaggle_submission/leaf/submission-fold-0.csv')
df1 = pd.read_csv('./kaggle_submission/leaf/submission-fold-1.csv')
df2 = pd.read_csv('./kaggle_submission/leaf/submission-fold-2.csv')
df3 = pd.read_csv('./kaggle_submission/leaf/submission-fold-3.csv')
df4 = pd.read_csv('./kaggle_submission/leaf/submission-fold-4.csv')

#convert the  result to a number
list_num_label0,list_num_label1,list_num_label2,list_num_label3,list_num_label4 = [],[],[],[],[]
for i in range(len(df0)):
    list_num_label0.append(label_map[df0['label'][i]])
    list_num_label1.append(label_map[df1['label'][i]]) 
    list_num_label2.append(label_map[df2['label'][i]])
    list_num_label3.append(label_map[df3['label'][i]])
    list_num_label4.append(label_map[df4['label'][i]])

# concate all the data
df_all = df0.copy()
df_all.drop(['label'],axis=1,inplace=True)
df_all['num_label0'] = list_num_label0
df_all['num_label1'] = list_num_label1
df_all['num_label2'] = list_num_label2
df_all['num_label3'] = list_num_label3
df_all['num_label4'] = list_num_label4
df_all.head()

# transpose the predict result (for calculate the mode of result easier)
df_all_transpose = df_all.copy().drop(['image'],axis=1).transpose()
df_all_transpose.head()

5 rows × 8800 columns

# vote the best result(mode)
df_mode = df_all_transpose.mode().transpose()
df_mode.head()

# conver the best result back to text
voting_class = []
for each in df_mode[0]:
    voting_class.append(label_inv_map[each])
df_all['label'] = voting_class
df_all.head()

# save the best result as csv
# choose columns image and lable as the result
df_submission = df_all[['image','label']].copy()
# save the the result file
df_submission.to_csv('./kaggle_submission/leaf/submission-resnet.csv', index=False)
print('Voting results of resnest successfully saved!')

Voting results of resnest successfully saved!

1.5 kaggle提交

2. resnext

2.1 resnext50_32x4d模型微调+冻结层

# 是否要冻住模型的前面一些层
def set_parameter_requires_grad(model, feature_extracting):
    if feature_extracting:
        model = model
        for param in model.parameters():
            param.requires_grad = False

# resnext50_32x4d模型
def resnext_model(num_classes, feature_extract = False, use_pretrained=True):
    model_ft = torchvision.models.resnext50_32x4d(pretrained=use_pretrained)
    set_parameter_requires_grad(model_ft, feature_extract)
    num_ftrs = model_ft.fc.in_features
    model_ft.fc = nn.Sequential(nn.Linear(num_ftrs, num_classes))

    return model_ft

# Configuration options
k_folds = 5
num_epochs = 30
learning_rate = 1e-3
weight_decay = 1e-3
loss_function = nn.CrossEntropyLoss()
# For fold results
results = {}

# Set fixed random number seed
torch.manual_seed(42)

# Define the K-fold Cross Validator
kfold = KFold(n_splits=k_folds, shuffle=True)

2.2 五折交叉验证

# Start print
print('--------------------------------------')

# K-fold Cross Validation model evaluation
for fold, (train_ids,valid_ids) in enumerate(kfold.split(train_dataset)):
    # Print
    print(f'FOLD {fold}')
    print('--------------------------------------')
    # Sample elements randomly from a given list of ids, no replacement.
    train_subsampler = torch.utils.data.SubsetRandomSampler(train_ids)
    valid_subsampler = torch.utils.data.SubsetRandomSampler(valid_ids)
    # Define data loaders for training and testing data in this fold
    trainloader = torch.utils.data.DataLoader(train_dataset,
                        batch_size=64, sampler=train_subsampler, num_workers=4)
    validloader = torch.utils.data.DataLoader(train_dataset,
                        batch_size=64, sampler=valid_subsampler, num_workers=4)
  
    # Initialize a model and put it on the device specified.
    model = resnext_model(176)
    model = model.to(device)
    model.device = device
  
    # Initialize optimizer
    optimizer = torch.optim.Adam(model.parameters(),lr=learning_rate,weight_decay= weight_decay)
    #   optimizer = SWA(our_optimizer, swa_start=5, swa_freq =5, swa_lr=0.05)
    scheduler = CosineAnnealingLR(optimizer,T_max=10)
    
    # Run the training loop for defined number of epochs
    for epoch in range(0,num_epochs):
        model.train()
        # Print epoch
        print(f'Starting epoch {epoch+1}')
        # These are used to record information in training
        train_losses = []
        train_accs = []
        # Iterate the training set by batches
        for batch in tqdm(trainloader):
            # Move images and labels to GPU
            imgs, labels = batch
            imgs = imgs.to(device)
            labels = labels.to(device)
            # Forward the data
            logits = model(imgs)
            # Calculate loss
            loss = loss_function(logits,labels)
            # Clear gradients in previous step
            optimizer.zero_grad()
            # Compute gradients for parameters
            loss.backward()
            # Update the parameters with computed gradients
            optimizer.step()
            # Compute the accuracy for current batch.
            acc = (logits.argmax(dim=-1) == labels).float().mean()
            # Record the loss and accuracy.
            train_losses.append(loss.item())
            train_accs.append(acc)
#         print("第%d个epoch的学习率：%f" % (epoch+1,optimizer.param_groups[0]['lr']))
        scheduler.step()
    # The average loss and accuracy of the training set is the average of the recorded values.
    train_loss = np.sum(train_losses) / len(train_losses)
    train_acc = np.sum(train_accs) / len(train_accs)
    # Print the information.
    print(f"[ Train | {epoch + 1:03d}/{num_epochs:03d} ] loss = {train_loss:.5f}, acc = {train_acc:.5f}")
#     print(f"[ Train | {epoch + 1:03d}/{num_epochs:03d} ] loss = {train_loss:.5f}")

    # Train process (all epochs) is complete
    print('Training process has finished. Saving trained model.')
    print('Starting validation')

    # Saving the model
    print('saving model with loss {:.3f}'.format(train_loss))
    save_path = f'./model/leaf/resneXt-fold-{fold}.pth'
    torch.save(model.state_dict(),save_path)
  
    # Start Validation
    model.eval()
    valid_losses = []
    valid_accs = []
    with torch.no_grad():
        for batch in tqdm(validloader):
            imgs, labels = batch
            # No gradient in validation
            logits = model(imgs.to(device))
            loss = loss_function(logits,labels.to(device))
            acc = (logits.argmax(dim=-1) == labels.to(device)).float().mean()
            # Record loss and accuracy
            valid_losses.append(loss.item())        
            valid_accs.append(acc)
        # The average loss and accuracy
        valid_loss = np.sum(valid_losses)/len(valid_losses)
        valid_acc = np.sum(valid_accs)/len(valid_accs)
        print(f"[ Valid | {epoch + 1:03d}/{num_epochs:03d} ] loss = {valid_loss:.5f}, acc = {valid_acc:.5f}")
        print('Accuracy for fold %d: %d' % (fold, valid_acc))
        print('--------------------------------------')
        results[fold] = valid_acc
# Print fold results
print(f'K-FOLD CROSS VALIDATION RESULTS FOR {k_folds} FOLDS')
print('--------------------------------')
total_summation = 0.0
for key, value in results.items():
    print(f'Fold {key}: {value} ')
    total_summation += value
print(f'Average: {total_summation/len(results.items())} ')

--------------------------------------
FOLD 0
--------------------------------------
...
--------------------------------------
K-FOLD CROSS VALIDATION RESULTS FOR 5 FOLDS
--------------------------------
Fold 0: 0.7881605625152588 
Fold 1: 0.7917798757553101 
Fold 2: 0.807229220867157 
Fold 3: 0.7923932075500488 
Fold 4: 0.7697393894195557 
Average: 0.7898604273796082 

2.3 Resnext模型预测

testloader = torch.utils.data.DataLoader(test_dataset,batch_size=64, num_workers=4)

## predict
model = resnext_model(176)

# create model and load weights from checkpoint
model = model.to(device)
# load the all folds
for test_fold in range(k_folds):
    model_path = f'./model/leaf/resneXt-fold-{test_fold}.pth'
    saveFileName = f'./kaggle_submission/leaf/resneXt-submission-fold-{test_fold}.csv'
    model.load_state_dict(torch.load(model_path))
    
    # Make sure the model is in eval mode.
    # Some modules like Dropout or BatchNorm affect if the model is in training mode.
    model.eval()
    tta_model = tta.ClassificationTTAWrapper(model, tta.aliases.five_crop_transform(200,200)) # Test-Time Augmentation

    # Initialize a list to store the predictions.
    predictions = []
    # Iterate the testing set by batches.
    for batch in tqdm(testloader):
        imgs = batch
        with torch.no_grad():
            logits = tta_model(imgs.to(device))
            # Take the class with greatest logit as prediction and record it.
            predictions.extend(logits.argmax(dim=-1).cpu().numpy().tolist())
    preds = []
    for i in predictions:
        preds.append(label_inv_map[i])
        
    test_data = pd.read_csv('./data/classify-leaves/test.csv')
    test_data['label'] = pd.Series(preds)
    submission = pd.concat([test_data['image'], test_data['label']], axis=1)
    submission.to_csv(saveFileName, index=False)
    print("ResNeSt Model Results Done!!!!!!!!!!!!!!!!!!!!!!!!!!!")

100%|██████████| 138/138 [01:45<00:00,  1.31it/s]

ResNeSt Model Results Done!!!!!!!!!!!!!!!!!!!!!!!!!!!

2.4 K-Fold模型投票获取最优解

# load the result of 5-fold cross validation
df0 = pd.read_csv('./kaggle_submission/leaf/resneXt-submission-fold-0.csv')
df1 = pd.read_csv('./kaggle_submission/leaf/resneXt-submission-fold-1.csv')
df2 = pd.read_csv('./kaggle_submission/leaf/resneXt-submission-fold-2.csv')
df3 = pd.read_csv('./kaggle_submission/leaf/resneXt-submission-fold-3.csv')
df4 = pd.read_csv('./kaggle_submission/leaf/resneXt-submission-fold-4.csv')

#convert the  result to a number
list_num_label0,list_num_label1,list_num_label2,list_num_label3,list_num_label4 = [],[],[],[],[]
for i in range(len(df0)):
    list_num_label0.append(label_map[df0['label'][i]])
    list_num_label1.append(label_map[df1['label'][i]]) 
    list_num_label2.append(label_map[df2['label'][i]])
    list_num_label3.append(label_map[df3['label'][i]])
    list_num_label4.append(label_map[df4['label'][i]])

# concate all the data
df_all = df0.copy()
df_all.drop(['label'],axis=1,inplace=True)
df_all['num_label0'] = list_num_label0
df_all['num_label1'] = list_num_label1
df_all['num_label2'] = list_num_label2
df_all['num_label3'] = list_num_label3
df_all['num_label4'] = list_num_label4
df_all.head()

# transpose the predict result (for calculate the mode of result easier)
df_all_transpose = df_all.copy().drop(['image'],axis=1).transpose()
df_all_transpose.head()

5 rows × 8800 columns

# vote the best result(mode)
df_mode = df_all_transpose.mode().transpose()
df_mode.head()

# conver the best result back to text
voting_class = []
for each in df_mode[0]:
    voting_class.append(label_inv_map[each])
df_all['label'] = voting_class
df_all.head()

# save the best result as csv
# choose columns image and lable as the result
df_submission = df_all[['image','label']].copy()
# save the the result file
df_submission.to_csv('./kaggle_submission/leaf/submission-resneXt.csv', index=False)
print('Voting results of resnest successfully saved!')

Voting results of resnest successfully saved!

一言难尽这个模型的精度还没有resnet高，就不贴出来了

三、总结

1. 数据增强对于数据量小的数据集训练特别重要
2. 由于GPU受限，对原始数据做了小裁减很大程度上影响了我最终的精度。
3. 对于数据的增强采用cutmix也是很好的方法
4. resnet和resnext模型对于图片分类是应用非常广的模型。
5. 微调模型是我们常用的方法，将类似的数据集中训练好的模型直接迁移过来可以极大地节省训练的时间和消耗，同时初始值更加符合数据的分布，更容易获得全局最优解。
6. 使用K-Fold交叉验证，然后进行模型结果投票，得到的结果更加准确。当然也可以使用多个不同类型的模型的结果进行投票，这样得到的结果往往误差更小。

关注博主即可阅读全文