基础
- FATAE服务版本: V1.6.1
- FATE-Serving服务版本: V2.0.4
- 部署:基于docker-compose部署,4台服务器集群10000–9997
- 用例保含: 模型训练, 推理服务
步骤
- guest方训练数据集上传: breast_homo_guest.csv;
2．host方训练数据集上传: breast_homo_host.csv;
3．模型训练流程/计划dsl文件编写: homo_lr_train_eval_dsl.json文件编写;
4．模型训练运行配置文件编写: homo_lr_train_eval_conf.json文件编写;
5．提交模型训练任务;
6．模型训练可视化;
7．预测流程/计划dsl文件编写: homo-lr-normal-predict-dsl.json
8．预测流程运行配置文件编写: homo-lr-normal-predict-conf.json
9．模型部署;
10．guest方预测数据集上传: breast_homo_guest.csv;
11．host方预测数据集上传: breast_homo_host.csv;
12．提交预测任务;
13．预测任务可视化;
14．结果模型,数据导出;
3.guest方训练数据集上传
breast_homo_guest.csv数据集文件在FATA1.6.1源码示例下:example/homo-feature-binning
/v1/data/upload
~~~~~~~~~~~~~~~
- request parames:
- namespace: Required,String: upload data table namespace
- table_name: Required,String: upload data table name
- work_mode: Required,Integer: eggroll’s working mode
- head: Required,Integer: determine if there is a data header
- partition: Required,Integer: set the number of partitions to save
data
- module: Optional,String: If you need to use the data of the
machine where the FATE-Flow server is located, this value is not
empty.
- use_local_data: Optional,String: If you need to use the data of the machine where the FATE-Flow server is located, this value is 0.
- drop: Optional, Integer: When the cluster deployment uses the same table to upload data, it is necessary to carry the drop parameter,0 represents overwriting upload, 1 represents deleting the previous data and re-uploading
- request Body:
- file: Required, String: upload file location
- response structure
- job_id: upload job id,String
- data: return data for submitting job ,Object
host方训练数据集上传:
breast_homo_host.csv在nextcloud共享盘下载,该文档同目录下;
上传接口同上;
模型训练流程/计划dsl文件编写:
homo_lr_train_eval_dsl.json文件在在FATA1.6.1源码示例下:example/homo-feature-binning; 内容:
{
"components": {
"reader_0": {
"module": "Reader",
"output": {
"data": [
"data"
]
}
},
"reader_1": {
"module": "Reader",
"output": {
"data": [
"data"
]
}
},
"dataio_0": {
"module": "DataIO",
"input": {
"data": {
"data": [
"reader_0.data"
]
}
},
"output": {
"data": [
"data"
],
"model": [
"model"
]
}
},
"dataio_1": {
"module": "DataIO",
"input": {
"data": {
"data": [
"reader_1.data"
]
},
"model": [
"dataio_0.model"
]
},
"output": {
"data": [
"data"
],
"model": [
"model"
]
}
},
"scale_0": {
"module": "FeatureScale",
"input": {
"data": {
"data": [
"dataio_0.data"
]
}
},
"output": {
"data": [
"data"
],
"model": [
"model"
]
}
},
"scale_1": {
"module": "FeatureScale",
"input": {
"data": {
"data": [
"dataio_1.data"
]
},
"model": [
"scale_0.model"
]
},
"output": {
"data": [
"data"
],
"model": [
"model"
]
}
},
"homo_lr_0": {
"module": "HomoLR",
"input": {
"data": {
"train_data": [
"scale_0.data"
],
"validate_data": [
"scale_1.data"
]
}
},
"output": {
"data": [
"data"
],
"model": [
"model"
]
}
},
"evaluation_0": {
"module": "Evaluation",
"input": {
"data": {
"data": [
"homo_lr_0.data"
]
}
},
"output": {
"data": [
"data"
]
}
}
}
}
模型训练运行配置文件编写
homo_lr_train_eval_conf.json文件位置同上,内容:
{
"dsl_version": 2,
"initiator": {
"role": "guest",
"party_id": 10000
},
"role": {
"guest": [
10000
],
"host": [
9999
],
"arbiter": [
9999
]
},
"job_parameters": {
"common": {
"job_type": "train",
"backend": 0,
"work_mode": 0
}
},
"component_parameters": {
"common": {
"dataio_0": {
"with_label": true,
"output_format": "dense"
},
"homo_lr_0": {
"penalty": "L2",
"tol": 1e-05,
"alpha": 0.01,
"optimizer": "rmsprop",
"batch_size": 320,
"learning_rate": 0.15,
"init_param": {
"init_method": "zeros"
},
"encrypt_param": {
"method": null
},
"max_iter": 3,
"early_stop": "diff",
"cv_param": {
"n_splits": 4,
"shuffle": true,
"random_seed": 33,
"need_cv": false
},
"validation_freqs": 1
},
"evaluation_0": {
"eval_type": "binary"
}
},
"role": {
"host": {
"0": {
"evaluation_0": {
"need_run": false
},
"reader_1": {
"table": {
"name": "breast_homo_host",
"namespace": "experiment"
}
},
"reader_0": {
"table": {
"name": "breast_homo_host",
"namespace": "experiment"
}
}
}
},
"guest": {
"0": {
"reader_1": {
"table": {
"name": "breast_homo_guest",
"namespace": "experiment"
}
},
"reader_0": {
"table": {
"name": "breast_homo_guest",
"namespace": "experiment"
}
}
}
}
}
}
}
提交模型训练任务
接口及说明:
/v1/job/submit
~~~~~~~~~~~~~~
- request structure
- job_runtime_conf: Required,Object:该对象为homo_lr_train_eval_conf.json文件中的内容
- job_dsl: Required,Object: 该对象为homo_lr_train_eval_dsl.json文件中的内容
- response structure
- job_id: job id of the currently submitted job,String
- data: return data for submitting job ,Object
模型训练可视化
访问FATE-Board地址: IP:8080
预测流程/计划dsl文件编写
homo-lr-normal-predict-dsl.json文件位置同上,内容:
{
"components": {
"dataio_0": {
"input": {
"data": {
"data": [
"reader_0.data"
]
},
"model": [
"pipeline.dataio_0.model"
]
},
"module": "DataIO",
"output": {
"data": [
"data"
]
}
},
"reader_0": {
"module": "Reader",
"output": {
"data": [
"data"
]
}
},
"scale_0": {
"input": {
"data": {
"data": [
"dataio_0.data"
]
},
"model": [
"pipeline.scale_0.model"
]
},
"module": "FeatureScale",
"output": {
"data": [
"data"
]
}
},
"homo_lr_0": {
"input": {
"data": {
"test_data": [
"scale_0.data"
]
},
"model": [
"pipeline.homo_lr_0.model"
]
},
"module": "HomoLR",
"output": {
"data": [
"data"
]
}
}
}
}
预测流程运行配置文件编写:
homo-lr-normal-predict-conf.json文件位置同上,内容:
{
"dsl_version": 2,
"initiator": {
"role": "guest",
"party_id": 10000
},
"role": {
"guest": [
10000
],
"host": [
9999
],
"arbiter": [
9999
]
},
"job_parameters": {
"common": {
"work_mode": 0,
"backend": 0,
"job_type": "predict",
"model_id": "arbiter-9999#guest-10000#host-9999#model",
"model_version": "2020101008251441870347"
}
},
"component_parameters": {
"role": {
"guest": {
"0": {
"reader_0": {
"table": {
"name": "breast_homo_guest",
"namespace": "experiment"
}
}
}
},
"host": {
"0": {
"reader_0": {
"table": {
"name": "breast_homo_host",
"namespace": "experiment"
}
}
}
}
}
}
}
模型部署
接口:
/v1/model/deploy
~~~~~~~~~~~~~~~~~~
- request structure
- model_version: Required, Integer: model version:FATE训练模型成功后返回的model version
- model_id: Required, String: model id:FATE训练模型成功后返回的model id
- cpn_list: Optional, String: array-like string that contains components
- cpn_path: Optional, String: file path of plain text which stores component list
- dsl_path: Optional, String: file path of plain text which stores dsl content
- response structure
- retcode: return code, Integer
- retmsg: return code description, String
- data: status info, Object
guest方预测数据集上传
因测试流程,直接使用训练数据集;
host方预测数据集上传
因测试流程,直接使用训练数据集;
提交预测任务
使用原来的提交训练任务的接口,将参数进行修改:
接口:
/v1/job/submit
~~~~~~~~~~~~~~
- request structure
- job_runtime_conf: Required,Object:该对象为homo-lr-normal-predict-conf.json文件中的内容
- job_dsl: Required,Object: 该对象为homo-lr-normal-predict-dsl.json文件中的内容
预测任务可视化
访问FATE-Board可视化页面,进行查看:
结果模型,数据导出
