概述
etcd集群默认情况下使用明文互相通讯，在某些场景下不安全，etcd走的都是HTTP协议，当然可以使用TLS/SSL方式加密通讯。
etcd集群用到三个证书：
client证书：用于客户端程序，如etcdctl连接
server证书：服务器证书，验证客户端身份
peer证书： 集群成员直接通讯使用
证书的X509v3 extensions信息部分，X509v3 Extended Key Usage表示证书的用途。
server证书"server auth"，client证书"client auth"，peer证书"server auth"、"client auth"
server证书、peer证书可以每个节点用一个，也可以用一个，但注意在证书SAN信息中添加节点的IP或主机名。
etcdctl等客户端工具使用client证书，“--peer-*-file”配置使用peer证书。“--*-file”配置使用server证书，也可以使用peer证书，但不能使用client证书。
相比明文的通信，TLS集群多出如下参数：
--cert-file
--client-cert-auth
--key-file
--trusted-ca-file
--peer-cert-file
--peer-client-cert-auth
--peer-key-file
--peer-trusted-ca-file
实验准备
本次实验使用3个主机，server证书和peer证书每个节点各一个，client证书通用。网上有的方案，所有节点peer用一个证书 serve用一个证书，有的方案peer和server都用一个证书。大家根据实际情况自行决定用多少证书，毕竟证书太多不好管理。
| 主机名? ? ? ? | IP地址 | 节点名 |
| vm53 | 192.168.0.53 | node1 |
| vm54 | 192.168.0.54 | node2 |
| vm55 | 192.168.0.55 | node3 |
生成证书
生成证书有很多种方式，如openssl、easy-rsa ，这里使用cfssl方法生成证书。
安装cfssl不在这里介绍，参考其它文档。
我的cfssl程序安装在主机vm53上，所有需要的证书制作完成后，复制到其它节点。
证书csr.json信息中hosts字段部分是关键配置，必须包含运行节点的IP。
CA证书
ca-config.json内容如下：
{
"signing": {
"default": {
"expiry": "43800h"
},
"profiles": {
"server": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth"
]
},
"client": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
},
"peer": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
}注意：peer部分同时启用了“server auth”和“client auth”.
ca-csr.json内容如下：
{
"CN": "My own CA",
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"L": "CA",
"O": "My Company Name",
"ST": "San Francisco",
"OU": "Org Unit 1",
"OU": "Org Unit 2"
}
]
}生成CA的证书和秘钥
cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
vm53的证书
server-node1.json内容如下
{
"CN": "node1",
"hosts": [
"192.168.0.53",
"127.0.0.1",
"example.net",
"www.example.net"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}生成server证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server-node1.json | cfssljson -bare server-node1
查看server-node1.pem证书文件详情：
openssl x509 -in /home/cfssl/server-node1.pem ?-text
?可以看到证书用于server端的验证。
peer-node1.json内容如下
{
"CN": "node1",
"hosts": [
"192.168.0.53",
"example.net",
"www.example.net"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}生成peer证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer peer-node1.json | cfssljson -bare peer-node1
vm54的证书
server-node2.json
{
"CN": "node2",
"hosts": [
"192.168.0.54",
"127.0.0.1",
"example.net",
"www.example.net"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server-node2.json | cfssljson -bare server-node2
?peer-node2.json
{
"CN": "node2",
"hosts": [
"192.168.0.54",
"example.net",
"www.example.net"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer peer-node2.json | cfssljson -bare peer-node2
vm55的证书
server-node3.json
{
"CN": "node3",
"hosts": [
"192.168.0.55",
"127.0.0.1",
"example.net",
"www.example.net"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server-node3.json | cfssljson -bare server-node3
peer-node3.json
{
"CN": "node3",
"hosts": [
"192.168.0.55",
"example.net",
"www.example.net"
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer peer-node3.json | cfssljson -bare peer-node3
client证书
?client.json
{
"CN": "client",
"hosts": [
""
],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"C": "US",
"ST": "CA",
"L": "San Francisco"
}
]
}cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client.json | cfssljson -bare client
启动程序
vm53上启动node1
etcd --name node1 \
--listen-peer-urls=https://192.168.0.53:2380 \
--listen-client-urls=https://127.0.0.1:2379,https://192.168.0.53:2379 \
--initial-advertise-peer-urls=https://192.168.0.53:2380 \
--advertise-client-urls=https://192.168.0.53:2379 \
--initial-cluster="node1=https://192.168.0.53:2380,node2=https://192.168.0.54:2380,node3=https://192.168.0.55:2380" \
--initial-cluster-token="test" \
--initial-cluster-state=new \
--peer-client-cert-auth=true \
--peer-trusted-ca-file=/home/cfssl/ca.pem \
--peer-cert-file=/home/cfssl/peer-node1.pem \
--peer-key-file=/home/cfssl/peer-node1-key.pem \
--client-cert-auth=true \
--trusted-ca-file=/home/cfssl/ca.pem \
--cert-file=/home/cfssl/server-node1.pem \
--key-file=/home/cfssl/server-node1-key.pem vm54上启动node2
etcd --name node2 \
--listen-peer-urls=https://192.168.0.54:2380 \
--listen-client-urls=https://127.0.0.1:2379,https://192.168.0.54:2379 \
--initial-advertise-peer-urls=https://192.168.0.54:2380 \
--advertise-client-urls=https://192.168.0.54:2379 \
--initial-cluster="node1=https://192.168.0.53:2380,node2=https://192.168.0.54:2380,node3=https://192.168.0.55:2380" \
--initial-cluster-token="test" \
--initial-cluster-state=new \
--peer-client-cert-auth=true \
--peer-trusted-ca-file=/home/cfssl/ca.pem \
--peer-cert-file=/home/cfssl/peer-node2.pem \
--peer-key-file=/home/cfssl/peer-node2-key.pem \
--client-cert-auth=true \
--trusted-ca-file=/home/cfssl/ca.pem \
--cert-file=/home/cfssl/server-node2.pem \
--key-file=/home/cfssl/server-node2-key.pem vm55启动node3
etcd --name node3 \
--listen-peer-urls=https://192.168.0.55:2380 \
--listen-client-urls=https://127.0.0.1:2379,https://192.168.0.55:2379 \
--initial-advertise-peer-urls=https://192.168.0.55:2380 \
--advertise-client-urls=https://192.168.0.55:2379 \
--initial-cluster="node1=https://192.168.0.53:2380,node2=https://192.168.0.54:2380,node3=https://192.168.0.55:2380" \
--initial-cluster-token="test" \
--initial-cluster-state=new \
--peer-client-cert-auth=true \
--peer-trusted-ca-file=/home/cfssl/ca.pem \
--peer-cert-file=/home/cfssl/peer-node3.pem \
--peer-key-file=/home/cfssl/peer-node3-key.pem \
--client-cert-auth=true \
--trusted-ca-file=/home/cfssl/ca.pem \
--cert-file=/home/cfssl/server-node3.pem \
--key-file=/home/cfssl/server-node3-key.pem 检查
验证集群是否成功创建，使用etcdctl需要跟上证书路径
etcdctl --endpoints=https://192.168.0.53:2379 --cacert=/home/cfssl/ca.pem --cert=/home/cfssl/client.pem --key=/home/cfssl/client-key.pem member list --write-out=table