[大数据]docker 部署 kafka

需要先安装好docker和docker-compose

1、新建docker-compose.yml 文件

[root@localhost kafka]# cat docker-compose.yml 
version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - 2181:2181
    privileged: true
  kafka:
    image: wurstmeister/kafka
    depends_on: [ zookeeper ]
    ports:
      - 9092:9092
    privileged: true
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 192.168.136.145
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
[root@localhost kafka]# 

KAFKA_ADVERTISED_HOST_NAME 是宿主机IP
KAFKA_ADVERTISED_PORT: 9092 默认kafka服务端口9092就行

这两个配置分别对应了kafka的配置文件/opt/kafka/config/server.properties的
advertised.host.name=192.168.136.145
advertised.port=9092

2、启动zookeeper和kafka

在docker-compose.yml 文件同级目录下执行

docker-compose up -d

3、部署过程遇到的问题

2021-09-01T06:52:42.596495000Z Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/c751ce1725f2/json": dial unix /var/run/docker.sock: connect: permission denied
2021-09-01T06:52:42.610503000Z [Configuring] 'port' in '/opt/kafka/config/server.properties'
2021-09-01T06:52:42.618265000Z [Configuring] 'advertised.host.name' in '/opt/kafka/config/server.properties'
2021-09-01T06:52:42.626435000Z Excluding KAFKA_HOME from broker config
2021-09-01T06:52:42.630749000Z [Configuring] 'log.dirs' in '/opt/kafka/config/server.properties'
2021-09-01T06:52:42.634005000Z Excluding KAFKA_VERSION from broker config
2021-09-01T06:52:42.638975000Z [Configuring] 'advertised.port' in '/opt/kafka/config/server.properties'
2021-09-01T06:52:42.648126000Z [Configuring] 'zookeeper.connect' in '/opt/kafka/config/server.properties'
2021-09-01T06:52:42.659601000Z [Configuring] 'broker.id' in '/opt/kafka/config/server.properties'
2021-09-01T06:52:43.636040000Z [2021-09-01 06:52:43,633] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
2021-09-01T06:52:43.958790000Z [2021-09-01 06:52:43,951] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
2021-09-01T06:52:43.959053000Z org.apache.kafka.common.config.ConfigException: Invalid value  for configuration advertised.port: Not a number of type INT
2021-09-01T06:52:43.959346000Z 	at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:727)
2021-09-01T06:52:43.959610000Z 	at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:475)
2021-09-01T06:52:43.959852000Z 	at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:468)
2021-09-01T06:52:43.960026000Z 	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
2021-09-01T06:52:43.960365000Z 	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)
2021-09-01T06:52:43.960651000Z 	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1309)
2021-09-01T06:52:43.961019000Z 	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1312)
2021-09-01T06:52:43.961330000Z 	at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:34)
2021-09-01T06:52:43.961540000Z 	at kafka.Kafka$.main(Kafka.scala:68)
2021-09-01T06:52:43.961699000Z 	at kafka.Kafka.main(Kafka.scala)

• Got permission denied while trying to connect to the Docker daemon socket
  因为挂载了 /var/run/docker.sock到容器内，所以启动容器时候需要添加权限。
  docker-compose.yml文件里面加上 privileged: true 参数就行
• Invalid value for configuration advertised.port: Not a number of type INT
  有些教程的yml文件没有配置 KAFKA_ADVERTISED_PORT，所以kafka启动时候读取配置文件失败了，导致容器无法启动。
  加上 KAFKA_ADVERTISED_PORT: 9092 就可以了

[参考]：
https://tomoyadeng.github.io/blog/2018/06/02/kafka-cluster-in-docker/index.html