1. 在logstash中配置es的域名无法使用
output {
elasticsearch {
hosts => ["http://estest_hostmy.com:10200"]
index => "%{[@metadata][index_0]}-%{+YYYY.MM.dd}"
}
}
报错如下
17378 [2018-11-22T15:46:00,659][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://:9200/][Manticore::ClientProtocolException] URI does not specify a valid host name: http:/"}
17379 [2018-11-22T15:46:05,660][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://:9200/, :path=>"/"}
17380 [2018-11-22T15:46:05,660][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://:9200/, :path=>"/"}
17381 [2018-11-22T15:46:05,660][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://:9200/, :path=>"/"}
17382 [2018-11-22T15:46:05,660][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://:9200/, :path=>"/"}
17383 [2018-11-22T15:46:05,660][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://:9200/, :path=>"/"}
17384 [2018-11-22T15:46:05,660][WARN ][logstash.outputs.elasticsearch] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://:9200/][Manticore::ClientProtocolException] URI does not specify a valid host name: http:/"}
一度以为就是logstash不支持域名配置。
但是logstash的官网介绍的是 type 为uri 链接
hosts
Value type is uri
Default value is [//127.0.0.1]
Sets the host(s) of the remote instance. If given an array it will load balance requests across the hosts specified in the hosts parameter. Remember the http protocol uses the http address (eg. 9200, not 9300). “127.0.0.1” [“127.0.0.1:9200”,“127.0.0.2:9200”] [“http://127.0.0.1”] [“https://127.0.0.1:9200”] [“https://127.0.0.1:9200/mypath”] (If using a proxy on a subpath) It is important to exclude dedicated master nodes from the hosts list to prevent LS from sending bulk requests to the master nodes. So this parameter should only reference either data or client nodes in Elasticsearch.
Any special characters present in the URLs here MUST be URL escaped! This means # should be put in as %23 for instance.
??左思右想觉得不太对劲，后来想到查到logstash的 output-elasticsearch-plugin的官方github，通过一个issue 才发现原来是我对域名配置不够熟悉(在这个issue看到一半的时候以为是logstash的 logstash-output-elasticsearch 插件版本问题，打脸，以后要看完)，域名里面的规范是不能有下划线(_)的。
??看来最基本的知识有时候不牢固会造成很多问题，一定要有质疑能力，还好看了es的官网是可以配置url的，所以坚持查了查，要不然估计就认知错误了。坑。要多学习，孩砸。
修改为下面的方式就ok了
output {
elasticsearch {
hosts => ["http://estes-thostmy.com:10200"]
index => "%{[@metadata][index_0]}-%{+YYYY.MM.dd}"
}
}
github issue 链接
2. logstash.yml配置不生效
在logstash.yml中配置监控数据收集到es当中，但是一直不生效，数据无法进入到es当中，根据实际经验，类似下面配置没有问题。
path.data: /home/dev/logstash/logstash_data
config.reload.interval: 3s
dead_letter_queue.enable: false
http.port: 9600-9700
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: "logstash_system"
xpack.monitoring.elasticsearch.password: "d1Acosm6Eo"
xpack.monitoring.elasticsearch.hosts: "http://uc-k8s3:9200"
xpack.monitoring.collection.pipeline.details.enabled: true
??后来猛然想起来是yml的格式有问题，因为配置项提交多，注释项也很多（最开始上面的配置中夹杂了很多其他注释），发现是因为前面的一些配置和后面的配置没有按照yml格式对齐，删掉所有的注释性文字，整理成上面就可以了
|