IT数码 购物 网址 头条 软件 日历 阅读 图书馆
TxT小说阅读器
↓语音阅读,小说下载,古典文学↓
图片批量下载器
↓批量下载图片,美女图库↓
图片自动播放器
↓图片自动播放器↓
一键清除垃圾
↓轻轻一点,清除系统垃圾↓
开发: C++知识库 Java知识库 JavaScript Python PHP知识库 人工智能 区块链 大数据 移动开发 嵌入式 开发工具 数据结构与算法 开发测试 游戏开发 网络协议 系统运维
教程: HTML教程 CSS教程 JavaScript教程 Go语言教程 JQuery教程 VUE教程 VUE3教程 Bootstrap教程 SQL数据库教程 C语言教程 C++教程 Java教程 Python教程 Python3教程 C#教程
数码: 电脑 笔记本 显卡 显示器 固态硬盘 硬盘 耳机 手机 iphone vivo oppo 小米 华为 单反 装机 图拉丁
 
   -> 大数据 -> Win10搭建ELK8.0.0环境 -> 正文阅读

[大数据]Win10搭建ELK8.0.0环境

下载

参考官方下载地址

Elasticsearch

配置

配置文件路径 .\elasticsearch-8.0.0\config\elasticsearch.yml

# 主机 IP,尽量不要使用回环地址,而是要使用私网地址
network.host: 127.0.0.1

# 端口
http.port: 9200

# 禁止下载 Geoip
ingest.geoip.downloader.enabled: false

# 配置跨域
http.cors.enabled: true
http.cors.allow-origin: "*"

启动

运行 .\elasticsearch-8.0.0\bin\elasticsearch.bat 脚本即可。

首次启动会自动配置安全信息:

-> Elasticsearch security features have been automatically configured!
-> Authentication is enabled and cluster connections are encrypted.

->  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  DycJB*X5KOjHuTq33tIu

->  HTTP CA certificate SHA-256 fingerprint:
  bb5dd53131d6e160892c406dc26e36963a5f8c32aa4c330b1e7b77aeac0ca45a

->  Configure Kibana to use this cluster:
* Run Kibana and click the configuration link in the terminal when Kibana starts.
* Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjAuMCIsImFkciI6WyIxMjcuMC4wLjE6OTIwMCJdLCJmZ3IiOiJiYjVkZDUzMTMxZDZlMTYwODkyYzQwNmRjMjZlMzY5NjNhNWY4YzMyYWE0YzMzMGIxZTdiNzdhZWFjMGNhNDVhIiwia2V5IjoiSC01OVFIOEJhb25naHNKU0x6RE46R0x4NnRLTkpRVUN4VGx0b2d6MVVoUSJ9

->  Configure other nodes to join this cluster:
* On this node:
  - Create an enrollment token with `bin/elasticsearch-create-enrollment-token -s node`.
  - Uncomment the transport.host setting at the end of config/elasticsearch.yml.
  - Restart Elasticsearch.
* On other nodes:
  - Start Elasticsearch with `bin/elasticsearch --enrollment-token <token>`, using the enrollment token that you generated.

在 .\elasticsearch-8.0.0\config\elasticsearch.yml 中自动新增安全配置:

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 28-02-2022 13:19:01
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["DESKTOP-L24D7IP"]

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

启动成功:

使用 https 访问?https:127.0.0.1:9200,输入控制台中给出的账号密码:

可以看到启动成功了。 当然,账号密码可以自己设定:

bin/elasticsearch-reset-password -u elastic

elasticsearch-head 插件?

# github 地址
https://github.com/mobz/elasticsearch-head
 
# npm 启动方式
git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
npm install
npm run start
open http://localhost:9100/

如果遇到安装错误:

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

执行忽略脚本安装指令即可:

 npm install phantomjs-prebuilt@2.1.16 --ignore-scripts

最终完成启动:

还需要再配置 elasticsearch.yml 实现访问 https:

# 开启权限认证后,es-head-master 访问 es 需要的配置
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

http://headIP:9100/?base_uri=https://ESIP:9200&auth_user=elastic&auth_password=yourPwd

重置 Logstash 和 Kibana?账号密码

elasticsearch-reset-password -i -u username

Logstash

配置

# ------------ X-Pack Settings (not applicable for OSS build)--------------
#
# X-Pack Monitoring
# https://www.elastic.co/guide/en/logstash/current/monitoring-logstash.html
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: logstash_system
#xpack.monitoring.elasticsearch.proxy: ["http://proxy:port"]
xpack.monitoring.elasticsearch.hosts: ["https://192.168.2.11:9200"]
# an alternative to hosts + username/password settings is to use cloud_id/cloud_auth
#xpack.monitoring.elasticsearch.cloud_id: monitoring_cluster_id:xxxxxxxxxx
#xpack.monitoring.elasticsearch.cloud_auth: logstash_system:password
# another authentication alternative is to use an Elasticsearch API key
#xpack.monitoring.elasticsearch.api_key: "id:api_key"
xpack.monitoring.elasticsearch.ssl.certificate_authority: "D:/elasticsearch-8.0.0/config/certs/http_ca.crt"
#xpack.monitoring.elasticsearch.ssl.truststore.path: path/to/file
#xpack.monitoring.elasticsearch.ssl.truststore.password: password
#xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
#xpack.monitoring.elasticsearch.ssl.keystore.password: password
xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
xpack.monitoring.elasticsearch.sniffing: false
#xpack.monitoring.collection.interval: 10s
#xpack.monitoring.collection.pipeline.details.enabled: true

在 config 目录下新建 logstash.conf 作为日志输入输出配置:

# logstash.conf 日志捕获从指定路径的 access.log 文件中获得
# 输出到 es 的 "access-%{+YYYY.MM.dd}" 索引中,索引不存在则自动创建
# 同时考虑到是 https 访问,需要配置 ssl


input {

    file {

        type => "nginx_access"

        path => "D:/testlogs/access.log"

    }

}

output {

    elasticsearch {

        hosts => ["https://192.168.2.11:9200"]

        index => "access-%{+YYYY.MM.dd}"

        user => "logstash_system"

        password => "logstash_system"
        
        ssl => true
        
        ssl_certificate_verification => true
        
        cacert => "D:/elasticsearch-8.0.0/config/certs/http_ca.crt"
    }

    stdout {

        codec => json_lines

    }

}

启动

D:\logstash-8.0.0\bin>logstash -f ../config/logstash.conf

在往 access.log 写入数据时,同步到 es :

Kibana

配置

配置文件在 .\kibana-8.0.0\config\kibana.yml
# 注意:IP 地址切勿使用回环地址,应使用私网地址
# SSL 中的 PEM 证书使用 elasticsearch 中的证书

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: 192.168.2.11

# The maximum payload size in bytes for incoming server requests.
server.maxPayload: 1048576

# The Kibana server's name. This is used for display purposes.
server.name: "kibaba-host"

# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://192.168.2.11:9200"]

# If your Elasticsearch is protected with basic authentication, these settings provide
# the username and password that the Kibana server uses to perform maintenance on the Kibana
# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
# is proxied through the Kibana server.
elasticsearch.username: "kibana_system"
elasticsearch.password: "LO50Eqdeow7v2Q7PVpTb"

# Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
# the elasticsearch.requestTimeout setting.
elasticsearch.pingTimeout: 1500

# Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
# must be a positive integer.
elasticsearch.requestTimeout: 30000

# Enables you to specify a path to the PEM file for the certificate
# authority for your Elasticsearch instance.
elasticsearch.ssl.certificateAuthorities: [ "D:/elasticsearch-8.0.0/config/certs/http_ca.crt" ]

# To disregard the validity of SSL certificates, change this setting's value to 'none'.
elasticsearch.ssl.verificationMode: certificate

启动

执行 .\kibana-8.0.0\bin\kibana.bat 即可。

?访问??http://localhost:5601 时需要输入 elastic 管理员账号密码:

创建数据视图:

management/kibana/dataViews

创建模板为 access-* 的数据视图,自动匹配?access-2022.03.01 索引,不使用时间过滤器。

在?discover 页面就可以使用该数据视图,看到索引里面的数据。

  大数据 最新文章
实现Kafka至少消费一次
亚马逊云科技:还在苦于ETL?Zero ETL的时代
初探MapReduce
【SpringBoot框架篇】32.基于注解+redis实现
Elasticsearch:如何减少 Elasticsearch 集
Go redis操作
Redis面试题
专题五 Redis高并发场景
基于GBase8s和Calcite的多数据源查询
Redis——底层数据结构原理
上一篇文章      下一篇文章      查看所有文章
加:2022-03-10 22:36:15  更:2022-03-10 22:38:58 
 
开发: C++知识库 Java知识库 JavaScript Python PHP知识库 人工智能 区块链 大数据 移动开发 嵌入式 开发工具 数据结构与算法 开发测试 游戏开发 网络协议 系统运维
教程: HTML教程 CSS教程 JavaScript教程 Go语言教程 JQuery教程 VUE教程 VUE3教程 Bootstrap教程 SQL数据库教程 C语言教程 C++教程 Java教程 Python教程 Python3教程 C#教程
数码: 电脑 笔记本 显卡 显示器 固态硬盘 硬盘 耳机 手机 iphone vivo oppo 小米 华为 单反 装机 图拉丁

360图书馆 购物 三丰科技 阅读网 日历 万年历 2024年11日历 -2024/11/24 8:41:22-

图片自动播放器
↓图片自动播放器↓
TxT小说阅读器
↓语音阅读,小说下载,古典文学↓
一键清除垃圾
↓轻轻一点,清除系统垃圾↓
图片批量下载器
↓批量下载图片,美女图库↓
  网站联系: qq:121756557 email:121756557@qq.com  IT数码