在Windows环境中IDEA下编译运行Elasticsearch 7.14.1

环境准备

JDK 16
Gradle 7.1

为了更快的下载，可以使用阿里的仓库服务，在解压的文件夹中的init.d文件夹下新建文件init.gradle

allprojects{
    repositories {
	    maven{ url 'https://maven.aliyun.com/repository/public'}
	    maven{ url 'https://maven.aliyun.com/repository/google'}
        maven{ url 'https://maven.aliyun.com/repository/gradle-plugin'}
        maven{ url 'https://maven.aliyun.com/repository/jcenter'}
		
        
	
        def REPOSITORY_URL = 'https://maven.aliyun.com/repository/public/'
        all {
            ArtifactRepository repo ->
    if (repo instanceof MavenArtifactRepository) {
                def url = repo.url.toString()
                if (url.startsWith('https://artifacts.elastic.co/maven')
				||url.startsWith('https://repo1.maven.org/maven2') 
				|| url.startsWith('https://repo.maven.org/maven2') 
				|| url.startsWith('https://jcenter.bintray.com/')
				||url.startsWith('https://repo.maven.apache.org/maven2')
				) {
                    project.logger.lifecycle "Repository ${repo.url} replaced by $REPOSITORY_URL."
                    remove repo
                }
            }
        }
        maven {
            url REPOSITORY_URL
        }
    }
}

Git 2.23
IDEA 2021.2
Elasticsearch Windows版的发布包

如何确认JDK以及Gradle的版本

操作步骤

获取源码

方式一：直接通过`git bash`执行`git clone`命令

打开这个命名行工具，git clone https://github.com/elastic/elasticsearch.git，切换到7.14分支上。

方式二：通过IDEA界面化操作

在这里插入图片描述

通过右下角工具获取远程分支到本地

github仓库比较卡，可以使用gitee与github做映射，先同步至gitee上，在从gitee上进行clone。

编译源码

用IDEA打开项目工程。

配置Gradle
在这里插入图片描述
自动编译

本地运行

在这里插入图片描述

VM配置

-Des.path.home=E:\tmp\es\elasticsearch-7.14.1
-Des.path.conf=E:\tmp\es\elasticsearch-7.14.1\config
-Dlog4j2.disable.jmx=true
-Xmx4g
-Xms4g

遇到的问题

1. 权限问题

[2021-09-07T17:49:40,721][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Unknown codebases [codebase.elasticsearch-plugin-classloader, codebase.elasticsearch, codebase.elasticsearch-secure-sm] in policy file [file:/E:/code/source-code/elasticsearch7/server/out/production/resources/org/elasticsearch/bootstrap/security.policy]
Available codebases: [codebase.jna-5.7.0-1.jar, codebase.lucene-grouping, codebase.t-digest-3.2.jar, codebase.joda-time, codebase.lucene-analyzers-common-8.9.0.jar, codebase.lucene-backward-codecs, codebase.joda-time-2.10.10.jar, codebase.lucene-highlighter, codebase.log4j-api-2.11.1.jar, codebase.lucene-memory, codebase.jopt-simple, codebase.jts-core, codebase.lucene-suggest-8.9.0.jar, codebase.hppc, codebase.jna, codebase.hppc-0.8.1.jar, codebase.spatial4j-0.7.jar, codebase.lucene-join, codebase.log4j-api, codebase.HdrHistogram, codebase.jackson-dataformat-cbor, codebase.lucene-core, codebase.lucene-spatial3d-8.9.0.jar, codebase.spatial4j, codebase.log4j-core-2.11.1.jar, codebase.snakeyaml-1.26.jar, codebase.snakeyaml, codebase.jackson-dataformat-yaml-2.10.4.jar, codebase.lucene-backward-codecs-8.9.0.jar, codebase.lucene-spatial-extras-8.9.0.jar, codebase.HdrHistogram-2.1.9.jar, codebase.lucene-misc-8.9.0.jar, codebase.t-digest, codebase.lucene-highlighter-8.9.0.jar, codebase.lucene-join-8.9.0.jar, codebase.lucene-analyzers-common, codebase.jackson-dataformat-cbor-2.10.4.jar, codebase.lucene-queryparser-8.9.0.jar, codebase.lz4-java-1.8.0.jar, codebase.lucene-spatial-extras, codebase.lucene-memory-8.9.0.jar, codebase.lucene-queries-8.9.0.jar, codebase.lucene-queryparser, codebase.jts-core-1.15.0.jar, codebase.jackson-core-2.10.4.jar, codebase.lz4-java, codebase.jackson-dataformat-yaml, codebase.jackson-core, codebase.log4j-core, codebase.lucene-queries, codebase.lucene-grouping-8.9.0.jar, codebase.lucene-sandbox, codebase.lucene-suggest, codebase.lucene-sandbox-8.9.0.jar, codebase.jackson-dataformat-smile, codebase.jopt-simple-5.0.2.jar, codebase.lucene-misc, codebase.lucene-spatial3d, codebase.lucene-core-8.9.0.jar, codebase.jackson-dataformat-smile-2.10.4.jar]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[classes/:?]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[classes/:?]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[classes/:?]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116) ~[classes/:?]
	at org.elasticsearch.cli.Command.main(Command.java:79) ~[classes/:?]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[classes/:?]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[classes/:?]
Caused by: java.lang.IllegalArgumentException: Unknown codebases [codebase.elasticsearch-plugin-classloader, codebase.elasticsearch, codebase.elasticsearch-secure-sm] in policy file [file:/E:/code/source-code/elasticsearch7/server/out/production/resources/org/elasticsearch/bootstrap/security.policy]
Available codebases: [codebase.jna-5.7.0-1.jar, codebase.lucene-grouping, codebase.t-digest-3.2.jar, codebase.joda-time, codebase.lucene-analyzers-common-8.9.0.jar, codebase.lucene-backward-codecs, codebase.joda-time-2.10.10.jar, codebase.lucene-highlighter, codebase.log4j-api-2.11.1.jar, codebase.lucene-memory, codebase.jopt-simple, codebase.jts-core, codebase.lucene-suggest-8.9.0.jar, codebase.hppc, codebase.jna, codebase.hppc-0.8.1.jar, codebase.spatial4j-0.7.jar, codebase.lucene-join, codebase.log4j-api, codebase.HdrHistogram, codebase.jackson-dataformat-cbor, codebase.lucene-core, codebase.lucene-spatial3d-8.9.0.jar, codebase.spatial4j, codebase.log4j-core-2.11.1.jar, codebase.snakeyaml-1.26.jar, codebase.snakeyaml, codebase.jackson-dataformat-yaml-2.10.4.jar, codebase.lucene-backward-codecs-8.9.0.jar, codebase.lucene-spatial-extras-8.9.0.jar, codebase.HdrHistogram-2.1.9.jar, codebase.lucene-misc-8.9.0.jar, codebase.t-digest, codebase.lucene-highlighter-8.9.0.jar, codebase.lucene-join-8.9.0.jar, codebase.lucene-analyzers-common, codebase.jackson-dataformat-cbor-2.10.4.jar, codebase.lucene-queryparser-8.9.0.jar, codebase.lz4-java-1.8.0.jar, codebase.lucene-spatial-extras, codebase.lucene-memory-8.9.0.jar, codebase.lucene-queries-8.9.0.jar, codebase.lucene-queryparser, codebase.jts-core-1.15.0.jar, codebase.jackson-core-2.10.4.jar, codebase.lz4-java, codebase.jackson-dataformat-yaml, codebase.jackson-core, codebase.log4j-core, codebase.lucene-queries, codebase.lucene-grouping-8.9.0.jar, codebase.lucene-sandbox, codebase.lucene-suggest, codebase.lucene-sandbox-8.9.0.jar, codebase.jackson-dataformat-smile, codebase.jopt-simple-5.0.2.jar, codebase.lucene-misc, codebase.lucene-spatial3d, codebase.lucene-core-8.9.0.jar, codebase.jackson-dataformat-smile-2.10.4.jar]
	at org.elasticsearch.bootstrap.PolicyUtil.readPolicy(PolicyUtil.java:261) ~[classes/:?]
	at org.elasticsearch.bootstrap.ESPolicy.<init>(ESPolicy.java:44) ~[classes/:?]
	at org.elasticsearch.bootstrap.Security.configure(Security.java:106) ~[classes/:?]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:214) ~[classes/:?]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[classes/:?]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[classes/:?]
	... 6 more

从错误信息看出在server模块下的security.policy文件中存在了对codebase.elasticsearch-plugin-classloader, codebase.elasticsearch, codebase.elasticsearch-secure-sm的使用但是却有加载这几个jar

修改建议：

将关于三个jar的授权信息拷贝至公共授权方法体中，之后注释这三个jar的授权方法体

2. 版本问题

[2021-09-07T17:48:05,568][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Plugin [repository-url] was built for Elasticsearch version 7.14.1 but version 7.14.2 is running

修改建议：修改文件org.elasticsearch.Version中的对象CURRENT的值为V_7_14_1。

3. geoip报错信息

[2021-09-07T17:18:24,147][ERROR][o.e.i.g.GeoIpDownloader  ] [] exception during geoip databases update
java.net.SocketTimeoutException: connect timed out
	at java.net.PlainSocketImpl.waitForConnect(Native Method) ~[?:?]
	at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107) ~[?:?]
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[?:?]
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[?:?]
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[?:?]
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]
	at java.net.Socket.connect(Socket.java:608) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:287) ~[?:?]
	at sun.net.NetworkClient.doConnect(NetworkClient.java:177) ~[?:?]
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[?:?]
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[?:?]
	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265) ~[?:?]
	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372) ~[?:?]
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:203) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1071) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1069) ~[?:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:795) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1068) ~[?:?]
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:189) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1592) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1512) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1510) ~[?:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
	at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:795) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509) ~[?:?]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) ~[?:?]
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:334) ~[?:?]
	at org.elasticsearch.ingest.geoip.HttpClient.lambda$get$0(HttpClient.java:55) ~[ingest-geoip-7.14.1.jar:7.14.1]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
	at org.elasticsearch.ingest.geoip.HttpClient.doPrivileged(HttpClient.java:97) ~[ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.ingest.geoip.HttpClient.get(HttpClient.java:49) ~[ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.ingest.geoip.HttpClient.getBytes(HttpClient.java:40) ~[ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.fetchDatabasesOverview(GeoIpDownloader.java:115) ~[ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:103) ~[ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:235) [ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:94) [ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:43) [ingest-geoip-7.14.1.jar:7.14.1]
	at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:40) [classes/:?]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732) [classes/:?]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [classes/:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]

此版本将GeoIp功能默认开启了采集。在默认的启动下是会去官网的默认地址下获取最新的Ip的GEO信息。
官方说明 geoip-processor

Cluster settings

ingest.geoip.downloader.enabled

(Dynamic, Boolean) If true, Elasticsearch automatically downloads and manages updates for GeoIP2 databases from the ingest.geoip.downloader.endpoint. If false, Elasticsearch does not download updates and deletes all downloaded databases. Defaults to true.
ingest.geoip.downloader.endpoint

(Static, string) Endpoint URL used to download updates for GeoIP2 databases. Defaults to https://geoip.elastic.co/v1/database. Elasticsearch stores downloaded database files in each node’s temporary directory at $ES_TMPDIR/geoip-databases/<node_id>.
ingest.geoip.downloader.poll.interval

(Dynamic, time value) How often Elasticsearch checks for GeoIP2 database updates at the ingest.geoip.downloader.endpoint. Must be greater than 1d (one day). Defaults to 3d (three days).