[Java知识库]spark3.0.3编译

1. spark 的源码中自带编译脚本build/mvn 该脚本会自动下载 scala zinc maven,通过命令行直接下载速度会非常慢，建议将下载链接复制到迅雷，然后将下载好的文件解压到build目录下，然后执行编译脚本命令
   https://downloads.lightbend.com/scala/2.12.10/scala-2.12.10.tgz
https://downloads.lightbend.com/zinc/0.3.15/zinc-0.3.15.tgz
2. export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g" 设置maven运行时内存
3. ./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes
4. ./build/mvn -DskipTests clean package 该过程可能会因为依赖包下载而阻塞，建议ctrl-c然后再多次执行

[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.3.0:compile (scala-compile-first) on project spark-sql_2.12: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.3.0:compile failed.: CompileFailed -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :spark-sql_2.12

    <dependency>
        <groupId>net.alchim31.maven</groupId>
        <artifactId>scala-maven-plugin</artifactId>
        <version>4.3.0</version>
    </dependency>