问题现象:
21/11/28 20:29:55 WARN Shell: Did not find winutils.exe: {}
java.io.FileNotFoundException: Could not locate Hadoop executable: D:\Program\hadoop-3.3.1\bin\winutils.exe -see https://wiki.apache.org/hadoop/WindowsProblems
at org.apache.hadoop.util.Shell.getQualifiedBinInner(Shell.java:619)
at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:592)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:689)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
at org.apache.hadoop.conf.Configuration.getTimeDurationHelper(Configuration.java:1886)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1846)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1819)
at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
at org.apache.hadoop.util.ShutdownHookManager$HookEntry.<init>(ShutdownHookManager.java:207)
at org.apache.hadoop.util.ShutdownHookManager.addShutdownHook(ShutdownHookManager.java:304)
at org.apache.spark.util.SparkShutdownHookManager.install(ShutdownHookManager.scala:181)
at org.apache.spark.util.ShutdownHookManager$.shutdownHooks$lzycompute(ShutdownHookManager.scala:50)
?
?
解决:
这个问题是大部分初学者都会犯的错误,解决办法就是将官网下的源码在Windows上编译一下,但是这个相当麻烦。
简单方法就是直接将编译好的winutils.exe文件放在Hadoop安装文件夹得bin包下,重启IDE
https://url68.ctfile.com/f/1445568-522738759-a23515 (访问密码:5719)
?
|