|
os: centos 7.6.1810
db: hbase 2.3.x
hadoop 2.10.x
zookeeper 3.6
jdk 1.8
NoNode for /hbase/master
[hbase@hb1 conf]$ hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.10.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hbase-2.3.6/lib/client-facing-thirdparty/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.3.6, r7414579f2620fca6b75146c29ab2726fc4643ac9, Wed Jul 28 22:24:42 UTC 2021
Took 0.0012 seconds
hbase(main):001:0> status
ERROR: KeeperErrorCode = NoNode for /hbase/master
For usage try 'help "status"'
Took 0.4600 seconds
hbase(main):002:0>
使用 jps 查看时，发现没有 HMaster
[hbase@hb1 ~]$ jps
5458 Main
4697 HRegionServer
6031 Jps
查看日志，发现有报错
$ cat hbase-hbase-master-hb1.log
2021-08-24 11:27:26,110 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000001
2021-08-24 11:27:26,139 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000002
2021-08-24 11:27:26,166 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000003
2021-08-24 11:27:26,195 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000004
2021-08-24 11:27:26,213 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000005
2021-08-24 11:27:26,241 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000006
2021-08-24 11:27:26,264 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000007
2021-08-24 11:27:26,317 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000008
2021-08-24 11:27:26,338 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000009
2021-08-24 11:27:26,357 WARN [main] util.FSTableDescriptors: Failed cleanup of hdfs://hadoop-ha/hbase/data/hbase/meta/.tmp/.tableinfo.0000000010
2021-08-24 11:27:26,358 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer
java.io.IOException: Failed update hbase:meta table descriptor
at org.apache.hadoop.hbase.util.FSTableDescriptors.tryUpdateMetaTableDescriptor(FSTableDescriptors.java:144)
at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:738)
at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:635)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:528)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:3163)
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:253)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:149)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3181)
2021-08-24 11:27:26,376 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster.
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:3170)
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:253)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:149)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3181)
Caused by: java.io.IOException: Failed update hbase:meta table descriptor
at org.apache.hadoop.hbase.util.FSTableDescriptors.tryUpdateMetaTableDescriptor(FSTableDescriptors.java:144)
at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:738)
at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:635)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:528)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:3163)
... 5 more
使用 hbase 用户执行创建目录权限，提示 Permission denied
$ hadoop fs -mkdir -p /hbase
mkdir: Permission denied: user=hbase, access=WRITE, inode="/":hadoop:supergroup:drwxr-xr-x
但是使用 hadoop 用户是可以创建目录的
$ hadoop fs -mkdir -p /peiyb2/tmp
$ hdfs dfs -fs hdfs://hadoop-ha -ls -R /peiyb2
drwxr-xr-x - hadoop supergroup 0 2021-08-24 14:55 /peiyb2/tmp
解决方法
# su - hadoop
$ cd $HADOOP_HOME
$ vi ./etc/hadoop/hdfs-site.xml
<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>
重启 hdfs,yarn
$ stop-yarn.sh
$ stop-dfs.sh
$ start-dfs.sh
$ start-yarn.sh
验证
# su - hbase
$ hadoop fs -mkdir -p /peiyb3/tmp
$
$
$ hdfs dfs -fs hdfs://hadoop-ha -ls -R /peiyb3
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:07 /peiyb3/tmp
1.停止 hbase 所有服务
# su - hbase
$ stop-hbase.sh
2.启动 hbase 所有服务
# su - hbase
$ start-hbase.sh
$ jps
16289 HRegionServer
27832 Jps
27454 HMaster
参考:
https://programmerah.com/solved-failed-update-hbasemeta-table-descriptor-hbase-startup-error-34463/
$ hdfs dfs -ls -R /
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/.hbck
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/.tmp
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/.tmp/data
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/.tmp/data/hbase
-rw-r--r-- 2 hbase supergroup 191 2021-08-24 15:12 /hbase/.tmp/hbase-hbck.lock
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/WALs
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/WALs/hb1,16000,1629789164352
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/WALs/hb1,16000,1629789164352/hb1%2C16000%2C1629789164352.1629789179423
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/archive
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/data
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/data/master
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/data/master/store
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682
-rw-r--r-- 2 hbase supergroup 34 2021-08-24 15:12 /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.regioninfo
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/MasterData/oldWALs
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/WALs
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/WALs/hb1,16020,1629784615445
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:13 /hbase/WALs/hb1,16020,1629784615445/hb1%2C16020%2C1629784615445.1629789193912
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/WALs/hb2,16020,1629784614918
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:13 /hbase/WALs/hb2,16020,1629784614918/hb2%2C16020%2C1629784614918.1629789192798
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/WALs/hb3,16020,1629784614801
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:13 /hbase/WALs/hb3,16020,1629784614801/hb3%2C16020%2C1629784614801.1629789192645
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:13 /hbase/WALs/hb3,16020,1629784614801/hb3%2C16020%2C1629784614801.meta.1629789190448.meta
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/archive
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/corrupt
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/default
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/.tabledesc
-rw-r--r-- 2 hbase supergroup 902 2021-08-24 15:12 /hbase/data/hbase/meta/.tabledesc/.tableinfo.0000000003
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/.tmp
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/1588230740
-rw-r--r-- 2 hbase supergroup 32 2021-08-24 15:12 /hbase/data/hbase/meta/1588230740/.regioninfo
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/1588230740/info
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/1588230740/recovered.edits
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/1588230740/recovered.edits/1.seqid
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/1588230740/rep_barrier
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/data/hbase/meta/1588230740/table
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase/namespace
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase/namespace/.tabledesc
-rw-r--r-- 2 hbase supergroup 286 2021-08-24 15:13 /hbase/data/hbase/namespace/.tabledesc/.tableinfo.0000000001
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase/namespace/.tmp
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase/namespace/e8495e71d82a0b96d3025d7a897597d8
-rw-r--r-- 2 hbase supergroup 42 2021-08-24 15:13 /hbase/data/hbase/namespace/e8495e71d82a0b96d3025d7a897597d8/.regioninfo
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase/namespace/e8495e71d82a0b96d3025d7a897597d8/info
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase/namespace/e8495e71d82a0b96d3025d7a897597d8/recovered.edits
-rw-r--r-- 2 hbase supergroup 0 2021-08-24 15:13 /hbase/data/hbase/namespace/e8495e71d82a0b96d3025d7a897597d8/recovered.edits/1.seqid
-rw-r--r-- 2 hbase supergroup 42 2021-08-24 15:12 /hbase/hbase.id
-rw-r--r-- 2 hbase supergroup 7 2021-08-24 15:12 /hbase/hbase.version
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/mobdir
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:12 /hbase/oldWALs
drwx--x--x - hbase supergroup 0 2021-08-24 15:12 /hbase/staging
drwxr-xr-x - hadoop supergroup 0 2021-08-24 11:11 /peiyb
drwxr-xr-x - hadoop supergroup 0 2021-08-24 11:11 /peiyb/tmp
drwxr-xr-x - hadoop supergroup 0 2021-08-24 14:55 /peiyb2
drwxr-xr-x - hadoop supergroup 0 2021-08-24 14:55 /peiyb2/tmp
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:07 /peiyb3
drwxr-xr-x - hbase supergroup 0 2021-08-24 15:07 /peiyb3/tmp
|