1. 异常描述
使用 beeline 连接 hivesever 时，抛出异常如下：
[root@vhost-120-28 hive]# beeline -u "jdbc:hive2://vhost-120-26:2181,vhost-120-27:2181,vhost-120-28:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" --verbose=true
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/wdp/1.0/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/wdp/1.0/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
!connect jdbc:hive2://vhost-120-26:2181,vhost-120-27:2181,vhost-120-28:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 '' [passwd stripped]
Connecting to jdbc:hive2://vhost-120-26:2181,vhost-120-27:2181,vhost-120-28:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper (state=,code=0)
java.sql.SQLException: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:170)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.hive.beeline.DatabaseConnection.connect(DatabaseConnection.java:145)
at org.apache.hive.beeline.DatabaseConnection.getConnection(DatabaseConnection.java:209)
at org.apache.hive.beeline.Commands.connect(Commands.java:1641)
at org.apache.hive.beeline.Commands.connect(Commands.java:1536)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.beeline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:56)
at org.apache.hive.beeline.BeeLine.execCommandWithPrefix(BeeLine.java:1384)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1423)
at org.apache.hive.beeline.BeeLine.connectUsingArgs(BeeLine.java:900)
at org.apache.hive.beeline.BeeLine.initArgs(BeeLine.java:795)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1048)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:538)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:520)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper
at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:147)
at org.apache.hive.jdbc.Utils.configureConnParamsFromZooKeeper(Utils.java:511)
at org.apache.hive.jdbc.Utils.parseURL(Utils.java:334)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168)
... 25 more
2. 原因分析：
HiveServer2.java startPrivilegeSynchonizer 里注册的 znode 和 hiveserver2 的 znode 在同一个目录下。
如系统有 3 台 hiveserver2，那么会出现 4 个 znode， beeline 连接的时候，会从里面随机选择一台，25% 的概率选择 leader。
3. 解决办法
3.1 HiveServer2 和 PrivilegeSynchonizer 放在不同的 znode 下
社区 jira:HIVE-19825 ，没有合入。
3.2 选择 znode 的时候过滤
社区 jira:HIVE-25096 ，没有合入。由于 HiveServer 的 znode 以 “serverUri=” 开头，所以仅保留以 “serverUri=” 开头的 znode，再从中随机选择一个。
3.3 不注册PrivilegeSynchonizer
不注册PrivilegeSynchonizer，就不会生成 leader 的 znode。
public void startPrivilegeSynchonizer(HiveConf hiveConf) throws Exception {
PolicyProviderContainer policyContainer = new PolicyProviderContainer();
HiveAuthorizer authorizer = SessionState.get().getAuthorizerV2();
if (authorizer.getHivePolicyProvider() != null) {
policyContainer.addAuthorizer(authorizer);
}
if (hiveConf.get(MetastoreConf.ConfVars.PRE_EVENT_LISTENERS.getVarname()) != null &&
hiveConf.get(MetastoreConf.ConfVars.PRE_EVENT_LISTENERS.getVarname()).contains(
"org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener") &&
hiveConf.get(MetastoreConf.ConfVars.HIVE_AUTHORIZATION_MANAGER.getVarname())!= null) {
List<HiveMetastoreAuthorizationProvider> providers = HiveUtils.getMetaStoreAuthorizeProviderManagers(
hiveConf, HiveConf.ConfVars.HIVE_METASTORE_AUTHORIZATION_MANAGER, SessionState.get().getAuthenticator());
for (HiveMetastoreAuthorizationProvider provider : providers) {
if (provider.getHivePolicyProvider() != null) {
policyContainer.addAuthorizationProvider(provider);
}
}
}
if (policyContainer.size() > 0) {
zKClientForPrivSync = startZookeeperClient(hiveConf);
String rootNamespace = hiveConf.getVar(HiveConf.ConfVars.HIVE_SERVER2_ZOOKEEPER_NAMESPACE);
String path = ZooKeeperHiveHelper.ZOOKEEPER_PATH_SEPARATOR + rootNamespace
+ ZooKeeperHiveHelper.ZOOKEEPER_PATH_SEPARATOR + "leader";
LeaderLatch privilegeSynchonizerLatch = new LeaderLatch(zKClientForPrivSync, path);
privilegeSynchonizerLatch.start();
Thread privilegeSynchonizerThread = new Thread(
new PrivilegeSynchonizer(privilegeSynchonizerLatch, policyContainer, hiveConf), "PrivilegeSynchonizer");
privilegeSynchonizerThread.start();
} else {
LOG.warn(
"No policy provider found, skip creating PrivilegeSynchonizer");
}
}
不注册的条件是 policyContainer.size() == 0。policyContainer 的 size 方法如下：
public int size() {
return authorizers.size() + authorizationProviders.size();
}
代表需要authorizers 和 authorizationProviders 都没有。
添加 authorizers 的条件：
if (authorizer.getHivePolicyProvider() != null) {
policyContainer.addAuthorizer(authorizer);
}
添加 AuthorizationProvider 的条件：
设置 PRE_EVENT_LISTENERS 包含 “org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListene”。MetastoreConf 中的对应参数如下：
PRE_EVENT_LISTENERS("metastore.pre.event.listeners", "hive.metastore.pre.event.listeners", "",
"List of comma separated listeners for metastore events."),
并且 provider 满足一下条件：
if (provider.getHivePolicyProvider() != null) {
policyContainer.addAuthorizationProvider(provider);
}
对于 provider ，当 HiveConf.ConfVars.HIVE_METASTORE_AUTHORIZATION_MANAGER 的值是 org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider ，就会创建 leader 节点。当 HiveConf.ConfVars.HIVE_METASTORE_AUTHORIZATION_MANAGER 的值是 org.apache.hadoop.hive.ql.security.authorization.DefaultHiveMetastoreAuthorizationProvider ，就不会创建 leader 节点。
3.3.1 设置有 leader
web-site.xml 中配置如下，则会出现 leader。当采用 Ranger 或者类似组件时，必须配置 metastore.pre.event.listeners。
<property>
<name>metastore.pre.event.listeners</name>
<value>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</value>
</property>
<property>
<name>hive.security.metastore.authorization.manager</name>
<value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
</property>
3.3.2 没有 leader
没有配置 metastore.pre.event.listeners，仅配置 hive.security.metastore.authorization.manager 不会有 leader。
<property>
<name>hive.security.metastore.authorization.manager</name>
<value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value>
</property>
3.3.3 没有 leader
以下配置也没有 leader，因为 DefaultHiveMetastoreAuthorizationProvider 的 getHivePolicyProvider() 返回 null。
<property>
<name>metastore.pre.event.listeners</name>
<value>org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener</value>
</property>
<property>
<name>hive.security.metastore.authorization.manager</name>
<value>org.apache.hadoop.hive.ql.security.authorization.DefaultHiveMetastoreAuthorizationProvider</value>
</property>
