[大数据]mongo-hadoop集成

当hadoop集群需要借助mongodb的数据进行辅助分析时，为了快速实施我们需要将hive集成mongodb

1. 下载jar包，放到hive节点的第三方包/etc/hive/auxlib目录下(这个目录通过hive? hive.aux.jars.path属性配置)

wget?https://repo1.maven.org/maven2/org/mongodb/mongo-hadoop/mongo-hadoop-core/2.0.2/mongo-hadoop-core-2.0.2.jar;?
?
wget?https://repo1.maven.org/maven2/org/mongodb/mongo-hadoop/mongo-hadoop-hive/2.0.2/mongo-hadoop-hive-2.0.2.jar;

wget?https://repo1.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.1/mongo-java-driver-3.2.1.jar;

2 . 重启hive

3.? 创建hive-mongo映射表

CREATE?external?TABLE?ods_t.o_tenant_config_external_mongo(
    id string,
    tenant_credit_config string,
    tenant_common_config string,
    tenant_freight_config string,
    sku_category string,
    last_updated_at?TIMESTAMP
)
STORED?BY?'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH?SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","tenant_credit_config":"tenantCreditConfig","tenant_common_config":"tenantCommonConfig","tenant_freight_config":"tenantFreightConfig","sku_category":"skuCategory","last_updated_at":"lastUpdatedAt"}')
TBLPROPERTIES('mongo.uri'='mongodb://account_name:pwd@255.255.255.255:27017/account_db.t_tenant_config');

注意：

? ? ①. mongo-java-driver jar版本不能低于mongodb组件的版本

? ? ②. 包访问权限不足时:??chmod 777?/etc/hive/auxlib/mongo-*

? ? ③. where条件过滤时，不能用 = 可以用 in 或 like，join时 = 可以