sqoop可以采集关系型数据库中的数据到hdfs 等
官网链接 https://sqoop.apache.org/
官网使用链接 https://sqoop.apache.org/docs/1.4.7/SqoopUserGuide.html
一、下载压缩包
下载链接 http://archive.apache.org/dist/sqoop/
?放在任意位置，解压到 /usr/local目录下
?在压缩包的目录下输入 tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /usr/local
?重命名
[root@node3 tmp]# cd /usr/local
[root@node3 local]# mv sqoop-1.4.7.bin__hadoop-2.6.0/ sqoop
?sqoop的安装目录
[root@node3 sqoop]# pwd
/usr/local/sqoop
[root@node3 sqoop]#
二、配置环境变量
vi /etc/profile
SQOOP_HOME=/usr/local/sqoop
export PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$ZOOKEEPER_HOME/bin:$FLUME_HOME/bin:$CATALINA_HOME/bin:$SQOOP_HOME/bin
export HADOOP_HOME ZOOKEEPER_HOME FLUME_HOME CATALINA_HOME SQOOP_HOME
source /etc/profile
三、其他配置
?记住hadoop的安装目录
[root@node3 sqoop]# echo $HADOOP_HOME
/usr/local/hadoop271
?向sqoop配置文件中添加hadoop路径
vi $SQOOP_HOME/conf/sqoop-env.sh
export HADOOP_HOME=/usr/local/hadoop271
?向sqoop目录下的lib加入mysql的驱动包
[root@node3 sqoop]# cd $SQOOP_HOME/lib
[root@node3 lib]# ls
?向hadoop目录下的lib加入sqoop的jar包
[root@node3 hadoop271]# cd $HADOOP_HOME/lib
[root@node3 lib]# ls
native sqoop-1.4.7.jar
?寻找jar包的链接 https://mvnrepository.com/
?sqoop1.4.7的jar包 https://mvnrepository.com/artifact/org.apache.sqoop/sqoop/1.4.7
?mysql8的驱动包
https://mvnrepository.com/artifact/mysql/mysql-connector-java/8.0.28
四、测试
?查看sqoop版本：
[root@node3 lib]# sqoop version
Warning: /usr/local/sqoop/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
22/06/28 19:19:15 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017
[root@node3 lib]#
?查看数据库:
sqoop list-databases --connect jdbc:mysql://localhost:3306/mysql?serverTimezone=UTC --username root --password mysql的密码 --verbose
?导入数据库中的某张表:
sqoop import --connect jdbc:mysql://node3:3306/test?serverTimezone=UTC --username root --password mysql的密码 --table 表名