第二章 Hadoop运行环境搭建

2.1 Hadoop环境搭建

确定有多台机器，机器之间网络互通
创建用户hadp，useradd?hadp
创建目录，/opt下创建module?software目录, 并且赋权限
拷贝安装包到software
1. JDK ；?tar -zxvf jdk-8u251-linux-x64.tar.gz -C ../module/? 安装；配置PATH
2. Hadoop? ：官网比较慢，腾讯地址?Index of /apache/hadoop/? 解压；配置PATH

# set java
export JAVA_HOME=/opt/module/jdk1.8.0_251
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin

# hadoop
export HADOOP_HOME=/opt/module/hadoop-2.10.1
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin

? 5. hadoop的目录

2.2?Hadoop运行模式

2.1.1 本地（独立）模式

? ?官网案例：??

[root@localhost hadoop-2.10.1]# mkdir input
[root@localhost hadoop-2.10.1]# cp etc/hadoop/*.xml input/
[root@localhost hadoop-2.10.1]# ./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.10.1.jar grep input output 'dfs[a-z.]+'
[root@localhost hadoop-2.10.1]# cat output/*
1       dfsadmin

# 注意事项output目录不能存在， 统计单词数量

? 例子二? wordcount

[root@localhost hadoop-2.10.1]# mkdir wcinput
[root@localhost hadoop-2.10.1]# cd wcinput/
[root@localhost hadoop-2.10.1]# touch wc.input
[root@localhost hadoop-2.10.1]# cat wcinput/wc.input
hadoop mapreduce
yarn
haha hello
hello
[root@localhost hadoop-2.10.1]#./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.10.1.jar wordcount wcinput wcoutput
[root@localhost hadoop-2.10.1]# cat wcoutput/*
hadoop  1
haha    1
hello   2
mapreduce       1
yarn    1

?
伪分布式模式
全分布式模式