windows部署Hadoop简要过程

0. 把hadoop-common-2.2.0-bin-master.zip里面的7个文件拷贝到hadoop\bin目录下
hadoop.dll, hadoop.exp hadoop.lib hadoop.pdb libwinutils.lib winutils.exe winutils.pdb
1. hadoop\etc\hadoop\hadoop-env.cmd 最后增加：

set JAVA_HOME=%JAVA_HOME%
set HADOOP_HOME=%~dp0..\..\
set HADOOP_PREFIX=%HADOOP_HOME%
set HADOOP_CONF_DIR=%HADOOP_HOME%\etc\hadoop
set YARN_CONF_DIR=%HADOOP_HOME%\etc\hadoop
set PATH=%PATH%;%HADOOP_HOME%\bin

2. hadoop\etc\hadoop\yarn-site.xml, <configuration>里面增加：

<property>
<name>yarn.server.resourcemanager.address</name>
<value>0.0.0.0:8020</value>
</property>
<property>
<name>yarn.server.resourcemanager.application.expiry.interval</name>
<value>60000</value>
</property>
<property>
<name>yarn.server.nodemanager.address</name>
<value>0.0.0.0:45454</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>0.0.0.0:8088</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.server.nodemanager.remote-app-log-dir</name>
<value>/app-logs</value>
</property>
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>/dep/logs/userlogs</value>
</property>
<property>
<name>yarn.server.mapreduce-appmanager.attempt-listener.bindAddress</name>
<value>0.0.0.0</value>
</property>
<property>
<name>yarn.server.mapreduce-appmanager.client-service.bindAddress</name>
<value>0.0.0.0</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
<property>
<name>yarn.log-aggregation.retain-seconds</name>
<value>-1</value>
</property>
<property>
<name>yarn.application.classpath</name>
<value>%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*</value>
</property>

3. hadoop\etc\hadoop\mapred-site.xml, <configuration>里面增加，并修改“Windows登录名”

<property>
<name>mapreduce.job.user.name</name>
<value>Windows登录名</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>yarn.apps.stagingDir</name>
<value>/user/Windows登录名/staging</value>
</property>
<property>
<name>mapreduce.jobtracker.address</name>
<value>local</value>
</property>

4. hadoop\etc\hadoop\core-site.xml <configuration>里面增加：

<property>
<name>fs.default.name</name>
<value>hdfs://0.0.0.0:19000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/hadooptmpdir</value>
<description>A base for other temporary directories.</description>
</property>

文件在 当前磁盘根目录 /hadooptmpdir 目录下。

5. hadoop\etc\hadoop\hdfs-site.xml <configuration>里面增加：

<property>
<name>dfs.replication</name>
<value>1</value>
</property>

格式化HDFS文件系统：
（1）hadoop\etc\hadoop\hadoop-env.cmd
（2）hadoop\bin\hdfs namenode -format

6. 启动

hadoop\etc\hadoop\hadoop-env.cmd

hadoop\sbin\start-dfs.cmd