window10安装单机hadoop

发布时间 2023-10-30 14:30:16作者: GGGSD

一、配置jdk

默认已配置

二、下载hadoop3.2.2

http://archive.apache.org/dist/hadoop/common/hadoop-3.2.2/

三、下载hadoop-winutils

https://github.com/cdarlint/winutils
下载后将bin中的文件全部放到hadoop/bin中

四、配置hadoop

1、进入hadoop安装文件夹下的etc/hadoop文件夹,修改以下文件内容:

(1)core-site.xml

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
    </property>
    <property>
    	<name>hadoop.tmp.dir</name>
		<value>/D:/soft/hadoop-3.2.2/tmp</value>
  	</property>
</configuration>

(2)hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
	 <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/D:/soft/hadoop-3.2.2/data/dfs/namenode</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/D:/soft/hadoop-3.2.2/data/dfs/datanode</value>
    </property>
	  <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>
</configuration>

(3)mapred-site.xml

<configuration>
   <property>
      <name>mapreduce.framework.name</name>
      <value>yarn</value>
    </property>

</configuration>

(4)yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
<property>
     <name>yarn.nodemanager.aux-services</name>
     <value>mapreduce_shuffle</value>
   </property>
  <property>
     <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
     <value>org.apache.hadoop.mapred.ShuffleHandler</value>
   </property>

</configuration>

五、格式化hdfs

进入cmd控制台输入:

hdfs namenode -format

六、启动

进入hadoop/sbin文件夹,双击start-all.cmd,出现四个弹窗

没有报错,即为成功

七、查看ui页面

http://localhost:8088

http://localhost:9870/

测试上传tif成功

八、停止hadoop
进入hadoop/sbin文件夹,双击stop-all.cmd,四个弹窗关闭。
完毕!!!