526互联

使用sqoop一直卡在：mapreduce.Job: Running job: job_1703173956074_0001

发布时间 2023-12-21 16:38:58作者: DogLeftover
  • 修改hadoop配置
vi /usr/local/software/hadoop-2.9.2/etc/hadoop/yarn-site.xml
# 添加如下
<configuration>
   <property>
       <name>yarn.nodemanager.resource.memory-mb</name>
       <value>20480</value>
   </property>
   <property>
       <name>yarn.scheduler.minimum-allocation-mb</name>
       <value>2048</value>
   </property>
   <property>
       <name>yarn.nodemanager.vmem-pmem-ratio</name>
       <value>2.1</value>
   </property>
</configuration>
  • 检查配置是否正确
vi /usr/local/software/hadoop-2.9.2/etc/hadoop/yarn-site.xml

# 将mapreduce\_shuffle改为mapreduce_shuffle
<configuration>
   <property>
       <name>yarn.nodemanager.aux-services</name>
       <value>mapreduce\_shuffle</value>
   </property>
</configuration>
  • VMware中修改内存大小，改大一点，改为4GB