问题
Flink 提交作业,直接报错:
java.lang.NoSuchMethodError: org.apache.hadoop.tracing.TraceUtils.wrapHadoopConf(Ljava/lang/String;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/HTraceConfiguration;
at org.apache.hadoop.fs.FsTracer.get(FsTracer.java:42)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:689)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:673)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:155)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at org.apache.hadoop.fs.viewfs.ChRootedFileSystem.<init>(ChRootedFileSystem.java:103)
at org.apache.hadoop.fs.viewfs.ViewFileSystem$1.getTargetFileSystem(ViewFileSystem.java:173)
at org.apache.hadoop.fs.viewfs.ViewFileSystem$1.getTargetFileSystem(ViewFileSystem.java:167)
at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:261)
at org.apache.hadoop.fs.viewfs.InodeTree.<init>(InodeTree.java:333)
at org.apache.hadoop.fs.viewfs.ViewFileSystem$1.<init>(ViewFileSystem.java:167)
at org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:172)
at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:770)
at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:593)
at org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:458)
at org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:67)
at org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:213)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1057)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
分析
查看2.7.x 的源码如下:
import org.apache.htrace.HTraceConfiguration;
/**
* This class provides utility functions for tracing.
*/
@InterfaceAudience.Private
public class TraceUtils {
private static List<ConfigurationPair> EMPTY = Collections.emptyList();
public static HTraceConfiguration wrapHadoopConf(final String prefix,
final Configuration conf) {
return wrapHadoopConf(prefix, conf, EMPTY);
}
发现报错的是
不仔细看,还真看不出来不是方法没有,是返回值的包不对。
NoSuchMethod | TraceUtils实际的Method |
---|---|
public static HTraceConfiguration wrapHadoopConf(final String prefix, final Configuration conf) | public static HTraceConfiguration wrapHadoopConf(final String prefix, final Configuration conf) |
Lorg/apache/htrace/core/HTraceConfiguration; | 返回值 org.apache.htrace.HTraceConfiguration |
hadoop 的发型版差异:
- Apache Hadoop Common:2.6.x 没有TraceUtils这个类,2.7.x 才有。
- Apache Hadoop Common: 2.6.0-CDH5.12.1 有这个类。
查看 2.6.0-CDH5.12.1的 TraceUtils
import org.apache.htrace.core.HTraceConfiguration;
查看 2.7.x 的TraceUtils
import org.apache.htrace.HTraceConfiguration;
所以错误发生在,2.6.0-cdh5.12.1的FsTracer通过hadoop-common 加载TraceUtils,而实际上加载了2.7.x的TraceUtils。
解决
- 方案1:删除2.7.x的 hadoop-common,或者shade了这个包的library。
- 方案2:增加2.7.x的 hadoop-hdfs,它会加载相应的 2.7.x的hadoop-common,或者shade了这个包的 library。
- NoSuchMethodError wrapHadoopConf TraceUtils tracing apachenosuchmethoderror wraphadoopconf traceutils traceutils wraphadoopconf nosuchmethoderror nosuchmethoderror addservletmapping container nosuchmethoderror usermodel cellstyl caused ljava lang nosuchmethoderror quot nosuchmethoderror lang charsequence stringutils ljava nosuchmethoderror lang atomicreference nosuchmethoderror internal platform okhttp3