java - sqoop 与 hadoop 集成抛出 ClassNotFoundException

Question

我是 hadoop 和 sqoop 的新手。我安装了 hadoop 2.7.3(伪模式) 并且它在我的系统上运行良好。

我想与 sqoop 集成。我正在使用 sqoop sqoop-1.99.7-bin-hadoop200。

1) 我提取 tar 文件并将提取的内容移动到 /usr/local/sqoop

2) 将Sqoop路径设置到.bashrc文件中。

3)进入/usr/local/sqoop/server/lib/sqoop.sh服务器启动

并收到以下错误消息。

hadoop_usr@sawai-Lenovo-G580:/usr/local/sqoop/server/lib$ sqoop.sh server start
Setting conf dir: /usr/local/sqoop/conf
Sqoop home directory: /usr/local/sqoop
Starting the Sqoop2 server...
0    [main] INFO  org.apache.sqoop.core.SqoopServer  - Initializing Sqoop server.
34   [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration
    at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38)
    at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36)
    at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:121)
    at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:237)
    at org.apache.sqoop.security.authentication.SimpleAuthenticationHandler.secureLogin(SimpleAuthenticationHandler.java:39)
    at org.apache.sqoop.security.AuthenticationManager.initialize(AuthenticationManager.java:98)
    at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)
    at org.apache.sqoop.server.SqoopJettyServer.<init>(SqoopJettyServer.java:67)
    at org.apache.sqoop.server.SqoopJettyServer.main(SqoopJettyServer.java:177)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.Configuration
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 9 more

4)所以当我在lib中寻找jar时我发现了。以下 jar 列表。

请你告诉我现在缺少哪个 jar 。所以我能够运行 sqoop 并开始我的学习。

谢谢。

Answer 1

我也遇到了同样的问题

为了解决这个问题，我在单独的终端中设置了以下变量并且它有效

export HADOOP_MAPRED_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/mapreduce
export HADOOP_COMMON_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/common
export HADOOP_HDFS_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/hdfs
export YARN_HOME=/opt/hadoop/hadoop-3.1.2/share/hadoop/yarn