Maven编译hadoop-2.2.0-src

175次阅读

共计 6864 个字符，预计需要花费 18 分钟才能阅读完成。

Maven 编译 Hadoop-2.2.0-src

1. 环境

Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)
Maven home: /opt/maven
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home: /usr/java/jdk1.7.0_45/jre
Default locale: en_US, platform encoding: UTF-8
OS name: “linux”, version: “2.6.18-371.3.1.el5”, arch: “amd64”, family: “unix”

2. 执行

mvn package -DskipTests -Pdist,native,docs -Dtar

3. 插曲 1

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
[ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle
[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found

打补丁

Patch :https://issues.apache.org/jira/browse/HADOOP-10110

方法：修改 hadoop 源码目录下的 hadoop-common-project/hadoop-auth/pom.xml

添加或修改如下部分内容：

<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-util</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<scope>test</scope>
</dependency>

4. 插曲 2

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program “cmake” (in directory “/opt/hadoop-2.2.0-src/hadoop-common-project/hadoop-common/target/native”): error=2, No such file or directory -> [Help 1]

没装 cmake

人家就是想要这个地方这可不行果断

yum install cmake

5. 插曲 3

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: ‘protoc –version’ did not return a version -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: ‘protoc –version’ did not return a version

没装 protobuf-2.5.0。

下载 protobuf-2.5.0.tar.gz 解压 3 部曲编译安装。

6. 继续 (这次学能了，带着 debug)

mvn package -Pdist,native -DskipTests -Dtar -e -X

等了好久好久

[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ………………………….. SUCCESS [2:18.165s]
[INFO] Apache Hadoop Project POM ……………………. SUCCESS [50.526s]
[INFO] Apache Hadoop Annotations ……………………. SUCCESS [1:14.522s]
[INFO] Apache Hadoop Assemblies …………………….. SUCCESS [0.760s]
[INFO] Apache Hadoop Project Dist POM ……………….. SUCCESS [5.983s]
[INFO] Apache Hadoop Maven Plugins ………………….. SUCCESS [13.161s]
[INFO] Apache Hadoop Auth ………………………….. SUCCESS [10.647s]
[INFO] Apache Hadoop Auth Examples ………………….. SUCCESS [5.010s]
[INFO] Apache Hadoop Common ………………………… SUCCESS [4:32.249s]
[INFO] Apache Hadoop NFS …………………………… SUCCESS [18.927s]
[INFO] Apache Hadoop Common Project …………………. SUCCESS [0.134s]
[INFO] Apache Hadoop HDFS ………………………….. SUCCESS [5:29.380s]
[INFO] Apache Hadoop HttpFS ………………………… SUCCESS [1:43.219s]
[INFO] Apache Hadoop HDFS BookKeeper Journal …………. SUCCESS [44.693s]
[INFO] Apache Hadoop HDFS-NFS ………………………. SUCCESS [13.217s]
[INFO] Apache Hadoop HDFS Project …………………… SUCCESS [0.219s]
[INFO] hadoop-yarn ………………………………… SUCCESS [1.957s]
[INFO] hadoop-yarn-api …………………………….. SUCCESS [1:36.233s]
[INFO] hadoop-yarn-common ………………………….. SUCCESS [1:31.987s]
[INFO] hadoop-yarn-server ………………………….. SUCCESS [0.334s]
[INFO] hadoop-yarn-server-common ……………………. SUCCESS [26.932s]
[INFO] hadoop-yarn-server-nodemanager ……………….. SUCCESS [49.536s]
[INFO] hadoop-yarn-server-web-proxy …………………. SUCCESS [7.882s]
[INFO] hadoop-yarn-server-resourcemanager ……………. SUCCESS [1:01.936s]
[INFO] hadoop-yarn-server-tests …………………….. SUCCESS [6.642s]
[INFO] hadoop-yarn-client ………………………….. SUCCESS [11.930s]
[INFO] hadoop-yarn-applications …………………….. SUCCESS [0.437s]
[INFO] hadoop-yarn-applications-distributedshell ……… SUCCESS [12.064s]
[INFO] hadoop-mapreduce-client ……………………… SUCCESS [1.363s]
[INFO] hadoop-mapreduce-client-core …………………. SUCCESS [58.193s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher …. SUCCESS [11.262s]
[INFO] hadoop-yarn-site ……………………………. SUCCESS [0.328s]
[INFO] hadoop-yarn-project …………………………. SUCCESS [17.453s]
[INFO] hadoop-mapreduce-client-common ……………….. SUCCESS [43.479s]
[INFO] hadoop-mapreduce-client-shuffle ………………. SUCCESS [6.048s]
[INFO] hadoop-mapreduce-client-app ………………….. SUCCESS [46.449s]
[INFO] hadoop-mapreduce-client-hs …………………… SUCCESS [9.770s]
[INFO] hadoop-mapreduce-client-jobclient …………….. SUCCESS [17.249s]
[INFO] hadoop-mapreduce-client-hs-plugins ……………. SUCCESS [9.929s]
[INFO] Apache Hadoop MapReduce Examples ……………… SUCCESS [12.193s]
[INFO] hadoop-mapreduce ……………………………. SUCCESS [6.817s]
[INFO] Apache Hadoop MapReduce Streaming …………….. SUCCESS [10.957s]
[INFO] Apache Hadoop Distributed Copy ……………….. SUCCESS [32.407s]
[INFO] Apache Hadoop Archives ………………………. SUCCESS [4.950s]
[INFO] Apache Hadoop Rumen …………………………. SUCCESS [18.817s]
[INFO] Apache Hadoop Gridmix ……………………….. SUCCESS [10.909s]
[INFO] Apache Hadoop Data Join ……………………… SUCCESS [6.662s]
[INFO] Apache Hadoop Extras ………………………… SUCCESS [11.594s]
[INFO] Apache Hadoop Pipes …………………………. SUCCESS [3.737s]
[INFO] Apache Hadoop Tools Dist …………………….. SUCCESS [3.623s]
[INFO] Apache Hadoop Tools …………………………. SUCCESS [0.127s]
[INFO] Apache Hadoop Distribution …………………… SUCCESS [2:55.243s]
[INFO] Apache Hadoop Client ………………………… SUCCESS [25.929s]
[INFO] Apache Hadoop Mini-Cluster …………………… SUCCESS [2.516s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 33:38.762s
[INFO] Finished at: Sat Dec 28 21:59:12 CST 2013
[INFO] Final Memory: 68M/200M
[INFO] ————————————————————————