微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

提交example案例到YARN上运行

0、使用官方例子,将MapReduce提交到YARN上运行

1、进入文件

[hadoop@hadoop000 mapreduce]$ pwd
/home/hadoop/app/hadoop-2.6.0-cdh5.15.1/share/hadoop/mapreduce
[hadoop@hadoop000 mapreduce]$ ls
hadoop-mapreduce-client-app-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-common-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-core-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-hs-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-hs-plugins-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-jobclient-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-jobclient-2.6.0-cdh5.15.1-tests.jar
hadoop-mapreduce-client-nativetask-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-client-shuffle-2.6.0-cdh5.15.1.jar
hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar

2、运行jar包里的类

(1)提示:计算pi值

pi: A map/reduce program that estimates Pi using a quasi-Monte Carlo method.

(2)提示:使用方法

Usage: org.apache.hadoop.examples.QuasiMonteCarlo <nMaps> <nSamples>

(3)运行运行jar包里的类

[hadoop@hadoop000 mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar pi 2 3

(4)刷新浏览器界面All Applications

3、wordcount

(1)找到wordcount

[hadoop@hadoop000 mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar

(2)使用vi重新创建文件

[hadoop@hadoop000 data]$ pwd
/home/hadoop/data
[hadoop@hadoop000 data]$ vi data.txt

hello   world   welcome
Hello   hello   welcome

(3)在hadoop中多层创建input文件,并传入文件

[hadoop@hadoop000 data]$ hadoop fs -mkdir -p /wc/input

[hadoop@hadoop000 data]$ hadoop fs -put data.txt /wc/input

(4)执行

[hadoop@hadoop000 mapreduce]$ hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.15.1.jar wordcount /wc/input/data.txt /wc/output

(5)查看结果

没有区分大小写,也不能自动删除重复性文档。

[hadoop@hadoop000 mapreduce]$ hadoop fs -text /wc/output/part-r-00000

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 [email protected] 举报,一经查实,本站将立刻删除。

相关推荐