输入数据123id 手机号 ip 网址 上行流量 下行流量 请求状态1 14530531269 102.132.54.44 www.ldbzo.ltd 1431 1360 4042 17690836832 88.58.44.150 ftp.j.icu 1135 718 302
FlowBean1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374package flowsum;import org.apache.hadoop.io.Writable;import java.io.DataInput;import java.io.DataOutput;import java.io.IOException;public class FlowBean implements Writable { ...
输入数据
订单id
商品id
商品价格
000000001
Pdt_01
222.8
000000002
Pdt_05
722.4
000000001
Pdt_02
33.8
000000003
Pdt_02
232.8
000000003
Pdt_02
33.8
000000002
Pdt_03
522.8
000000002
Pdt_04
122.4
需求求出每一个订单中最贵的商品。
OrderBean123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172package order;import org.apache.hadoop.io.WritableComparable;import java.io.DataInput;import java.io.DataOutput;import java.io.IOException;pu ...
输入数据123456789101112131415hadoop hello hadoopmapreduce hellospark hello scalahdfs java hadoopyarn hello javahadoop hello hadoopmapreduce hellospark2 hello scalahdfs java hadoopyarn2 hello javahadoop2 hello hadoopmapreduce hellospark4 hello scalahdfs java hadoopyarn hello java
KVTextMapper123456789101112131415package keyvaluetest;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Mapper;import java.io.IOException;public class KVTextMapper e ...
题目描述关于对于学生成绩相关的练习题,之前是一个入门级别的需求,现在对这些需求进行增强,首先看数据的改变:
1234567891011121314151617181920212223242526272829303132computer,huangxiaoming,85,86,41,75,93,42,85computer,xuzheng,54,52,86,91,42computer,huangbo,85,42,96,38english,zhaobenshan,54,52,86,91,42,85,75english,liuyifei,85,41,75,21,85,96,14algorithm,liuyifei,75,85,62,48,54,96,15computer,huangjiaju,85,75,86,85,85english,liuyifei,76,95,86,74,68,74,48english,huangdatou,48,58,67,86,15,33,85algorithm,huanglei,76,95,86,74,68,74,48algorithm,huangjiaju,85, ...
关于对于学生成绩相关的练习题,之前是一个入门级别的需求,现在对这些需求进行增强,首先看数据的改变:
1234567891011121314151617181920212223242526272829303132computer,huangxiaoming,85,86,41,75,93,42,85computer,xuzheng,54,52,86,91,42computer,huangbo,85,42,96,38english,zhaobenshan,54,52,86,91,42,85,75english,liuyifei,85,41,75,21,85,96,14algorithm,liuyifei,75,85,62,48,54,96,15computer,huangjiaju,85,75,86,85,85english,liuyifei,76,95,86,74,68,74,48english,huangdatou,48,58,67,86,15,33,85algorithm,huanglei,76,95,86,74,68,74,48algorithm,huangjiaju,85,75,8 ...
题目描述关于对于学生成绩相关的练习题,之前是一个入门级别的需求,现在对这些需求进行增强,首先看数据的改变:
1234567891011121314151617181920212223242526272829303132computer,huangxiaoming,85,86,41,75,93,42,85computer,xuzheng,54,52,86,91,42computer,huangbo,85,42,96,38english,zhaobenshan,54,52,86,91,42,85,75english,liuyifei,85,41,75,21,85,96,14algorithm,liuyifei,75,85,62,48,54,96,15computer,huangjiaju,85,75,86,85,85english,liuyifei,76,95,86,74,68,74,48english,huangdatou,48,58,67,86,15,33,85algorithm,huanglei,76,95,86,74,68,74,48algorithm,huangjiaju,85, ...
需求2:求该成绩表每门课程当中出现了相同分数的分数,还有次数,以及该分数的人数
返回结果的格式:科目 分数 次数 该分数的人例子:computer 85 3 huangzitao,liujialing,huangxiaoming
解题思路:
对于mapper阶段,输出的key-value分别是:
key: 课程,分数
value: 名字
对于reducer阶段,reduce方法接收的参数是:
key: 课程,分数
values: 课程中的某个分数的多个学生的名字的迭代器
看代码实现:
1234567891011121314151617181920package com.xiaojia.stu2;import org.apache.hadoop.io.LongWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Mapper;import java.io.IOException;public class StuMapper extends Mapper<Lon ...
1234567891011121314151617181920212223242526272829303132computer,huangxiaoming,85computer,xuzheng,54computer,huangbo,86computer,liutao,85computer,huanglei,99computer,liujialing,85computer,liuyifei,75computer,huangdatou,48computer,huangjiaju,88computer,huangzitao,85english,zhaobenshan,57english,liuyifei,85english,liuyifei,76english,huangdatou,48english,zhouqi,85english,huangbo,85english,huangxiaoming,96english,huanglei,85english,liujialing,75algorithm,liuyifei,75algorithm,huanglei,76algorithm,huan ...