Java/Web调用Hadoop进行MapReduce示例代码

Java/Web调用Hadoop进行MapReduce的完整攻略涉及以下步骤：

Java/Web调用Hadoop进行MapReduce的完整攻略涉及以下步骤：

1. 准备Hadoop集群
   在进行Java/Web调用Hadoop进行MapReduce前，首先需要准备好Hadoop集群环境。Hadoop集群环境的准备可以参考Hadoop官方文档或其他网络资料。

2. 编写MapReduce程序
   MapReduce是Hadoop中一种经典的计算框架，用于处理大规模数据。编写MapReduce程序需要实现Mapper和Reducer两个组件。下面是一个WordCount的示例程序：

public class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();

    public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
        String line = value.toString();
        StringTokenizer tokenizer = new StringTokenizer(line);
        while (tokenizer.hasMoreTokens()) {
            word.set(tokenizer.nextToken());
            context.write(word, one);
        }
    }
}

public class WordCountReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
        int sum = 0;
        for (IntWritable val : values) {
            sum += val.get();
        }
        context.write(key, new IntWritable(sum));
    }
}

1. 将MapReduce程序打包
   编写好MapReduce程序后，需要将其打包成一个可执行的Jar包。在Eclipse中可以使用Export功能来打包。

2. 通过Java程序执行MapReduce
   通过Java程序执行MapReduce需要使用Hadoop中提供的API，下面是一个简单的Java API调用示例：

public class WordCount {
    public static void main(String[] args) throws Exception {
        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf, "word count");
        job.setJarByClass(WordCount.class);
        job.setMapperClass(WordCountMapper.class);
        job.setReducerClass(WordCountReducer.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);
        FileInputFormat.addInputPath(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job, new Path(args[1]));
        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }
}

1. 通过Web程序执行MapReduce
   通过Web程序执行MapReduce需要将打包好的Jar包上传到Hadoop集群中，并调用Hadoop的API来执行MapReduce任务。下面是一个简单的Web调用MapReduce的示例代码：

public class WordCountServlet extends HttpServlet {
    public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        response.setContentType("text/html");
        PrintWriter out = response.getWriter();
        try {
            Configuration conf = new Configuration();
            Job job = Job.getInstance(conf, "word count");
            job.setJarByClass(WordCount.class);
            job.setMapperClass(WordCountMapper.class);
            job.setReducerClass(WordCountReducer.class);
            job.setOutputKeyClass(Text.class);
            job.setOutputValueClass(IntWritable.class);
            FileInputFormat.addInputPath(job, new Path("input"));
            FileOutputFormat.setOutputPath(job, new Path("output"));
            job.waitForCompletion(true);
            out.println("MapReduce finished successfully.");
        } catch (Exception e) {
            out.println("MapReduce failed with error message:" + e.getMessage());
        } finally {
            out.close();
        }
    }
}

以上示例中，第一个示例演示了如何通过Java程序调用MapReduce，将文本中的单词统计出现次数，并输出到指定路径。第二个示例演示了如何通过Web程序调用MapReduce，完成同样的任务，并通过Web页面来显示任务执行情况。