python - 我们可以使用hadoop运行现有程序还是只需要以mapreduce样式对其进行修改？

Question

我写了下面的python程序来处理excel文件中的数据。但是现在可以使用hadoop mapreduce运行相同的程序了，传统程序和mapreduce程序有何不同？

import xlrd

with xlrd.open_workbook('interference.xlsx') as book:
# 0 corresponds for 1st worksheet, usually named 'Book1'
sheet = book.sheet_by_index(0)

# gets col C values
B = [ B for B in sheet.col_values(1) ]


# gets col D values
D = [ D for D in sheet.col_values(3) ]


# combines D and E elements to tuples, combines tuples to list
# ex. [ ('Incoming', 18), ('Outgoing', 99), ... ]
data = list( zip(B,D) )

# gets total no of get request attempts for each UID

x=1
for uid in data:
    while x <=44 :
        attempts = sum( tup[1] for tup in data if tup[0] == x )
        print("Total attempts for UID",x, attempts)
        x=x+1

Answer 1

与Hadoop一起运行与MapReduce作业相同的程序是不可能的。

MapReduce是一种编程范例，其思想是将计算分为两个阶段:第一个(映射)将问题分为许多子问题并解决每个子问题；第二阶段(归约)将所有子问题的结果放在一起以获得最终解决方案。

我建议您看一下WordCount程序，它与Hello World的Hadoop等效:http://wiki.apache.org/hadoop/WordCount