Monday, February 01, 2010
Hadoop Combiners and Map-Reduce
According to the Hadoop documentation, a Hadoop Combiner can be used to speedup Hadoop Map-Reduce if the Reduce function is both commutative and associative. It would be interesting to write a program that introspects a Hadoop Reduce function to draw a conclusion regarding the validity of this constraint for the use of Hadoop Combiner but it will not be trivial.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment