Q13Big Data Analytics
Question
4 marks
Walk through a MapReduce word count example.
Answer
The Word Count program splits text, maps words to a count of 1, and reduces by summing counts for each word.
1. Input Split: A text file is split into chunks.
2. Map Phase: Each chunk is processed line by line. For each word found, a key-value pair (word, 1) is emitted.
3. Shuffle and Sort: The framework groups all values associated with the same key (word). For example, ('data', [1, 1, 1]).
4. Reduce Phase: The reducer iterates over the values list, summing them up to output the final count ('data', 3).