Map Reduce explained with Example

The Map-Reduce framework lets us make programs that can run huge amounts of data at the same time on a large cluster of standard hardware.

Different Phases of Map Reduce:-

Map Reduce model has three major and one optional phase.​

• Mapping

• Shuffling and Sorting

• Reducing

• Combining

Mapping: It is the initial phase of Map Reduce programming. Key-value pairs (k, v) are accepted as input by the Mapping Phase, where the key denotes the key address of each record and the value denotes the entire record content. The output of the mapping phase will also be in the key-value format (k’, v’).

Shuffling and Sorting :- The output of various mapping parts (k’, v’), then goes into Shuffling and Sorting phase.​ All the same values are deleted, and different values are grouped together based on same keys.​ The output of the Shuffling and Sorting phase will be key-value pairs, again as key and array of values (k, v[ ]).

Reducer :- The output of the Shuffling and Sorting phase (k, v[]) will be the input of the Reducer phase.​ In this phase reducer function’s logic is executed and all the values are collected against their corresponding keys. ​Reducer stabilize outputs of various mappers and computes the final output.​

Combining :- It is an optional phase in the Map Reduce phases. The combiner phase is used to optimize the performance of Map Reduce phases. This phase makes the Shuffling and Sorting phase work even quicker by enabling additional performance features in Map Reduce phases.

Let us consider an example of usage of Map Reduce, where a given problem is to count the number of occurrence of words in a text file. First, the file is divided into chunks & then Map Reduce is applied.

Thanks for reading. Next, I will write about Cloud Security.