Q11Big Data Analytics
Question
4 marks
Explain the HDFS read and write process.
Answer
Reading and writing to HDFS involves interactions between the Client, NameNode, and DataNodes.
For a Write: The client asks the NameNode for permission and a list of DataNodes to store the blocks. The client then writes data directly to the first DataNode in a pipeline. The first DataNode replicates it to the second, which replicates to the third. For a Read: The client queries the NameNode for the block locations of a file. The NameNode returns a list of DataNodes sorted by their proximity to the client. The client then reads directly from the closest DataNode.