Q21Cloud Computing
Question
Discuss Cloud Data Storage. Explain the Google File System (GFS) architecture in detail.
Answer
Google File System (GFS) is a scalable distributed file system designed to handle large-scale data processing workloads on commodity hardware.
The Google File System (GFS) architecture was designed to meet the rapidly growing demands of Google's data processing needs. It consists of a single Master node and multiple Chunkservers.
1. Files and Chunks: Files are divided into fixed-size chunks (typically 64 MB). Each chunk is identified by an immutable and globally unique 64-bit chunk handle assigned by the Master at the time of chunk creation.
2. GFS Master: The single master maintains all file system metadata. This includes the namespace, access control information, the mapping from files to chunks, and the current locations of chunks. It periodically communicates with each Chunkserver in HeartBeat messages to give it instructions and collect its state.
3. Chunkservers: These store the chunks on local disks as standard Linux files and read/write chunk data specified by a chunk handle and byte range. For reliability, each chunk is replicated on multiple Chunkservers (default is 3 replicas).
Data flows directly from Chunkservers to the client, while the Master only provides metadata, preventing the Master from becoming a bottleneck.