Question 1

Define the 'Veracity' characteristic of Big Data.

Accepted Answer

Veracity refers to the quality, accuracy, and trustworthiness of the data.

Question 2

What is a Data Lake?

Accepted Answer

A Data Lake is a centralized repository that stores vast amounts of raw data in its native format.

Question 3

What is the function of the Secondary NameNode?

Accepted Answer

The Secondary NameNode merges the HDFS namespace image (FSImage) with the edit logs to prevent the edit logs from becoming too large.

Question 4

What is a DataFrame in Spark?

Accepted Answer

A DataFrame is a distributed collection of data organized into named columns, similar to a table in a relational database.

Question 5

State the purpose of GraphX.

Accepted Answer

GraphX is Apache Spark's API for graphs and graph-parallel computation.

Question 6

What is Zookeeper used for?

Accepted Answer

Apache ZooKeeper is a centralized service for maintaining configuration information, naming, and providing distributed synchronization.

Question 7

What is Flume?

Accepted Answer

Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating, and moving large amounts of log data.

Question 8

Define supervised machine learning.

Accepted Answer

Supervised learning is an ML approach where models are trained using labeled datasets.

Question 9

What is a Document Store database?

Accepted Answer

A Document Store is a type of NoSQL database that stores data in JSON, BSON, or XML documents.

Question 10

Define TF-IDF.

Accepted Answer

TF-IDF (Term Frequency-Inverse Document Frequency) is a numerical statistic used to reflect the importance of a word in a document relative to a corpus.

Question 11

Explain the HDFS read and write process.

Accepted Answer

Reading and writing to HDFS involves interactions between the Client, NameNode, and DataNodes.

Question 12

Describe the 5 V's of Big Data.

Accepted Answer

The 5 V's define the core characteristics of big data: Volume, Velocity, Variety, Veracity, and Value.

Question 13

Walk through a MapReduce word count example.

Accepted Answer

The Word Count program splits text, maps words to a count of 1, and reduces by summing counts for each word.

Question 14

Distinguish between transformations and actions in Spark.

Accepted Answer

Transformations build a logical execution plan lazily, while actions trigger the actual execution of the plan.

Question 15

Explain the HBase architecture.

Accepted Answer

HBase is a distributed, column-oriented database built on top of HDFS, relying on HMaster and RegionServers.

Question 16

Describe the PageRank algorithm.

Accepted Answer

PageRank is a link analysis algorithm used to measure the relative importance of web pages within a graph.

Question 17

Explain the Machine learning lifecycle using Spark MLlib.

Accepted Answer

The Spark MLlib lifecycle involves data ingestion, feature engineering, model training, evaluation, and deployment via Pipelines.

Question 18

Explain HDFS architecture and its fault tolerance mechanisms.

Accepted Answer

HDFS ensures high availability and fault tolerance through replication, heartbeats, and secondary NameNodes.

Question 19

Detail the complete MapReduce execution flow.

Accepted Answer

The MapReduce execution flow encompasses input splitting, mapping, shuffling/sorting, reducing, and output generation.

Question 20

Classify NoSQL databases and explain the CAP Theorem in detail.

Accepted Answer

NoSQL databases fall into four categories, each making specific trade-offs according to the CAP Theorem.

Question 21

Describe Apache Mahout and its machine learning capabilities.

Accepted Answer

Apache Mahout is a scalable machine learning library designed to build scalable ML applications on top of Hadoop.

Question 22

Explain Streaming data processing with Spark Streaming.

Accepted Answer

Spark Streaming enables scalable, high-throughput, fault-tolerant processing of live data streams using micro-batching.

Big Data Analytics