UrbanPro
true

Learn Big Data from the Best Tutors

  • Affordable fees
  • 1-1 or Group class
  • Flexible Timings
  • Verified Tutors

Search in

BigDATA HADOOP Infrastructure & Services: Basic Concept

Sayan Goswami
14/04/2017 0 0

Hadoop Cluster & Processes

What is Hadoop Cluster?

Hadoop cluster is  the collections of one or more than one Linux Boxes. In a Hadoop cluster there should be a single Master(Linux machine/box) machine and all the rest of machines  are called Slaves(Linux machines/boxes).

Hadoop Modes to start up :

  1. Single Mode
  2. Pseudo distributed Mode
  3. Fully distributed Mode (it should minimum 3 Linux box)

Single Mode actually acts as a single Master box and it generally used for testing purpose(mainly black box testing).

Fully distributed Mode requires minimum 3 machines/boxes. It is actually the Production mode of Hadoop cluster.

But for R&D(Research & Development) purpose Apache Software Org. has introduced another useful mode i.e Pseudo distributed Mode. In this mode we can achieve all the core functionalities of Fully distributed cluster. That means in this mode one single machine acts as Master & as well as Slave recursively. We will configure our own Hadoop cluster in this mode.

Hadoop Processes & Layers :

Hadoop has 5  different processes for different functionalities.

  1. NameNode.
  2. Secondary NameNode.
  3. DataNode.
  4. JobTracker.
  5. TaskTracker.

Also Hadoop handle all the storage & analytics part in between 2 layers. These are-

  1. HDFS Layer

NameNode

Secondary NameNode

DataNode

  1. Application / MapReduce Layer

JobTracker

TaskTracker

Note: NameNode is called the single point of failure in Hadoop cluster.Why??? Because NameNode is the highest priority than JobTracker. Simply without data what’s the value of Application.

Hadoop Process orientation in a Cluster:

   Brief functionalities of the Hadoop processes-

  1. NameNode(NN)- NN holds the total HDFS(Hadoop Distributed File System) . That means all the HDFS metadata (that send as report by each DataNode) is stored in NameNode(NN).
  2. Secondary NameNode(SNN)- This is process is actually kind of a housekeeper of NameNode(NN). All the activities which generated by NameNode, SNN captured those as snapshot.
  3. DataNode(DN)- DN is responsible for data writing in HDFS. Also DN periodically sends block report to NameNode(NN)
  4. JobTracker(JT)- Each MapReduce job distributed by this process i.e JobTracker. JT distributes the job by parallel to all of the existing TaskTracker’s(TT) with the help of NN.
  5. TaskTracker(TT)- This process is actually executing the Job’s which are distributed by JT.

We can’t edit/modify the data in HDFS. The rule is “Write Once Read Many Times”. Yes we can append the new data but can’t edit.

0 Dislike
Follow 0

Please Enter a comment

Submit

Other Lessons for You

Big data Training Catalogue
Course: 1 Understanding Fundamentals of Big Data Duration: 1 Day Level: Basic Fundamentals of Big Data Understanding Big Data Big Data Drivers Big Data Use cases Understanding Big Data Dimensions Characteristics...
X

Xcelframeworks

0 0
0

REFERENCE BOOKS FOR DATA SCIENCE
Dear All, You can use the following books to master the DATA SCIENCE Concepts 1) First Course in Probability-Ronald Russel 2)Applied Regression Analysis-Drapper and Smith 3)Applied Multivariate Analysis-Richard...

Big Data & Hadoop - Introductory Session - Data Science for Everyone
Data Science for Everyone An introductory video lesson on Big Data, the need, necessity, evolution and contributing factors. This is presented by Skill Sigma as part of the "Data Science for Everyone" series.

HDFS Commands - Data Engineering
HDFS commands : HDFS commands will interact with namenode to show results some commands like cat ,tail will interact with datanode to show results HDFS help commands: hadoop fs -help ls list commands...

Understanding Big Data
Introduction to Big Data This blog is about Big Data, its meaning, and applications prevalent currently in the industry.It’s an accepted fact that Big Data has taken the world by storm and has become...
M

MyMirror

0 0
0
X

Looking for Big Data Classes?

The best tutors for Big Data Classes are on UrbanPro

  • Select the best Tutor
  • Book & Attend a Free Demo
  • Pay and start Learning

Learn Big Data with the Best Tutors

The best Tutors for Big Data Classes are on UrbanPro

This website uses cookies

We use cookies to improve user experience. Choose what cookies you allow us to use. You can read more about our Cookie Policy in our Privacy Policy

Accept All
Decline All

UrbanPro.com is India's largest network of most trusted tutors and institutes. Over 55 lakh students rely on UrbanPro.com, to fulfill their learning requirements across 1,000+ categories. Using UrbanPro.com, parents, and students can compare multiple Tutors and Institutes and choose the one that best suits their requirements. More than 7.5 lakh verified Tutors and Institutes are helping millions of students every day and growing their tutoring business on UrbanPro.com. Whether you are looking for a tutor to learn mathematics, a German language trainer to brush up your German language skills or an institute to upgrade your IT skills, we have got the best selection of Tutors and Training Institutes for you. Read more