Hadoop Course:

Hardware Requirements: -- Systems must have at least 2 GB RAM.

Software Requirements: -- I will provide all software (Operating System also).

Contents:

Virtual box/VM Ware

1. Basics & Installations

Linux

1. Basics

Hadoop

1. What is Hadoop?
2. Why Hadoop and flow of Hadoop
3. Scaling
4. Distributed Framework
5. Hadoop v/s RDBMS
6. Brief history of Hadoop

Hadoop installation in pseudo mode

Hadoop installation in cluster mode

1. Adding and removing nodes (without down time)
2. Decommissioning nodes
3. Block size
4. Hadoop Processes ( NN, SNN, JT, DN, TT)

Common errors when running Hadoop cluster, solutions

HDFS- Hadoop distributed File System

1. HDFS Design and Architecture
2. HDFS Concepts
3. Interacting HDFS using command line
4. Dataflow
5. Introduction about Blocks
6. Data Replication
7. Admin Commands
8. Hadoop archives

Hadoop Processes

1. Name node and its functionality
2. Secondary name node and its functionality
3. Job tracker and its functionality
4. Task tracker and its functionality
5. Data node and its functionality
6. Resource manager and its functionality

7. Node manager and its functionality

Map Reduce

1. Developing Map Reduce Application
2. Phases in Map Reduce Framework
3. Map Reduce Input and Output Formats
4. Advanced Concepts
5. Combiner
6. HAR
7. Partitioner, sorting, shuffling
8. Different phases of MapReduce programs
9. Data localization
10. Different unstructured data processing examples
11. Image processing by using MapReduce

Joining datasets in MapReduce jobs

1. Map-side join
2. Reduce-Side join

Hadoop Programming Languages:-

PIG

1. Introduction (Basics)
2. Installation and Configuration
3. Different datatypes in PIG

4. Interacting HDFS using PIG
5. Map Reduce Programs through PIG
6. PIG Commands
7. Execution mechanisms (grunt, script...)
8. Loading, Filtering, Grouping, joins....
9. Sample programs in PIG with Real time

Hive

1. Basics (Introduction)
2. Installation and Configurations
3. Datatypes and operators
4. HQL Commands
5. Interacting HDFS using Hive
6. MapReduce programs through Hive
7. Joins, groups, filter......
8. Sample Programs in hive with real-time
9. Join vs Map Join

Impala

1. Basics
2. Commands

Sqoop

1. Introduction to sqoop
2. Installations & Configurations
3. Sqoop commands
4. Connect to relational database using sqoop and downloading lakhs of records to

Hadoop (in single minute)

Flume

1. Basics (Introduction)
2. Installation and Configurations

NOSQL Databases Concepts

1. Hbase
2. Basics & Installations
3. commands

III. Interacting Hbase with HDF

1. MongoDb

Basics & Installations

1. All queries for processing data

OOZIE Introduction

Zookeeper introduction

Apache Spark

1. Introduction
2. Installations and configurations
3. RDD , SC....
4. Scala Introduction
5. Interacting spark with HDFS
6. Programs in Spark through Scala

Specialties:--

ETL tool (Data Warehousing BI Tools):

PDI -

1. Introduction
2. Creating RDBMS database
3. Establishing Connection between PDI to RDMS database
4. Creating data in Hadoop
5. Establishing Connection between PDI to Hadoop data
6. Moving data from Hadoop to RDBMS and vice versa
7. Summarization

Highlights:

1. Working with Apache & cloudera Hadoop
2. Practical's on Hadoop cluster
3. Real life use cases
4. Will cover old version of Hadoop and latest version of Hadoop