Big Data Hadoop

Big Data & Analytics

Course Description

Hadoop is an Apache project (i.e. an open source software) to store & process Big Data. Hadoop stores Big Data in a distributed & fault tolerant manner over commodity hardware. Afterwards, Hadoop tools are used to perform parallel data processing over HDFS (Hadoop Distributed File System).

As organisations have realized the benefits of Big Data Analytics, so there is a huge demand for Big Data & Hadoop professionals. Companies are looking for Big data & Hadoop experts with the knowledge of Hadoop Ecosystem and best practices about HDFS, MapReduce, Spark, HBase, Hive, Pig, Oozie, Sqoop & Flume.

32 Hours

  • In-depth knowledge of Big Data and Hadoop including HDFS (Hadoop Distributed File System), YARN (Yet Another Resource Negotiator) & MapReduce
  • Comprehensive knowledge of various tools that fall in Hadoop Ecosystem like Pig, Hive, Sqoop, Flume, Oozie, and HBase
  • The capability to ingest data in HDFS using Sqoop & Flume, and analyze those large datasets stored in the HDFS
  • The exposure to many real world industry-based projects which will be executed in Edureka’s CloudLab
  • Projects which are diverse in nature covering various data sets from multiple domains such as banking, telecommunication, social media, insurance, and e-commerce
  • Rigorous involvement of a Hadoop expert throughout the Big Data Hadoop Training to learn industry standards and best practices
  • Software Developers, Project Managers
  • Software Architects
  • ETL and Data Warehousing Professionals
  • Data Engineers
  • Data Analysts & Business Intelligence Professionals
  • DBAs and DB professionals
  • Senior IT Professionals
  • Testing professionals
  • Mainframe professionals
  • Graduates looking to build a career in Big Data Field

There are no such prerequisites for Big Data & Hadoop Course. However, prior knowledge of Core Java and SQL will be helpful but is not mandatory.

Understanding Big Data & Hadoop

  • Introduction to Big Data & Big Data Challenges
  • Limitations & Solutions of Big Data Architecture
  • Hadoop & its Features
  • Hadoop Ecosystem
  • Hadoop 2.x Core Components
  • Hadoop Storage: HDFS (Hadoop Distributed File System)
  • Hadoop Processing: MapReduce Framework
  • Different Hadoop Distributions

Hadoop Architecture & HDFS

  • Hadoop 2.x Cluster Architecture
  • Federation and High Availability Architecture
  • Typical Production Hadoop Cluster
  • Hadoop Cluster Modes
  • Common Hadoop Shell Commands
  • Hadoop 2.x Configuration Files
  • Single Node Cluster & Multi-Node Cluster set up
  • Basic Hadoop Administration

Hadoop MapReduce Framework

  • Traditional way vs MapReduce way
  • Why MapReduce
  • YARN Components
  • YARN Architecture
  • YARN MapReduce Application Execution Flow
  • YARN Workflow
  • Anatomy of MapReduce Program
  • Input Splits, Relation between Input Splits and HDFS Blocks
  • MapReduce: Combiner & Partitioner
  • Demo of Health Care Dataset
  • Demo of Weather Dataset

Advance Hadoop MapReduce

  • Counters
  • Distributed Cache
  • MRunit
  • Reduce Join
  • Custom Input Format
  • Sequence Input Format
  • XML file Parsing using MapReduce

Apache Pig

  • Introduction to Apache Pig
  • MapReduce vs Pig
  • Pig Components & Pig Execution
  • Pig Data Types & Data Models in Pig
  • Pig Latin Programs
  • Shell and Utility Commands
  • Pig UDF & Pig Streaming
  • Testing Pig scripts with Punit
  • Aviation use-case in PIG
  • Pig Demo of Healthcare Dataset

Apache Hive

  • Introduction to Apache Hive
  • Hive vs Pig
  • Hive Architecture and Components
  • Hive Metastore
  • Limitations of Hive
  • Comparison with Traditional Database
  • Hive Data Types and Data Models
  • Hive Partition
  • Hive Bucketing
  • Hive Tables (Managed Tables and External Tables)
  • Importing Data
  • Querying Data & Managing Outputs
  • Hive Script & Hive UDF
  • Retail use case in Hive
  • Hive Demo on Healthcare Dataset

Advance Apache Hive & HBase

  • Hive QL: Joining Tables, Dynamic Partitioning
  • Custom MapReduce Scripts
  • Hive Indexes and views
  • Hive Query Optimizers
  • Hive Thrift Server
  • Hive UDF
  • Apache HBase: Introduction to NoSQL Databases and HBase
  • HBase v/s RDBMS
  • HBase Components
  • HBase Architecture
  • HBase Run Modes
  • HBase Configuration
  • HBase Cluster Deployment

Advance Apache HBase

  • HBase Data Model
  • HBase Shell
  • HBase Client API
  • Hive Data Loading Techniques
  • Apache Zookeeper Introduction
  • ZooKeeper Data Model
  • Zookeeper Service
  • HBase Bulk Loading
  • Getting and Inserting Data
  • HBase Filters

Processing Distributed Data with Apache Spark

  • What is Spark
  • Spark Ecosystem
  • Spark Components
  • What is Scala
  • Why Scala
  • SparkContext
  • Spark RDD

Oozie and Hadoop Project

  • Oozie
  • Oozie Components
  • Oozie Workflow
  • Scheduling Jobs with Oozie Scheduler
  • Demo of Oozie Workflow
  • Oozie Coordinator
  • Oozie Commands
  • Oozie Web Console
  • Oozie for MapReduce
  • Combining flow of MapReduce Jobs
  • Hive in Oozie
  • Hadoop Project Demo
  • Hadoop Talend Integration
Close Menu
error: Content is protected !!