The Apache Kafka course offered by BSAI Academy is a key requirement for those aspiring to become Big Data Hadoop architects. Apache Kafka is an open-source stream processing platform and a high-performance real-time messaging system that can process millions of messages per second. It provides a distributed and partitioned messaging system that is highly fault tolerant. This Kafka Training course will guide participants through Kafka architecture, installation, interfaces and configuration on their way to learning the advanced concepts of Big Data.

Introduction to Big Data and Apache Kafka

In this module, you will understand where Kafka fits in the Big Data space, and Kafka Architecture. In addition, you will learn about Kafka Cluster, its Components, and how to Configure a Cluster

Skills:

• Kafka Concepts
• Kafka Installation
• Configuring Kafka Cluster

Objectives: At the end of this module, you should be able to:

• Explain what is Big Data
• Understand why Big Data Analytics is important
• Describe the need of Kafka
• Know the role of each Kafka Components
• Understand the role of ZooKeeper
• Install ZooKeeper and Kafka
• Classify different type of Kafka Clusters
• Work with Single Node-Single Broker Cluster

Topics:

• Introduction to Big Data
• Big Data Analytics
• Need for Kafka
• What is Kafka?
• Kafka Features
• Kafka Concepts
• Kafka Architecture
• Kafka Components
• ZooKeeper
• Where is Kafka Used?
• Kafka Installation
• Kafka Cluster
• Types of Kafka Clusters
• Configuring Single Node Single Broker Cluster

Hands on:

• Kafka Installation
• Implementing Single Node-Single Broker Cluster

Kafka Producer

Kafka Producers send records to topics. The records are sometimes referred to as Messages. In this Module, you will work with different Kafka Producer APIs.

Skills:

• Configure Kafka Producer
• Constructing Kafka Producer
• Kafka Producer APIs
• Handling Partitions

Objectives:

At the end of this module, you should be able to:

• Construct a Kafka Producer
• Send messages to Kafka
• Send messages Synchronously & Asynchronously
• Configure Producers
• Serialize Using Apache Avro
• Create & handle Partitions

Topics:

• Configuring Single Node Multi Broker Cluster
• Constructing a Kafka Producer
• Sending a Message to Kafka
• Producing Keyed and Non-Keyed Messages
• Sending a Message Synchronously & Asynchronously
• Configuring Producers
• Serializers
• Serializing Using Apache Avro
• Partitions

Hands On:

• Working with Single Node Multi Broker Cluster
• Creating a Kafka Producer
• Configuring a Kafka Producer
• Sending a Message Synchronously & Asynchronously

Kafka Consumer

Applications that need to read data from Kafka use a Kafka Consumer to subscribe to Kafka topics and receive messages from these topics. In this module, you will learn to construct Kafka Consumer, process messages from Kafka with Consumer, run Kafka Consumer and subscribe to Topics

Skills:

•Configure Kafka Consumer
•Kafka Consumer API
•Constructing Kafka Consumer

Objectives: At the end of this module, you should be able to:

• Perform Operations on Kafka
• Define Kafka Consumer and Consumer Groups
• Explain how Partition Rebalance occurs
• Describe how Partitions are assigned to Kafka Broker
• Configure Kafka Consumer
• Create a Kafka consumer and subscribe to Topics
• Describe & implement different Types of Commit
• Deserialize the received messages

Topics:

• Consumers and Consumer Groups
• Standalone Consumer
• Consumer Groups and Partition Rebalance
• Creating a Kafka Consumer
• Subscribing to Topics
• The Poll Loop
• Configuring Consumers
• Commits and Offsets
• Rebalance Listeners
• Consuming Records with Specific Offsets
• Deserializers

Hands On:

• Creating a Kafka Consumer
• Configuring a Kafka Consumer
• Working with Offsets

Kafka Internals

Kafka provides a unified, high-throughput, low-latency platform for handling real-time data feeds. Learn more about tuning Kafka to meet your high performance needs.

Skills:

• Kafka APIs
• Kafka Storage
• Configure Broker

Objectives:

At the end of this module, you should be able to:

• Understand Kafka Internals
• Explain how Replication works in Kafka
• Differentiate between In-sync and Out-off-sync Replicas
• Understand the Partition Allocation
• Classify and Describe Requests in Kafka
• Configure Broker, Producer, and Consumer for a Reliable System
• Validate System Reliabilities
• Configure Kafka for Performance Tuning

Topics:

• Cluster Membership
• The Controller
• Replication
• Request Processing
• Physical Storage
• Reliability
• Broker Configuration
• Using Producers in a Reliable System
• Using Consumers in a Reliable System
• Validating System Reliability
• Performance Tuning in Kafka

Hands On:

• Create topic with partition & replication factor 3 and execute it on multi-broker cluster
• Show fault tolerance by shutting down 1 Broker and serving its partition from another broker

Kafka Cluster Architectures & Administering Kafka

Kafka Cluster typically consists of multiple brokers to maintain load balance. ZooKeeper is used for managing and coordinating Kafka broker. Learn about Kafka Multi-Cluster Architectures, Kafka Brokers, Topic, Partitions, Consumer Group, Mirroring, and ZooKeeper Coordination in this module.

Skills:

• Administer Kafka

Objectives:

At the end of this module, you should be able to

• Understand Use Cases of Cross-Cluster Mirroring
• Learn Multi-cluster Architectures
• Explain Apache Kafka’s MirrorMaker
• Perform Topic Operations
• Understand Consumer Groups
• Describe Dynamic Configuration Changes
• Learn Partition Management
• Understand Consuming and Producing
• Explain Unsafe Operations

Topics:

• Use Cases – Cross-Cluster Mirroring
• Multi-Cluster Architectures
• Apache Kafka’s MirrorMaker
• Other Cross-Cluster Mirroring Solutions
• Topic Operations
• Consumer Groups
• Dynamic Configuration Changes
• Partition Management
• Consuming and Producing
• Unsafe Operations

Hands on:

• Topic Operations
• Consumer Group Operations
• Partition Operations
• Consumer and Producer Operations

Kafka Monitoring and Kafka Connect

Learn about the Kafka Connect API and Kafka Monitoring. Kafka Connect is a scalable tool for reliably streaming data between Apache Kafka and other systems.

Skills:

• Kafka Connect
• Metrics Concepts
• Monitoring Kafka

Objectives:

At the end of this module, you should be able to use,

• Explain the Metrics of Kafka Monitoring
• Understand Kafka Connect
• Build Data pipelines using Kafka Connect
• Understand when to use Kafka Connect vs Producer/Consumer API
• Perform File source and sink using Kafka Connect

Topics:

• Considerations When Building Data Pipelines
• Metric Basics
• Kafka Broker Metrics
• Client Monitoring
• Lag Monitoring
• End-to-End Monitoring
• Kafka Connect
• When to Use Kafka Connect?
• Kafka Connect Properties

Hands on:

• Kafka Connect

Kafka Stream Process

Learn about the Kafka Streams API in this module. Kafka Streams is a client library for building mission-critical real-time applications and microservices, where the input and/or output data is stored in Kafka Clusters.

Skills:

• Stream Processing using Kafka

Objectives:

At the end of this module, you should be able to,

• Describe What is Stream Processing
• Learn Different types of Programming Paradigm
• Describe Stream Processing Design Patterns
• Explain Kafka Streams & Kafka Streams API

Topics:

• Stream Processing
• Stream-Processing Concepts
• Stream-Processing Design Patterns
• Kafka Streams by Example
• Kafka Streams: Architecture Overview

Hands on:

• Kafka Streams
• Word Count Stream Processing

Integration of Kafka With Hadoop, Storm and Spark

In this module, you will learn about Apache Hadoop, Hadoop Architecture, Apache Storm, Storm Configuration, and Spark Ecosystem. In addition, you will configure Spark Cluster, Integrate Kafka with Hadoop, Storm, and Spark.

Skills:

• Kafka Integration with Hadoop
• Kafka Integration with Storm
• Kafka Integration with Spark

Objectives:

At the end of this module, you will be able to,

• Understand What is Hadoop
• Explain Hadoop 2.x Core Components
• Integrate Kafka with Hadoop
• Understand What is Apache Storm
• Explain Storm Components
• Integrate Kafka with Storm
• Understand What is Spark
• Describe RDDs
• Explain Spark Components
• Integrate Kafka with Spark

Topics:

• Apache Hadoop Basics
• Hadoop Configuration
• Kafka Integration with Hadoop
• Apache Storm Basics
• Configuration of Storm
• Integration of Kafka with Storm
• Apache Spark Basics
• Spark Configuration
• Kafka Integration with Spark

Hands On:

• Kafka integration with Hadoop
• Kafka integration with Storm
• Kafka integration with Spark

Integration of Kafka With Talend and Cassandra

Understanding the need for Kafka Integration, successfully integrating it with Apache Flume, steps in integration of Flume with Kafka as a Source. Learn how to integrate Kafka with Flume, Cassandra and Talend.

Skills:

• Kafka Integration with Flume
• Kafka Integration with Cassandra
• Kafka Integration with Talend

Objectives:

At the end of this module, you should be able to,

• Understand Flume
• Explain Flume Architecture and its Components
• Setup a Flume Agent
• Integrate Kafka with Flume
• Understand Cassandra
• Learn Cassandra Database Elements
• Create a Keyspace in Cassandra
• Integrate Kafka with Cassandra
• Understand Talend
• Create Talend Jobs
• Integrate Kafka with Talend

Topics:

• Flume Basics
• Integration of Kafka with Flume
• Cassandra Basics such as and KeySpace and Table Creation
• Integration of Kafka with Cassandra
• Talend Basics
• Integration of Kafka with Talend

Hands On:

• Kafka demo with Flume
• Kafka demo with Cassandra
• Kafka demo with Talend

Kafka In-Class Project

In this module, you will work on a project, which will be gathering messages from multiple
sources.

Scenario:

In E-commerce industry, you must have seen how catalog changes frequently. Most deadly problem they face is “How to make their inventory and price
consistent?”.

There are various places where price reflects on Amazon, Flipkart or Snapdeal. If you will visit Search page, Product Description page or any ads on Facebook/google. You will find there are some mismatch in price and availability. If we see user point of view that’s very disappointing because he spends more time to find better products and at last if he doesn’t purchase just because of consistency.

Here you have to build a system which should be consistent in nature. For example, if you are getting product feeds either through flat file or any event stream you have to make sure you don’t lose any events related to product specially inventory and price.

If we talk about price and availability it should always be consistent because there might be possibility that the product is sold or the seller doesn’t want to sell it anymore or any other reason. However, attributes like Name, description doesn’t make that much noise if not updated on time.

Problem Statement

You have given set of sample products. You have to consume and push products to Cassandra/MySQL once we get products in the consumer. You have to save below-mentioned fields in Cassandra.
1. PogId
2. Supc
3. Brand
4. Description
5. Size
6. Category
7. Sub Category
8. Country
9. Seller Code

In MySQL, you have to store
1. PogId
2. Supc
3. Price
4. Quantity

Project

This Project enables you to gain Hands-On experience on the concepts that you have learned as part of this Course.

You can email the solution to our Support team within 2 weeks from the Course Completion Date. Edureka will evaluate the solution and award a Certificate with a Performance-based Grading.

Problem Statement:
You are working for a website techreview.com that provides reviews for different technologies. The company has decided to include a new feature in the website which will allow users to compare the popularity or trend of multiple technologies based on twitter feeds. They want this comparison to happen in real time. So, as a big data developer of the company, you have been task to implement following things:

• Near Real Time Streaming of the data from Twitter for displaying last minute’s count of people tweeting about a particular technology.

• Store the twitter count data into Cassandra.

What if I miss a class?

Don’t worry. You will always get a recording for the class in your inbox. Have a look at that and reach out to the faculty in case of doubts. All our live classes are recorded for self-study purpose. Hence, in case you miss a class, you can refer to the video recording and then reach out to the faculty during their doubts clearing time or ask your question in the beginning of the subsequent class.

Can I download the recordings?

Yes. We provide url for the video downloads.

Recordings are integral part of BSAI Academy intellectual property. The downloading/distribution of these recordings in anyway is strictly prohibited and illegal as they are protected under copyright act. Incase a student is found doing the same, it will lead to an immediate and permanent suspension in the services, access to all the learning resources will be blocked, course fee will be forfeited and the institute will have all the rights to take strict legal action against the individual.

Will I get a certificate in the end?

Yes. All our course are certified. As part of the course, students get weekly assignments and module-wise case studies and Exam will be taken at end after course completion. Candidates must bring 85% score on exam. Once all your submissions are received and evaluated, the certificate shall be awarded.

Do you help in placements?

We follow a comprehensive and a self-sustaining system to help our students with placements. This is a win-win situation for our candidates and corporate clients. As a pre-requisite for learning validation, candidates are required to submit the case studies and project work provided as a part of the course (flexible deadline). Support from our side is continuous and encompasses help in profile building, CV referrals (as and when applicable) through our ex-students, HR consultants and companies directly reaching out to us.

We will provide guidance to you in terms of what are the right profiles for you based on your education and experience, interview preparation and conducting mock interviews, if required. The placement process for us doesn’t end at a definite time post your course completion, but is a long relationship that we will like to build.

Do you guarantee placements?

No institute can guarantee placements, unless they are doing so as a marketing gimmick! It is on a best effort basis.

In professional environment, it is not feasible for any institute to do so, except for a marketing gimmick. For us, it is on a best effort basis but not time – bound – in some cases students reach out to us even after 3 years for career support.

Do you have a classroom option?

No. We provide only online let training.

What do I need to attend the online classes?

To attend the online classes, all you need is a laptop/PC with a basic internet connection. Students have often shared good feedback of attending these live classes through their data card or even their mobile 3G connection, though we recommend a basic broadband connection.

For best user experience, a mic-headphone is recommended to enhance the voice quality, though the laptop’s in-built mic works fine and you can ask your question over the chat as well.

How can I reach out to someone if I have doubts post class?

Students can always connect with the trainer or even schedule one-to-one time via online. During the course we also schedule periodic doubts-clearing classes though students can also ask doubts of a class in the subsequent class.

I am having difficulty coping up with my classes. What can I do?

For all the courses, we also provide the recordings of each class for their self-reference as well as revision in case you miss any concept in the class. In case you still have doubts after revising through the recordings, you can communicate with your trainer/tutor via email.

Can I pay in instalments?

Not for this course. The instalment options are available only for our courses which are atleast 3 months long.

What are the system requirements for the software?

It is recommended to have 64-bit operating system with minimum 8GB RAM so that the virtual lab can be installed easily

Is it feasible to attend a demo session before enrollment?

Unfortunately participation in a live class without enrollment is not possible. However, We can provide you the sample class recording and it would give you a clear insight about how are the classes conducted, quality of instructors and the level of interaction in the class.

Apache Kafka Certification

The entire training course content is in line with the certification program and helps you clear Apache Kafka  Certification Exam with ease and get the best jobs in the top MNCs. As part of this training you will be working on real time projects and assignments that have immense implications in the real world industry scenario thus helping you fast track your career effortlessly.

At the end of this training program there will be a exam that helps you score better marks in certification exam.