Kafka Fundamentals

An intro training on Apache Kafka, the open-source distributed event streaming platform. Well look at the architectural features of Kafka that enable high-performance data delivery.
Code: EAS-026
Duration: 24 hours

Description

This training will help you get a proper understanding of the architecture and functioning of Apache Kafka, an open-source distributed event streaming platform. We will implement Java-based and REST-based clients for Kafka cluster access, discuss cluster and client configuration to achieve tradeoffs between latency, throughput, durability, and availability. Well also consider a multi-cluster setting as it is vital to achieve fault-tolerance and promote scalability.

Kafka Connect allows us to resolve common tasks such as moving data between Kafka and external systems (DBMS, file system, etc.). Using Kafka Streams is the recommended way to build fast and resilient streaming processing solutions.

Roadmap

Overview
  • What is Kafka?
  • Kafka Use Cases and Anti Use Cases
  • Kafka vs Database
  • Kafka vs Message Queue
  • Kafka Storage Architecture: Brokers, Topics, Partitions, Segments, Replicas
  • Kafka Cluster Architecture: Zookeeper, Partition Assignment, Replica Leaders and Followers, In-Sync Replicas
  • Kafka Clients and APIs

Cluster Setup
  • Practice: Setting up Kafka Cluster using Docker
  • Zookeeper Ensemble Installation
  • Kafka Brokers Installation

Applications Development in Java
  • Kafka Producer API: Basics
  • Practice: Single Message Producer
  • Practice: Group Message Producer
  • Practice: Group Producer with Explicit Partitioning
  • Practice: No-key Message Producer
  • Kafka Consumer API: Basics, Consumer Groups
  • Practice: Single Consumer
  • Practice: Multiple Consumers in Same Group
  • Advanced Producing and Consuming: Batching, Retries, Idempotence, Transactions, Consumer Group Rebalance
  • Practice: Producer with no batching and retries
  • Practice: Idempotent Producer
  • Practice: Transactional Producer
  • Practice: Transactional Producer and Consumer
  • Practice: Transactional Consume-Copy-Produce

External Access to Kafka Cluster
  • Security: SSL, SASL, Kerberos
  • Confluent REST Proxy: Producing and consuming JSON, Binary and Avro files
  • Practice: Access via REST Proxy
  • Confluent Schema Registry
  • Practice: Use REST Proxy with Schema Registry

Operations and Monitoring
  • Performance Goals and Modes of Execution
  • Zookeeper: Configuration, Standalone and Quorum Modes, CLI
  • Cluster Management and Monitoring
  • Practice: Monitoring Kafka using Prometheus

Multi Clusters
  • Why Multi Clusters in Kafka?
  • MirrorMakerAccess
  • Practice: Using MirrorMaker to Synchronize Clusters
  • Confluence Replicator

Kafka Connect
  • Kafka Connect Architecture
  • Kafka Connect Sinks
  • Practice: FileStream Sink Connector
  • Practice: JDBC Sink Connector
  • Kafka Connect Sources
  • Practice: FileStream Source Connector
  • Practice: JDBC Source Connector

Kafka Streams Basics
  • Kafka Streams: Streams DSL, Processor API, StreamBuilder, Topology Creating and Implementing
  • Demo: Implementing Streams Topology
  • Demo: Create Streams application using StreamBuilder
  • Kafka Streams Internals: Multihreading, Multiple Instances and Tasks
  • Schemata in Kafka: Serialization, JSON Schema, Avro Schema, Schema Registry, Schema Evolution
  • Demo: Using JSON Schema
  • Stateful Processing: State Store, Fault Tolerance for State Stores
  • KTable: Update Streams, Caching, Emit Rate, Aggregation
  • Windowing in Streams: Timestamp Extractors, Tumbling, Hopping and Session Windows
  • Streaming Joins: Stream with Stream, KTable with KTable, Stream with KTable

Objectives

  • Understand Kafka architecture
  • Understand the deployment and configuration of Kafka
  • Use REST-based access to Kafka
  • Create Kafka Java API clients
  • Design multi-cluster architectures
  • Use Kafka Connect tools
  • Create Kafka Streams programs

Target Audience

  • Software Developers
  • Software Architects
  • Data Engineers

Prerequisites

  • Development experience in Java over 3 months


Trainer in Online
Alexander Breyman Expert in Software Development and Architecture, Oracle
Alexander is a Doctor of Science and associate professor. Since 1991 he has been involved in the development of information and communication systems - during this time he rose from a programming position to head of the department of information systems. Alexander also has extensive experience in teaching, since 1994 he has been teaching on subjects related to data management, information systems design and software development.

He is currently an assistant professor of software development for the software engineering department at the Faculty of Business Informatics in one of the key universities in Russia. Alexander is involved in the development of training programs but he also gives lectures, conducts seminars and facilitating practical and laboratory works.

Since 2014 Alexander has been a regular expert for Luxoft Training especially in the area of Design and Architecture of Oracle Software and DBMS, where he is engaged in carrying out training courses, developing training programs and putting together a strategy for this area.

Alexander is a member of the IEEE, and ACM.
Schedule in Online Prices
Course is scheduled in: Online
16.08.2021 - 23.08.2021
Time: 10:00-14:00
Timezone: UTC +2
City: Online
Duration: 24 hours
Language: English
Trainer
Alexander Breyman
Expert in Software Development and Architecture, Oracle
-15% €410
€349
Register for the next course
+
Registering in advance ensures you have priority. Well notify you when we schedule the next course on this topic
16.08.2021 - 23.08.2021
Time: 10:00-14:00
Timezone: UTC +2
City: Online
Duration: 24 hours
Language: English
Trainer
Alexander Breyman
Expert in Software Development and Architecture, Oracle
-15% €410
€349
Register for the next course
+
Registering in advance ensures you have priority. Well notify you when we schedule the next course on this topic
Online €410
Bucharest €410
Krakow €410
Wroclaw €410

Invoices for the course will be issued in local currency. All fees above can change according to training location and delivery mode and are subject to change while scheduling. Price does not include VAT.

Courses you may be interested in:
EAS-016
8 hours
EAS-019
8 hours

Sign up

Choose dates
First Name*
Last Name*
Company
Email*
Phone
Your Comment
Still have questions?
Connect with us
Thank you.
Your request has been received.
Thank you!
The form has been submitted successfully.