Kafka for Application Modernization

Kafka for Application Modernization Courseware (WA2708)

This Kafka for Application Modernization training class is a general introduction course to get students understanding and working with Kafka.

Publisher: Web Age Solutions

Benefits

  • Understand the use of Kafka for high performance messaging.
  • Identify the usages for Kafka in Microservices.
  • Explain the benefits of Kafka patterns.
  • Differentiate between messaging and message brokers.
  • Describe Kafka messaging environments.
  • Develop producers and consumers for Kafka.
  • Recognize how Kafka enables Cloud-native applications.
  • Summarize characteristics and architecture for Kafka.
  • Demonstrate how to process messages with Kafka.
  • Design distributed high throughput systems based on Kafka.
  • Describe the built-in partitioning, replication and inherent fault-tolerance of Kafka.

Outline

  1. Introduction to KAFKA
    1. Messaging Architectures – What is Messaging?
    2. Messaging Architectures – Steps to Messaging
    3. Messaging Architectures – Messaging Models
    4. What is Kafka?
    5. Kafka Overview
    6. Need for Kafka
    7. When to Use Kafka?
    8. Kafka Architecture
    9. Core concepts in Kafka
    10. Kafka Topic
    11. Kafka Partitions
    12. Kafka Producer
    13. Kafka Consumer
    14. Kafka Broker
    15. Kafka Cluster
    16. Why Kafka Cluster?
    17. Sample Multi-Broker Cluster
    18. Overview of ZooKeeper
    19. Kafka Cluster & ZooKeeper
    20. Who Uses Kafka?
    21. Summary
  2. Using Apache Kafka
    1. Installing Apache Kafka
    2. Configuration Files
    3. Starting Kafka
    4. Using Kafka Command Line Client Tools
    5. Setting up a Multi-Broker Cluster
    6. Using Multi-Broker Cluster
    7. Kafka Connect
    8. Kafka Connect – Configuration Files
    9. Using Kafka Connect to Import/Export Data
    10. Creating a Spring Boot Producer
    11. Adding Kafka dependency to pom.xml
    12. Defining a Spring Boot Service to Send Message(s)
    13. Defining a Spring Boot Controller
    14. Testing the Spring Boot Producer
    15. Creating a Nodejs Consumer
    16. Summary
  3. Building Data Pipelines
    1. Building Data Pipelines
    2. Considerations When Building Data Pipelines
    3. Timeliness
    4. Reliability
    5. High and Varying Throughput
    6. Data Formats
    7. Transformations
    8. Security
    9. Failure Handling
    10. Coupling and Agility
    11. Ad-hoc Pipelines
    12. Loss of Metadata
    13. Extreme Processing
    14. Kafka Connect Versus Producer and Consumer
    15. Summary
  4. Integrating Kafka with Other Systems
    1. Introduction to Kafka Integration
    2. Kafka Connect
    3. Running Kafka Connect
    4. Key Configurations for Connect workers:
    5. Kafka Connect API
    6. Kafka Connect Example – File Source
    7. Kafka Connect Example – File Sink
    8. Kafka Connector Example – MySQL to Elasticsearch
    9. Write the data to Elasticsearch
    10. Building Custom Connectors
    11. Kafka Connect – Connectors
    12. Kafka Connect - Tasks
    13. Kafka Connect - Workers
    14. Kafka Connect - Converters and Connect’s data model
    15. Kafka Connect - Offset management
    16. Alternatives to Kafka Connect
    17. Introduction to Storm
    18. Other Components of Spark
    19. Integrating Storm with Kafka
    20. Integrating Storm with Kafka – Sample Code
    21. Integrating Storm with Kafka
    22. Introduction to Hadoop
    23. Hadoop Components
    24. Integrating Hadoop with Kafka
    25. Hadoop Consumers
    26. Hadoop Consumers – Produce Topic
    27. Hadoop Consumers – Fetch Generated Topic
    28. Summary
  5. Kafka Security
    1. Kafka Security
    2. Encryption and Authentication using SSL
    3. Configuring Kafka Brokers
    4. Configuring Kafka Brokers – Optional Settings
    5. Authenticating Using SASL
    6. Authenticating Using SASL – Configuring Kafka Brokers
    7. Authorization and ACLs
    8. Securing a Running Cluster
    9. ZooKeeper Authentication
    10. Summary
  6. Monitoring Kafka
    1. Introduction
    2. Metrics Basics
    3. JVM Monitoring
    4. Garbage collection
    5. Java OS monitoring
    6. OS Monitoring
    7. Kafka Broker Metrics
    8. Under-Replicated Partitions
    9. Active controller count
    10. Request handler idle ratio
    11. Intelligent Thread Usage
    12. All topics bytes in
    13. All topics bytes out
    14. All topics messages in
    15. Partition count
    16. Leader count
    17. Offline partitions
    18. Request metrics
    19. Logging
    20. Client Monitoring
    21. Producer Metrics
    22. Overall producer metrics
    23. Per-broker and per-topic metrics
    24. Consumer Metrics
    25. Fetch Manager Metrics
    26. Per-broker and per-topic metrics
    27. Consumer coordinator metrics
    28. Quotas
    29. Lag Monitoring
    30. End-to-End Monitoring
    31. Summary
  7. Kafka Operational Aspects
    1. Kafka Administration
    2. Kafka Cluster Management
    3. Kafka Replica Distribution
    4. Partition Reassignment
    5. Kafka Topic Management
    6. Kafka Cluster Mirroring
    7. Integration with Other Tools
    8. Summary
  8. Lab Exercises
    1. Kafka Basics
    2. Kafka Multiple Brokers and Import/Export Messages
    3. Apache Kafka with Java
    4. Apache Kafka with Node.js
    5. Kafka Integration With Spark
    6. KSQL Basics
    7. KSQL Create and Deploy UDF

Required Prerequisites

None

Useful Prerequisites

  • Basic understanding of messaging, cloud, development, architecture, and virtualization would be beneficial.
License

Length: 2 days | $150.00 per copy

LicenseRequest More InformationRequest Trainer Evaluation Copy