Oct 31, 2020Stream Aggregation In KafkaIntroducing the aggregation in Kafka and explained this in easy way to implement the Aggregation on real time streaming. In order to aggregate the stream we need do two steps operations. Group the stream — groupBy(k,v) (if Key exist in stream) or groupByKey() — Data must partitioned by key. groupBy…Kafka Streams5 min readKafka Streams5 min read
Oct 30, 2020Kafka Streaming — AdvanceStore, KTable, GlobalKTable, If application requires aggregation, Kafka uses the store in order to aggregate the stream for further processing. The use case like word count, live trend of any event and live voting can be consider for the candidate for aggregation. The Stream processing basically requires to consider for…Wordcount5 min readWordcount5 min read
Oct 26, 2020Kafka Streaming API: AdvanceAbout kafka Streaming Kafka DSL-Streaming Event Stream — Continuous flow of events, unbounded dataset and immutable data records. Streaming Operations — Stateless, State full and window based. Used for transform, aggregate, filter and enrich the stream.Serdes7 min readSerdes7 min read
Oct 26, 2020Kafka — Custom Serializer and DeserializerKafka sends the events though network which requires the message to be serialize before sending over the network. Publisher API provides the serializer like IntegerSerializer, StringSerializer etc, same sense of deserializer. Serializer is used by publisher while Deserializer by consumer. If the default serializer may not full-fill the need specially…Serialization5 min readSerialization5 min read
Oct 24, 2020Kafka Streaming API: IntroductionA sample Streaming Application Concept: While doing the Kafka stream client development, need to take step wise approach. Steps can be- Understand the event stream type and NFR Identify and model Event Transport event Process the event stream Example: Customer Check-in is an event stream Customer check-in system generate the…Kafka Streams4 min readKafka Streams4 min read
Oct 23, 2020Event Streaming using Kafka-NoviceThis story talks about the event and processing of events using Kafka. Event- Shows the change in state of object. The unbounded form of flow of event is called event stream. It shows something happened in the source of the event. Streaming- The unbounded form of source of event Producer…Streaming3 min readStreaming3 min read
Oct 23, 2020Kafka Producer-AdvanceThis story talk about advance feature of Kafka like - Interceptor Transaction Partitioning Idempotent Producer Compression Batching Interceptor Helps to change the behaviour of Kafka client application without changing the code. It can be used for the message enrichment or to any this before sending the message or after the message…9 min read9 min read
Oct 19, 2020Kafka Architecture & InternalThis talks about the Apache Kafka Architecture, its components and the internal structure for Kafka for its components. Why Kafka? Multiple producers Multiple consumers Disk based persistence Offline messaging Messaging replay Distributed Super Scalable Low-latency High volume Fault tolerance Real-time Processing Apache Kafka Architecture Single Cluster: Apache Kafka Components: Broker Producer ConsumerKafka6 min readKafka6 min read
Oct 18, 2020Kafka — ProducerPrerequisites Kafka Messaging — Introduction Messaging: Application Integrationmedium.com Producer Flow Event is published as ProducerRecord to the topic which resides in and managed by broker. The ProducerRecord is formed and pushed by Producer. It is followed in a step of steps that is depicted as below. Create the Configuration Properties Create the Producer as below Send the message How…5 min read5 min read
Oct 18, 2020Kafka Messaging — IntroductionMessaging: Application Integration Messaging is the backbone of Enterprise wherein the applications talks to each other over integration layer. There are two basic factors are considered as a main driving factors for implementation of the integration layer. 1. Protocol 2. Design Principle Protocol are like HTTP/ HTTPS/JMS. I will take…Message Format3 min readMessage Format3 min read
