Kafka — Custom Serializer and Deserializer

Kafka sends the events though network which requires the message to be serialize before sending over the network. Publisher API provides the serializer like IntegerSerializer, StringSerializer etc, same sense of deserializer.

Serializer is used by publisher while Deserializer by consumer.

If the default serializer may not full-fill the need specially in case of any custom object required to pass through network. For example employee, customer objects etc. In this case we can build the customer serailizer and configure as serializer.

Example:

Lets consider a json based message need to send to Kafka topic, then follow the below steps.

  • Create Json
  • Generate java class using the Maven plugin
  • Create the Custom serializer and deserializer
  • Configure these in producer and consumer side respectively
  • Produces can create the java object and send it.

Create Json

Add the dependencies and maven plugin in pom.xml

Generate the java classes from defined json. Select the json file and compile it as below.

Generate JSON to Java

The above steps creates the below java file, implements Serializable inteface to all three classes.

Generated Java Classes

Create the CustomSerailizer class

Create the CustomDeserailizer class

Producer Code

Constant class

Consumer Class

Start Producer

Producer published the events

Start Consumer

Message Consumer

--

--

Middleware Expert in Design and Development, Working on Kafka, Eager to investigate and learn — Software Engineer.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Narayan Kumar

Middleware Expert in Design and Development, Working on Kafka, Eager to investigate and learn — Software Engineer.