Friday, August 25, 2017

Unlocking Kafka Exactly-one Semantic

How to implement “Exactly-one Semantics” at Kafka side. Before that you, you have to check Kafka version:

If its Kafka 0.11 release then you can try enabling Exactly-once semantics feature. Prior to 0.11 Kafka support at least one delivery but that can cause duplicate results

Exactly-once one semantics has been implemented using concept. Idempotence and Transaction Management features
How to enable Exactly-once Semantics

In Producer config, please add/modify below property
enable.idempotence = True
max.inflight.requests.per.connection=1 =  < some unique id>

In Consumer config, please add/modify below property
isolation.level =  <read_committed or read_uncommitted>