Skip to content

Default start offset for kafka sources

Peter Fischer requested to merge default-kafka-start-offsets into main

add a default start offset for kafka sources

In case either job (producer/aggregator or consumer/indexer) gets executed without specifying start and/or end offsets there we want to control the default offsets to start from: Both applications, aggregator and indexer, fall back to the committed offsets (watermark-indicated offset stored in kafka/zookeeper per consumer group). If the committed offsets cannot be found (which happens with kafka 1.x) they fall back to latest and earliest, respectively.

Bug: T348112 Change-Id: Idc39fb07e076938dec6a6c0bffcd976e8a97dc8f

Merge request reports