2

I've read that kafka provides a consumer client library that allows recovery by saving the last offset read in zookeeper (not 100% sure about where it's stored).

Is it possible to do the same with Sarama consumers?

Let's say that I'm reading until offset 550, my consumer crashes for 5 min, we are now at offset 700 but I want to resume consuming from offset 550.

Is that possible without having to save the state by myself? I would assume it does but I don't understand how.

I've found sarama.OffsetNewest/Oldestbut that's not what I'm looking for...

0

Kafka consumers used to store offsets in Zookeeper but now they store them directly in Kafka. See the Consumer section in the Kafka docs.

Sarama handles that very well and Sarama consumers will commit (store) offsets in Kafka by default.

Have a look at the Sarama Consumer example. Initially this example starts at the end of the topic but upon restarting, it will restart from its last position.

  • awesome, thank you for pointing out the part of doc explaining it. – Jay Cee Apr 11 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.