Hi,
We have been collecting around 1 billion events events from all the country in every day and all the events are sent to only one topic with 150 partition. While processing the events there is a lag and we cannot optimize the processing the events payload more since it is very simple. Every events has city fields to set which city it is sent to our system. Therefore, I can use this fields to send a city related topic and every city related topic has its own consumer or I can increase the partition size of the main topic, which is currently 150 in order to increase the consume speed and eradicate the lag. We have 5 broker and the replication factor is 5. Our main go is to overcome the lag so we are open to use more memory. Which method do you recommend us to apply? Thanks a lot.