Kafka - TimeoutException: Timed out waiting for next producer ID block

DevOps Kafka
1

Hi Folks,
I am getting below error from my 2 node kafka cluster having 2 zk nodes too:

research-kafka_2-1      | [2022-10-19 19:05:08,516] INFO [BrokerToControllerChannelManager broker=2 name=forwarding]: Recorded new controller, from now on will use broker kafka_2:9092 (id: 2 rack: null) (kafka.server.BrokerToControllerRequestThread)
research-kafka_2-1      | [2022-10-19 19:05:08,566] INFO [BrokerToControllerChannelManager broker=2 name=forwarding]: Recorded new controller, from now on will use broker kafka_2:9092 (id: 2 rack: null) (kafka.server.BrokerToControllerRequestThread)
research-kafka_2-1      | [2022-10-19 19:05:08,570] INFO [BrokerToControllerChannelManager broker=2 name=forwarding] Client requested disconnect from node 2 (org.apache.kafka.clients.NetworkClient)
research-kafka_2-1      | [2022-10-19 19:05:08,571] INFO [BrokerToControllerChannelManager broker=2 name=forwarding]: Recorded new controller, from now on will use broker kafka_2:9092 (id: 2 rack: null) (kafka.server.BrokerToControllerRequestThread)
research-kafka_2-1      | [2022-10-19 19:05:08,618] ERROR [KafkaApi-2] Unexpected error handling request RequestHeader(apiKey=INIT_PRODUCER_ID, apiVersion=4, clientId=producer-1, correlationId=3) -- InitProducerIdRequestData(transactionalId=null, transactionTimeoutMs=2147483647, producerId=-1, producerEpoch=-1) with context RequestContext(header=RequestHeader(apiKey=INIT_PRODUCER_ID, apiVersion=4, clientId=producer-1, correlationId=3), connectionId='192.168.48.4:29093-192.168.48.1:53722-5', clientAddress=/192.168.48.1, principal=User:ANONYMOUS, listenerName=ListenerName(PLAINTEXT_EXT), securityProtocol=PLAINTEXT, clientInformation=ClientInformation(softwareName=apache-kafka-java, softwareVersion=3.3.0), fromPrivilegedListener=false, principalSerde=Optional[org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder@681811d1]) (kafka.server.KafkaApis)
research-kafka_2-1      | org.apache.kafka.common.errors.TimeoutException: Timed out waiting for next producer ID block
research-kafka_2-1      | [2022-10-19 19:05:08,621] INFO [BrokerToControllerChannelManager broker=2 name=forwarding]: Recorded new controller, from now on will use broker kafka_2:9092 (id: 2 rack: null) (kafka.server.BrokerToControllerRequestThread)
research-kafka_2-1      | [2022-10-19 19:05:08,622] WARN [RPC ProducerId Manager 2]: Timed out when requesting AllocateProducerIds from the controller. (kafka.coordinator.transaction.RPCProducerIdManager)```
Above error comes repeatedly whenever I try to send record through producer. Below is my producer config:
```Properties producerProperties = new Properties();

        producerProperties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:29092,localhost:29093");
        producerProperties.setProperty(ProducerConfig.ACKS_CONFIG, "all");
        producerProperties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        producerProperties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        producerProperties.setProperty(ProducerConfig.LINGER_MS_CONFIG, "25000");```
invoking send() method on producer returns
```Unexpected error in InitProducerIdResponse; The request timed out.```
in java producer app.
2

This value seems unusually high: producerProperties.setProperty(ProducerConfig.LINGER_MS_CONFIG, "25000");

That could be what’s triggering the error. Can you try a much lower value like 5 or the default value of 0 to see if it resolves the error?

3

Yes, it worked. But what’s the reason ? When I tried with same value of [linger.ms](http://linger.ms) in a single node kafka cluster with 2 zk nodes, it worked fine. Can you let me know why its happening with multi-node kafka cluster only ?

4

Moreover, setting any value greater than 0 for linger.ms causes this issue

5

From the log you shared:

(multiple occurrences…)
Recorded new controller, from now on will use broker kafka_2:9092 (id: 2 rack: null)
WARN [RPC ProducerId Manager 2]: Timed out when requesting AllocateProducerIds from the controller.
I suspect one of your brokers isn’t reachable. If you deployed this with Docker images check your listener and advertised listener is set correctly.