Kafka topics, treating internal components, protection, default marking

DevOps Kafka
1

Hey all :wave:

Quick question about Schema Registry and Kafka Connect topics like _schemas`, `__connect-configs`, `__connect-status, and __connect-offsets.

When I run:
kafka-topics.sh --bootstrap-server <BROKER> --list --exclude-internal
I still see these topics — unlike __consumer_offsets, which is hidden. This suggests they’re not actually marked as internal.

:mag: So I’m wondering:

  1. Is there any way to force Kafka to treat these topics as internal (e.g. via Schema Registry or Connect configuration)?
  2. More importantly — is there a way to protect these topics so that even if someone has All ACLs on topics, they can’t delete or modify them? (just like how __consumer_offsets is protected)
  3. Why doesn’t Kafka mark these topics as internal by default, even though they are clearly used by internal components?

Would really appreciate your insights :pray:

2

_schemas is a Confluent Schema Registry topic, so from an Apache Kafka standpoint it isn’t internal nor should be hidden. I do not believe there are means for someone to create a topic and mark it as internal. In any case, this topic is not internal to AK.

3

the same would be true for any connect-config topic as well, those are created by Kafka Connect and the 3 topics can be set to whatever you want in the connect configuration. (in case you have more than 1 connect cluster). So to Core Apache Kafka, they are not hidden as well.

4

personally, I do not see these as internal topics.

5

may I ask the reason/desire to have them hidden?

6

Thanks for your answer!

I work at a company that manages many Kafka clusters for internal teams in our organization. As part of the service, we often bootstrap those clusters with extensions like Kafka Connect and Schema Registry for our internal clients.

We want to give each client full control over their topics — meaning broad ACLs like All on Topic:* — so they can self-manage independently.

At the same time, we also want to protect the infrastructure-related topics (like _schemas, __connect-configs, etc.) that are critical for those extensions to work properly.