Acls | Security | Kafka Tutorial

Introduction to Kafka ACLs

Access Control Lists (ACLs) are used in Apache Kafka to control access to various resources such as topics, consumer groups, and more. ACLs help in managing permissions and ensuring that only authorized users and applications can perform operations on Kafka resources.

Kafka ACL Basics

Kafka ACLs are defined using the following components:

Principal: The user or service account to which the ACL applies.
Resource: The Kafka resource (e.g., topic or consumer group) that the ACL is applied to.
Operation: The action that the principal is allowed or denied to perform on the resource (e.g., Read, Write).
Permission Type: Whether the operation is allowed or denied.

Setting Up Kafka ACLs

To set up ACLs in Kafka, you need to use the kafka-acls.sh script, which is located in the bin directory of your Kafka installation.

Example: Adding a Read ACL for a Topic

To allow a user (e.g., User:alice) to read messages from a topic (e.g., my-topic), use the following command:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:alice --operation Read --topic my-topic

Example: Adding a Write ACL for a Topic

To allow a user (e.g., User:bob) to write messages to a topic (e.g., my-topic), use the following command:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:bob --operation Write --topic my-topic

Listing and Removing Kafka ACLs

You can list existing ACLs using the --list option and remove ACLs using the --remove option in the kafka-acls.sh script.

Example: Listing ACLs for a Topic

To list ACLs for a specific topic (e.g., my-topic), use the following command:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list --topic my-topic

ACLs for resource `Topic:LITERAL:my-topic`:
  User:alice has Allow permission for operations: Read from hosts: *
  User:bob has Allow permission for operations: Write from hosts: *

Example: Removing an ACL for a Topic

To remove a specific ACL (e.g., Read permission for User:alice) from a topic (e.g., my-topic), use the following command:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --remove --allow-principal User:alice --operation Read --topic my-topic

Advanced ACL Configurations

Kafka ACLs can also be applied to other resources such as consumer groups, transactional IDs, and more. The general format for adding, listing, and removing ACLs remains the same, with the appropriate resource type and name specified.

Example: Adding an ACL for a Consumer Group

To allow a user (e.g., User:charlie) to consume messages from a consumer group (e.g., my-group), use the following command:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:charlie --operation Read --group my-group

Example: Adding an ACL for a Transactional ID

To allow a user (e.g., User:dave) to use a transactional ID (e.g., my-transactional-id), use the following command:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:dave --operation Describe --transactional-id my-transactional-id

Best Practices for Kafka ACLs

When configuring ACLs in Kafka, consider the following best practices:

Define specific ACLs for individual users or service accounts to limit access to only what is necessary.
Regularly review and audit ACLs to ensure they are up-to-date and meet security requirements.
Use wildcard characters cautiously to avoid granting broader permissions than intended.
Implement a structured naming convention for topics, consumer groups, and other resources to simplify ACL management.

Conclusion

Kafka ACLs provide a robust mechanism for controlling access to Kafka resources and ensuring the security of your Kafka cluster. By understanding and properly configuring ACLs, you can manage permissions effectively and secure your Kafka deployment.

Kafka ACLs Tutorial