Kafka 자주 사용하는 명령어 정리

topic 생성:

 

./kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test-topic

---

topic list 확인:

 

./kafka-topics.sh --list --bootstrap-server localhost:9092

---

console에서 topic에 data 전달:

 

./kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test-topic

---

'>'가 나오면 문자열 입력

 

데이터 개수 확인:

 

./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic test-topic --time -1

---

 

특정 topic에서 메세지 받기:

 

./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic --from-beginning

---

group에서 메세지 받아오기:

 

./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic -group testgroup
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic -group testgroup --from-beginning

---

consumer group list 보기:

 

./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list

---

 

group의 상태 보기:

 

./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group testgroup --describe

---

메시지 offset 확인하기:

 

./kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic test-topic --time -1

---

컨슈머 그룹 리스트 확인하기:

 

./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list

---

컨슈머 그룹의 상태 확인하기:

 

./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group testgroup --describe

---

토픽 삭제하기:

 

./kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic test-topic

---

토픽 파티션 추가하기:

 

./kafka-topics.sh --bootstrap-server localhost:9092 --alter --topic test-topic --partitions 3

---

토픽 리플리케이션 팩터 변경하기:

 

./kafka-topics.sh --bootstrap-server localhost:9092 --alter --topic test-topic --replication-factor 3

---

프로듀서에서 데이터 전송 시 Key와 Value 함께 전송하기:

 

./kafka-console-producer.sh --broker-list localhost:9092 --topic test-topic --property parse.key=true --property key.separator=,

---

컨슈머 그룹 offset 리셋하기:

 

./kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group testgroup --reset-offsets --to-earliest --execute --topic test-topic

---