Skip to content

zookeeper.md

Latest commit

 

History

History
67 lines (48 loc) · 2.56 KB

zookeeper.md

File metadata and controls

67 lines (48 loc) · 2.56 KB
id zookeeper
title ZooKeeper

Apache Druid uses Apache ZooKeeper (ZK) for management of current cluster state.

Minimum ZooKeeper versions

Apache Druid supports all stable versions of ZooKeeper. For information about ZooKeeper's stable version, see ZooKeeper releases.

ZooKeeper operations

The operations that happen over ZK are:

  1. Coordinator leader election
  2. Overlord leader election
  3. Service (node) announcement and discovery — services announce their presence so other services can find them
  4. Overlord and Middle Manager task management

Segment loading, dropping, and discovery no longer use ZooKeeper — they are served over HTTP.

Coordinator leader election

Druid uses the Curator LeaderLatch recipe to perform leader election at path

${druid.zk.paths.coordinatorPath}/_COORDINATOR

Overlord leader election

Druid uses the same LeaderLatch recipe for Overlord leader election at path

${druid.zk.paths.overlordPath}/_OVERLORD

Service announcement and discovery

Each Druid service announces a DruidNode record (host, port, role, services) under the internal-discovery path so that other services can enumerate cluster members by role:

${druid.zk.paths.base}/internal-discovery/${nodeRole}/${druid.host}

Brokers and Coordinators use this path to find Historicals, Peons, and Indexers. They then poll each discovered service's HTTP /druid-internal/v1/segments endpoint to get its current set of served segments.