Unevenly placed leaders and copies slow the whole NATS cluster, not only the account that owns them. The platform team balances across every account; an application team refines within pools of its own streams, so that hot streams spread among themselves instead of piling onto one server while the leader count still looks even. The two are layered: the account balancer yields to the system one, and both wait while the NATS cluster is not Settled.
Across the NATS cluster #
The system balancer runs on system credentials. Placement moves work for any account; leader moves work for accounts that grant them with an export preset.
01-system.yaml
# Owned by the platform team: evenness across the whole NATS cluster, every
# account included, on system credentials alone.
apiVersion: nats.mikluko.io/v1beta1
kind: NatsConnection
metadata:
name: demo-sys
namespace: nats-system
spec:
servers: ["nats://demo.nats-system.svc:4222"]
credentials:
secretKeyRef:
name: jetstream-controller-creds # the preset jetstream-controller system user
---
apiVersion: jetstream.nats.mikluko.io/v1beta1
kind: NatsSystemBalancer
metadata:
name: demo
namespace: nats-system
spec:
# At most one per NATS cluster.
connectionRef:
name: demo-sys
moves:
# Opt-in, as every placement move is.
placement: true
# Runs for every account carrying the jetstream-stepdown export preset
# (below); the rest are reported in status.capabilities.
leader: true
# One move per pass, none while the NATS cluster is not Settled.
interval: 1m
# Moves stay inside the NATS cluster a stream is placed in. Moving a stream
# to another NATS cluster is a spec edit (story 8), never a balancer move.
kubectl apply -f https://nats-operator.io/docs/stories/07-balancing/01-system.yaml01-account-export.yaml
# What lets the system balancer move this account's leaders. The preset
# expands to two service exports, $JS.API.STREAM.LEADER.STEPDOWN.* and
# $JS.API.CONSUMER.LEADER.STEPDOWN.*.*, and the auth controller adds the
# matching imports to the system account's JWT under a per-account prefix.
# The account is story 4's, limits and all; a manifest that left them out
# would take them away.
apiVersion: auth.nats.mikluko.io/v1beta1
kind: NatsAccount
metadata:
name: payments
namespace: nats-system
spec:
operatorRef:
name: demo
limits:
connections: 200
jetstream:
memoryStorage: 512Mi
diskStorage: 100Gi
streams: 10
consumers: 100
exports:
- preset: jetstream-stepdown
kubectl apply -f https://nats-operator.io/docs/stories/07-balancing/01-account-export.yamlIts status says what it can do, how uneven each server is, and the last move it made. With three servers and three copies of every stream, every server holds every copy, so only leaders have anywhere to move.
01-status-natssystembalancer.yaml
status:
observedGeneration: 1
conditions:
- type: Ready
status: "True"
reason: Balancing
- type: Holding
status: "False"
reason: Settled
# orders, from story 2, carries no jetstream-stepdown export: its leaders
# stay where they are, and the others move around them.
capabilities:
placement: true
leader: Partial
leaderReason: 1 of 2 accounts carry no jetstream-stepdown export; their leaders are not moved
# ORDERS, PAYMENTS and REQ_07, three copies each on three servers.
servers:
- name: demo-0
leaders: 1
replicas: 3
- name: demo-1
leaders: 1
replicas: 3
- name: demo-2
leaders: 1
replicas: 3
skew:
leaders: 0
replicas: 0
# Absent until a pass has moved something; where the streams' leaders
# landed decides whether one had to.
lastMove:
kind: Leader
account: ABFAGFB4TX4F4GEPOPBDD4SUQR23BDIBE3NCLZFE3ZF3VP77JJHFHGQF
stream: PAYMENTS
from: demo-2
to: demo-0
time: "2026-09-26T21:00:10Z"
pending: []
Within an account #
The account balancer runs on the account’s own connection and judges evenness per pool. Pools select streams, key-value buckets and object stores by label.
01-account.yaml
# Owned by the payments team: evenness within pools of its own streams. It
# yields to the system balancer: no move on a stream the system balancer has
# pending, and none while the NATS cluster is not Settled.
apiVersion: jetstream.nats.mikluko.io/v1beta1
kind: NatsBalancer
metadata:
name: payments
namespace: payments
spec:
# The account comes from the connection's credentials, as for streams.
connectionRef:
name: demo
# A pool selects NatsStream, NatsKeyValue and NatsObjectStore resources in
# this namespace by label; each stream's consumers go with it. Streams in no
# pool, and streams with no resource, form the default pool. With no pools
# declared the whole account is one pool. A stream matching several pools
# belongs to the first in this list, and status reports Overlapping.
pools:
- name: requests
selector:
matchLabels:
traffic: requests
- name: responses
selector:
matchLabels:
traffic: responses
moves:
leader: true # on by default
placement: false # opt-in
---
# What the selectors match.
apiVersion: jetstream.nats.mikluko.io/v1beta1
kind: NatsStream
metadata:
name: req-07
namespace: payments
labels:
traffic: requests
spec:
connectionRef:
name: demo
name: REQ_07
subjects: ["req.07.>"]
replicas: 3
kubectl apply -f https://nats-operator.io/docs/stories/07-balancing/01-account.yamlAt rest it reports each pool’s streams and how unevenly their leaders sit. While the system balancer has a move pending on one of its streams, Holding turns True with reason YieldingToSystemBalancer, naming the stream, and the account balancer moves nothing.
01-status-natsbalancer.yaml
status:
observedGeneration: 1
conditions:
- type: Ready
status: "True"
reason: Balancing
- type: Holding
status: "False"
reason: Settled
- type: Overlapping
status: "False"
reason: PoolsDisjoint
# REQ_07 in requests; PAYMENTS, from story 4, carries no pool label and
# falls to the default pool. A pool of one stream is skewed by one
# whichever server leads it.
pools:
- name: requests
streams: 1
leaderSkew: 1
- name: responses
streams: 0
- name: (default)
streams: 1
leaderSkew: 1