# Balance JetStream leaders and copies


This guide shows you how to keep the leaders and the copies of streams spread evenly over the servers of a NATS cluster.
The manifests create a `NatsSystemBalancer` that balances every account of the NATS cluster `demo`, and a `NatsBalancer` that balances the account `payments` in pools of its own streams.

## Before you begin

You need:

- The NATS cluster `demo` and its NATS operator from [Put a NATS cluster under a NATS operator](https://nats-operator.io/docs/stories/02-auth-plane/index.md), with the Secret `jetstream-controller-creds`.
- The account `payments`, and the `NatsConnection` `demo` in the namespace `payments`, from [Let a team declare its own users and streams](https://nats-operator.io/docs/stories/04-team-self-service/index.md).

## Balance every account of the NATS cluster

As the platform team, apply a `NatsConnection` with the creds of the system account user `jetstream-controller`, and a `NatsSystemBalancer` on it.
A NATS cluster has at most one `NatsSystemBalancer`.

`01-system.yaml`

```yaml
apiVersion: nats-operator.io/v1beta1
kind: NatsConnection
metadata:
  name: demo-sys
  namespace: nats-system
spec:
  servers: ["nats://demo.nats-system.svc:4222"]
  credentials:
    secretKeyRef:
      name: jetstream-controller-creds
---
apiVersion: jetstream.nats-operator.io/v1beta1
kind: NatsSystemBalancer
metadata:
  name: demo
  namespace: nats-system
spec:
  connectionRef:
    name: demo-sys
  moves:
    placement: true
    leader: true
  interval: 1m
```

```sh
kubectl apply -f https://nats-operator.io/docs/stories/07-balancing/01-system.yaml
```


- `moves.placement` lets the balancer move copies between servers. It is off unless you set it.
- `moves.leader` lets the balancer move leaders, in the accounts that have the `jetstream-stepdown` export.
- `interval` is the time between passes. A pass makes at most one move.

A balancer keeps a stream inside its NATS cluster.
To move a stream to another NATS cluster, see [Move a stream to another NATS cluster](https://nats-operator.io/docs/stories/08-stream-transfer/index.md).
[NatsSystemBalancerSpec](https://nats-operator.io/docs/reference/api/index.md#NatsSystemBalancerSpec) in the API reference lists every field.

## Let the system balancer move the leaders of an account

The system balancer can make a placement move in any account.
It makes a leader move only in an account that exports the preset `jetstream-stepdown`.
Apply the account `payments` with that export added:

`01-account-export.yaml`

```yaml
apiVersion: auth.nats-operator.io/v1beta1
kind: NatsAccount
metadata:
  name: payments
  namespace: nats-system
spec:
  operatorRef:
    name: demo
  limits:
    connections: 200
    jetstream:
      memoryStorage: 512Mi
      diskStorage: 100Gi
      streams: 10
      consumers: 100
  exports:
    - preset: jetstream-stepdown
```

```sh
kubectl apply -f https://nats-operator.io/docs/stories/07-balancing/01-account-export.yaml
```


The manifest repeats the limits that the account already has.
If you leave them out, the account loses them.
[jetstream-stepdown](https://nats-operator.io/docs/reference/nats-permissions/index.md#jetstream-stepdown) in the NATS permissions reference lists the subjects of the preset.

## Check the system balancer

Read its status:

```sh
kubectl -n nats-system get natssystembalancer demo -o yaml
```

The `status` in the output is similar to this:

`01-status-natssystembalancer.yaml`

```yaml
status:
  observedGeneration: 1
  conditions:
    - type: Ready
      status: "True"
      reason: Balancing
    - type: Holding
      status: "False"
      reason: Settled
  capabilities:
    placement: true
    leader: Partial
    leaderReason: 1 of 2 accounts carry no jetstream-stepdown export; their leaders are not moved
  servers:
    - name: demo-0
      leaders: 1
      replicas: 3
    - name: demo-1
      leaders: 1
      replicas: 3
    - name: demo-2
      leaders: 1
      replicas: 3
  skew:
    leaders: 0
    replicas: 0
  lastMove:
    kind: Leader
    account: ABFAGFB4TX4F4GEPOPBDD4SUQR23BDIBE3NCLZFE3ZF3VP77JJHFHGQF
    stream: PAYMENTS
    from: demo-2
    to: demo-0
    time: "2026-09-26T21:00:10Z"
  pending: []
```


- `capabilities.leader` is Partial, because the account `orders` has no `jetstream-stepdown` export. The balancer leaves the leaders of `orders` where they are.
- `servers` counts the leaders and the copies on each server, and `skew` is how uneven they are.
  The NATS cluster has three servers, and each of its three streams has three copies, so every server has a copy of every stream and the balancer can move only leaders.
- `lastMove` is absent until the balancer has made a move.

## Balance the streams of an account in pools

As the payments team, apply a `NatsBalancer` on the team's own connection, and a stream with the label that the pool `requests` selects:

`01-account.yaml`

```yaml
apiVersion: jetstream.nats-operator.io/v1beta1
kind: NatsBalancer
metadata:
  name: payments
  namespace: payments
spec:
  connectionRef:
    name: demo
  pools:
    - name: requests
      selector:
        matchLabels:
          traffic: requests
    - name: responses
      selector:
        matchLabels:
          traffic: responses
  moves:
    leader: true
    placement: false
---
apiVersion: jetstream.nats-operator.io/v1beta1
kind: NatsStream
metadata:
  name: req-07
  namespace: payments
  labels:
    traffic: requests
spec:
  connectionRef:
    name: demo
  name: REQ_07
  subjects: ["req.07.>"]
  replicas: 3
```

```sh
kubectl apply -f https://nats-operator.io/docs/stories/07-balancing/01-account.yaml
```


The balancer evens the leaders in each pool separately.
A pool selects `NatsStream`, `NatsKeyValue` and `NatsObjectStore` resources in the namespace of the balancer by label, and the consumers of a stream belong to the pool of the stream.

- A stream that no pool selects, or that has no resource, is in the default pool.
- A stream that several pools select is in the first of them, and the condition `Overlapping` is True.
- If you declare no pools, the whole account is one pool.

[NatsBalancerSpec](https://nats-operator.io/docs/reference/api/index.md#NatsBalancerSpec) in the API reference lists every field.

## Check the account balancer

Read its status:

```sh
kubectl -n payments get natsbalancer payments -o yaml
```

The `status` in the output is similar to this:

`01-status-natsbalancer.yaml`

```yaml
status:
  observedGeneration: 1
  conditions:
    - type: Ready
      status: "True"
      reason: Balancing
    - type: Holding
      status: "False"
      reason: Settled
    - type: Overlapping
      status: "False"
      reason: PoolsDisjoint
  pools:
    - name: requests
      streams: 1
      leaderSkew: 1
    - name: responses
      streams: 0
    - name: (default)
      streams: 1
      leaderSkew: 1
```


`pools` counts the streams of each pool, and `leaderSkew` is how uneven their leaders are.
`REQ_07` is in the pool `requests`, and `PAYMENTS` has no pool label, so it is in the default pool.
A pool of one stream has a `leaderSkew` of 1.

The account balancer yields to the system balancer.
While the system balancer has a move pending on a stream of the account, `Holding` is True with the reason `YieldingToSystemBalancer`, and the account balancer moves nothing.
Neither balancer moves anything while the NATS cluster is not Settled.
[Balancing](https://nats-operator.io/docs/design/v1/index.md#65-balancing) in the design describes how the two balancers work together.

