Should we use hostPath for jetstream cluster running in K8S

[royatanu94]

Hi all,

We are planning to run NATS JetStream cluster on Kubernetes with multiple worker nodes, where each node may run one or more JetStream pods.
Should we use hostPath on the worker nodes when creating PVCs for JetStream containers? If we do, will this impact HA or other JetStream behaviors?

Thanks in advance.

[MILLERMARRU]

Do not use hostPath for JetStream in production Kubernetes. Here is why and what to use instead:

hostPath binds the JetStream data directory to a specific node. If the pod is rescheduled to a different node (node drain, node failure, rolling update), the new pod starts on a different node with no data — effectively losing that server's portion of the stream. In a 3-node JetStream cluster, losing one server's data means you lose quorum history and need a full resync, which can be disruptive.

What to use instead:

PersistentVolumeClaim (PVC) via StatefulSet — this is the correct pattern and what the NATS Helm chart uses by default:

# In the NATS Helm chart values.yaml
nats:
  jetstream:
    enabled: true
    fileStorage:
      enabled: true
      storageClassName: "gp3"   # or your cloud provider's SSD class
      size: "20Gi"

Each StatefulSet pod gets its own PVC via the volumeClaimTemplates. When a pod is rescheduled, Kubernetes reattaches the same PVC to the pod on the new node (for network-attached storage like AWS EBS, GCP PD, Azure Disk).

For the best throughput, use SSD-backed storage classes (gp3 on AWS, premium-rwo on GKE). JetStream is write-heavy and benefits significantly from low-latency storage.

HA behavior: Each of your 3 NATS nodes maintains its own copy of the stream state (RAFT replication). With PVCs, a node failure means another node takes leadership and the recovered pod rejoins with its PVC intact — no data loss and no full resync needed.