# Appendix ## Table of contents - [Appendix: Cluster sizes](./appendix-cluster-sizes/) - [Appendix: Prepare for swap and upgrade to v26.0](./upgrade-to-swap/) --- ## Cluster sizes ## Default Cluster Sizes For Self-Managed Materialize, the cluster sizes are configured with the following default resource allocations:

Size	Scale	CPU Limit	Disk Limit	Memory Limit
`25cc`	`1`	`0.5`	`7762MiB`	`3881MiB`
`50cc`	`1`	`1`	`15525MiB`	`7762MiB`
`100cc`	`1`	`2`	`31050MiB`	`15525MiB`
`200cc`	`1`	`4`	`62100MiB`	`31050MiB`
`300cc`	`1`	`6`	`93150MiB`	`46575MiB`
`400cc`	`1`	`8`	`124201MiB`	`62100MiB`
`600cc`	`1`	`12`	`186301MiB`	`93150MiB`
`800cc`	`1`	`16`	`248402MiB`	`124201MiB`
`1200cc`	`1`	`24`	`372603MiB`	`186301MiB`
`1600cc`	`1`	`31`	`481280MiB`	`240640MiB`
`3200cc`	`1`	`62`	`962560MiB`	`481280MiB`
`6400cc`	`2`	`62`	`962560MiB`	`481280MiB`

## Custom Cluster Sizes When installing the Materialize Helm chart, you can override the [default cluster sizes and resource allocations](#default-cluster-sizes). These cluster sizes are used for both internal clusters, such as the `system_cluster`, as well as user clusters. > **Tip:** In general, you should not have to override the defaults. At minimum, we > recommend that you keep the 25-200cc cluster sizes. ```yaml operator: clusters: sizes: : workers: scale: 1 # Generally, should be set to 1. cpu_exclusive: cpu_limit: # e.g., 6 cpu_request: # e.g., 4 (optional, defaults to cpu_limit, may not be higher than cpu_limit) credits_per_hour: "0.0" # N/A for self-managed. disk_limit: # e.g., "93150MiB" memory_limit: # e.g., "46575MiB" swap_enabled: # optional, defaults to the cluster-level swap_enabled selectors: # k8s label selectors # ex: kubernetes.io/arch: amd64 ``` | Field | Type | Description | Recommendation | | --- | --- | --- | --- | | workers | int | The number of timely workers in your cluster replica. | Use 1 worker per CPU core, with a minimum of 1 worker. | | scale | int | The number of pods (i.e., processes) to use in a cluster replica; used to scale out replicas horizontally. Each pod will be provisioned using the settings defined in the size definition. | Generally, this should be set to 1. This should only be greater than 1 when a replica needs to take on limits that are greater than the maximum limits permitted on a single node. | | cpu_exclusive | bool | The flag that determines if the workers should attempt to pin to a particular CPU core. |

Set to true if and only if the cpu_limit is a whole number and the CPU management policy in the k8s cluster is set to static.

Prefer whole number values to enable CPU affinity. Kubernetes only allows CPU Affinity for pods taking a whole number of cores.

If the value is not a whole number, set cpu_exclusive to false.

| | cpu_request | float | The Kubernetes CPU request for a replica pod, in cores. If not set, defaults to the value of cpu_limit. | In most cases, you do not need to set this. It is useful when you want to allow CPU bursting by setting a request lower than the limit. | | memory_limit | string | The Kubernetes memory limit for a replica pod (e.g., "46575MiB"). |

For most workloads, use an approximate 1:8 CPU-to-memory ratio (1 core: 8 GiB). This can vary depending on your workload characteristics.

| | disk_limit | string | The size of the persistent volume to provision for a replica pod (e.g., "93150MiB"). | When spill-to-disk is enabled, use a 1:2 memory-to-disk ratio. Materialize spills data to disk when memory is insufficient, which can impact performance. When swap_enabled is true, this field is automatically set to "0" by the Helm chart. | | credits_per_hour | string | This is a cloud attribute that should be set to “0.00” in self-managed. | Set to “0.00” for self-managed deployments. | | swap_enabled | bool | Enables swap as the spill-to-disk mechanism for this size. When enabled, the replica uses swap instead of a provisioned persistent volume for spilling data. This also causes disk_limit to be set to "0". | This defaults to the global swap_enabled value if not specified per size. Swap generally performs better than spill-to-disk via persistent volumes. | | selectors | map | A map of Kubernetes label selector keys to values used to schedule pods for this cluster size on specific nodes. | It is generally not required to set this. | > **Note:** If you have modified the default cluster size configurations, you can query the > [`mz_cluster_replica_sizes`](/reference/system-catalog/mz_catalog/#mz_cluster_replica_sizes) > system catalog table for the specific resource allocations. --- ## Legacy Terraform: Releases and configurations ## Table of contents --- ## Prepare for swap and upgrade to v26.0 > **Disambiguation:** This page outlines the general steps for upgrading from v25.2 to v26.0 if you are **not** using Materialize provided Terraforms. If you are using Materialize-provided Terraforms, `v0.6.1` and higher of the Terraforms handle the preparation for you. If using Materialize-provided Terraforms, upgrade your Terraform version to `v0.6.1` or higher and follow the Upgrade notes: - AWS Terraform v0.6.1 Upgrade Notes. - GCP Terraform v0.6.1 Upgrade Notes. - Azure Terraform v0.6.1 Upgrade Notes. See also [Upgrade Overview](/self-managed-deployments/upgrading/).

Starting in v26.0.0, Self-Managed Materialize enables swap by default. Swap allows for infrequently accessed data to be moved from memory to disk. Enabling swap reduces the memory required to operate Materialize and improves cost efficiency.

To facilitate upgrades, Self-Managed Materialize added new labels to the node selectors for clusterd pods. To upgrade, you must prepare your nodes with the new labels. This guide provides general instructions for preparing for swap and upgrading to v26.0.0 if you are not using the Materialize-provided Terraforms.

Upgrade to v26.0 without Materialize-provided Terraforms

> **Tip:**

Whe upgrading:

>
Always check the version specific upgrade > notes.
>
>
Always upgrade the operator first and ensure version compatibility > between the operator and the Materialize instance you are upgrading to.
>
>
Always upgrade your Materialize instances after upgrading the operator > to ensure compatibility.
>

> See also [General notes for upgrades](/self-managed-deployments/upgrading/)

Label existing scratchfs/lgalloc node groups.

If using lgalloc on scratchfs volumes, add the additional "materialize.cloud/scratch-fs": "true" label to your existing node groups and nodes running Materialize workloads.

Adding this label to the node group (or nodepool) configuration will apply the label to newly spawned nodes, but depending on your cloud provider may not apply the label to existing nodes.

If not automatically applied, you may need to use kubectl label to apply the change to existing nodes.
Modify existing scratchfs/lgalloc disk setup daemonset selector labels

If using our ephemeral-storage-setup image as a daemonset to configure scratchfs LVM volumes for lgalloc, you must add the additional "materialize.cloud/scratch-fs": "true" label to multiple places:
- spec.selector.matchLabels
- spec.template.metadata.labels
- (if using nodeAffinity) spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms
- (if using nodeSelector) spec.template.spec.nodeSelector
You must use at least one of nodeAffinity or nodeSelector.

It is recommended to rename this daemonset to make it clear that it is only for the legacy scratchfs/lgalloc nodes (for example, change the name disk-setup to disk-setup-scratchfs).

Create a new node group for swap

Create a new node group (or ec2nodeclass and nodepool if using Karpenter in AWS) using an instance type with local NVMe disks. If in GCP, the disks must be in raw mode.
Label the node group with "materialize.cloud/swap": "true".

If using AWS Bottlerocket AMIs (highly recommended if running in AWS), set the following in the userdata to configure the disks for swap, and enable swap in the kubelet:

[settings.oci-defaults.resource-limits.max-open-files]
soft-limit = 1048576
hard-limit = 1048576

[settings.bootstrap-containers.diskstrap]
source = "docker.io/materialize/ephemeral-storage-setup-image:v0.4.0"
mode = "once"
essential = "true"
# ["swap", "--cloud-provider", "aws", "--bottlerocket-enable-swap"]
user-data = "WyJzd2FwIiwgIi0tY2xvdWQtcHJvdmlkZXIiLCAiYXdzIiwgIi0tYm90dGxlcm9ja2V0LWVuYWJsZS1zd2FwIl0="

[kernel.sysctl]
"vm.swappiness" = "100"
"vm.min_free_kbytes" = "1048576"
"vm.watermark_scale_factor" = "100"

If not using AWS or not using Bottlerocket AMIs, and your node group supports it (Azure does not as of 2025-11-05), add a startup taint. This taint will be removed after the disk is configured for swap.
```
taints:
  - key: startup-taint.cluster-autoscaler.kubernetes.io/disk-unconfigured
    value: "true"
    effect: NoSchedule
```

Create a new disk-setup-swap daemonset

If using Bottlerocket AMIs in AWS, you may skip this step, as you should have configured swap using userdata previously.

Create a new daemonset using our ephemeral-storage-setup image to configure the disks for swap and to enable swap in the kubelet.

The arguments to the init container in this daemonset need to be configured for swap. See the examples in the linked git repository for more details.

This daemonset should run only on the new swap nodes, so we need to ensure it has the "materialize.cloud/swap": "true" label in several places:
- spec.selector.matchLabels
- spec.template.metadata.labels
- (if using nodeAffinity) spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms
- (if using nodeSelector) spec.template.spec.nodeSelector
You must use at least one of nodeAffinity or nodeSelector.

It is recommended to name this daemonset to clearly indicate that it is for configuring swap (ie: disk-setup-swap), as opposed to other disk configurations.
(Optional) Configure environmentd to also use swap

Swap is enabled by default for clusterd, but not for environmentd. If you’d like to enable swap for environmentd, add "materialize.cloud/swap": "true" to the environmentd.node_selector helm value.
Upgrade the Materialize operator helm chart to v26

The cluster size definitions for existing Materialize instances will not be changed at this point, but any newly created Materialize instances, or upgraded Materialize instances will pick up the new sizes.

Do not create any new Materialize instances at versions less than v26, or perform any rollouts to existing Materialize instances to versions less than v26.
Upgrade existing Materialize instances to v26

The new v26 pods should go to the new swap nodes.

You can verify that swap is enabled and working by execing into a clusterd pod and running cat /sys/fs/cgroup/memory.swap.max. If you get a number greater than 0, swap is enabled and the pod is allowed to use it.
(Optional) Delete old scratchfs/lgalloc node groups and disk-setup-scratchfs daemonset

If you no longer have anything running on the old scratchfs/lgalloc nodes, you may delete their node group and the disk-setup-scratchfs daemonset.

How to disable swap

If you wish to opt out of swap and retain the old behavior, you may set operator.clusters.swap_enabled: false in your Helm values.