# Deployment guidelines Self-managed Materialize requires: a Kubernetes (v1.31+) cluster; PostgreSQL as a metadata database; blob storage; and a license key. ## Available deployment guidelines The following guides outline recommended configurations for deploying Materialize across different cloud environments. - [AWS Deployment Guidelines](/self-managed-deployments/deployment-guidelines/aws-deployment-guidelines/) - [Azure Deployment Guidelines](/self-managed-deployments/deployment-guidelines/azure-deployment-guidelines/) - [GCP Deployment Guidelines](/self-managed-deployments/deployment-guidelines/gcp-deployment-guidelines/) --- ## AWS deployment guidelines Self-managed Materialize requires: a Kubernetes (v1.31+) cluster; PostgreSQL as a metadata database; blob storage; and a license key. ## Recommended instance types As a general guideline, we recommend: - ARM-based CPU - A 1:8 ratio of vCPU to GiB memory. - A 8:1 ratio of GiB local instance storage to GiB memory when using swap. When operating in AWS, we recommend the following instances: | EC2 Instances | | ---------------| | `r8g`, `r7g`, and `r6g` families when running without local disk. | | `r7gd` and `r6gd` families (and `r8gd` once available) when running with local disk. *Recommended for production.* | Starting in v0.3.1, the Materialize on AWS Terraform uses `["r7gd.2xlarge"]` as the default [`node_group_instance_types`]. [`node_group_instance_types`]: https://github.com/MaterializeInc/terraform-aws-materialize?tab=readme-ov-file#input_node_group_instance_types ## Locally-attached NVMe storage Configuring swap on nodes to use locally-attached NVMe storage allows Materialize to spill to disk when operating on datasets larger than main memory. This setup can provide significant cost savings and provides a more graceful degradation rather than OOMing. Network-attached storage (like EBS volumes) can significantly degrade performance and is not supported. ### Swap support **New Terraform:** #### New Terraform The new Materialize [Terraform module](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main/aws/examples/simple) supports configuring swap out of the box. **Legacy Terraform:** #### Legacy Terraform The Legacy Terraform provider adds preliminary swap support in v0.6.1, via the [`swap_enabled`](https://github.com/MaterializeInc/terraform-aws-materialize?tab=readme-ov-file#input_swap_enabled) variable. With this change, the Terraform: - Creates a node group for Materialize. - Configures NVMe instance store volumes as swap using a daemonset. - Enables swap at the Kubelet. See [Upgrade Notes](https://github.com/MaterializeInc/terraform-aws-materialize?tab=readme-ov-file#v061). > **Note:** If deploying `v25.2`, Materialize clusters will not automatically use swap unless they are configured with a `memory_request` less than their `memory_limit`. In `v26`, this will be handled automatically. ## TLS When running with TLS in production, run with certificates from an official Certificate Authority (CA) rather than self-signed certificates. ## Upgrading guideline

Whe upgrading:

--- ## Azure deployment guidelines ## Recommended instance types As a general guideline, we recommend: - ARM-based CPU. - A 1:8 ratio of vCPU to GiB memory. - An 8:1 ratio of GiB local instance storage to GiB memory when using swap. ### Recommended Azure VM Types with Local NVMe Disks When operating on Azure in production, we recommend [Epdsv6 sizes](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/memory-optimized/epdsv6-series?tabs=sizebasic#sizes-in-series) Azure VM Types with Local NVMe Disk: | VM Size | vCPUs | Memory | Ephemeral Disk | Disk-to-RAM Ratio | | ------------------ | ----- | ------- | -------------- | ----------------- | | Standard_E2pds_v6 | 2 | 16 GiB | 75 GiB | ~4.7:1 | | Standard_E4pds_v6 | 4 | 32 GiB | 150 GiB | ~4.7:1 | | Standard_E8pds_v6 | 8 | 64 GiB | 300 GiB | ~4.7:1 | | Standard_E16pds_v6 | 16 | 128 GiB | 600 GiB | ~4.7:1 | | Standard_E32pds_v6 | 32 | 256 GiB | 1,200 GiB | ~4.7:1 | > **Warning:** These VM types provide **ephemeral** local NVMe SSD disks. Data is > **lost** when the VM is stopped or deleted. ## Locally-attached NVMe storage Configuring swap on nodes to use locally-attached NVMe storage allows Materialize to spill to disk when operating on datasets larger than main memory. This setup can provide significant cost savings and provides a more graceful degradation rather than OOMing. Network-attached storage (like EBS volumes) can significantly degrade performance and is not supported. ### Swap support **New Terraform:** #### New Terraform The new Materialize [Terraform module](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main/azure/examples/simple) supports configuring swap out of the box. **Legacy Terraform:** #### Legacy Terraform The Legacy Terraform provider, adds preliminary swap support in v0.6.1, via the [`swap_enabled`](https://github.com/MaterializeInc/terraform-azurerm-materialize?tab=readme-ov-file#input_swap_enabled) variable. With this change, the Terraform: - Creates a node group for Materialize. - Configures NVMe instance store volumes as swap using a daemonset. - Enables swap at the Kubelet. See [Upgrade Notes](https://github.com/MaterializeInc/terraform-azurerm-materialize?tab=readme-ov-file#v061). > **Note:** If deploying `v25.2`, Materialize clusters will not automatically use swap unless they are configured with a `memory_request` less than their `memory_limit`. In `v26`, this will be handled automatically. ## Recommended Azure Blob Storage Materialize writes **block** blobs on Azure. As a general guideline, we recommend **Premium block blob** storage accounts. ## TLS When running with TLS in production, run with certificates from an official Certificate Authority (CA) rather than self-signed certificates. ## Upgrading guideline

Whe upgrading:

--- ## GCP deployment guidelines ## Recommended instance types As a general guideline, we recommend: - ARM-based CPU. - A 1:8 ratio of vCPU to GiB memory. - An 8:1 ratio of GiB local instance storage to GiB memory when using swap. When operating on GCP in production, we recommend the following machine types that support local SSD attachment: | Series | Examples | | ------ | ---------- | | [N2 high-memory series] | `n2-highmem-16` or `n2-highmem-32` with local NVMe SSDs | | [N2D high-memory series] | `n2d-highmem-16` or `n2d-highmem-32` with local NVMe SSDs | To maintain the recommended 8:1 disk-to-RAM ratio for your machine type, see [Number of local SSDs](#number-of-local-ssds) to determine the number of local SSDs to use. See also [Locally attached NVMe storage](#locally-attached-nvme-storage). ## Number of local SSDs Each local NVMe SSD in GCP provides 375GB of storage. Use the appropriate number of local SSDs to ensure your total disk space is at least twice the amount of RAM in your machine type for optimal Materialize performance. > **Note:** Your machine type may only supports predefined number of local SSDs. For instance, `n2d-highmem-32` allows only the following number of local > SSDs: `4`,`8`,`16`, or `24`. To determine the valid number of Local SSDs to attach for your machine type, see the [GCP > documentation](https://cloud.google.com/compute/docs/disks/local-ssd#lssd_disk_options). For example, the following table provides a minimum local SSD count to ensure the 2:1 disk-to-RAM ratio. Your actual count will depend on the [your machine type](https://cloud.google.com/compute/docs/disks/local-ssd#lssd_disk_options). | Machine Type | RAM | Required Disk | Minimum Local SSD Count | Total SSD Storage | |-----------------|---------|---------------|-----------------------------|-------------------| | `n2-highmem-8` | `64GB` | `128GB` | 1 | `375GB` | | `n2-highmem-16` | `128GB` | `256GB` | 1 | `375GB` | | `n2-highmem-32` | `256GB` | `512GB` | 2 | `750GB` | | `n2-highmem-64` | `512GB` | `1024GB` | 3 | `1125GB` | | `n2-highmem-80` | `640GB` | `1280GB` | 4 | `1500GB` | [N2 high-memory series]: https://cloud.google.com/compute/docs/general-purpose-machines#n2-high-mem [N2D high-memory series]: https://cloud.google.com/compute/docs/general-purpose-machines#n2d_machine_types ## Locally-attached NVMe storage Configuring swap on nodes to use locally-attached NVMe storage allows Materialize to spill to disk when operating on datasets larger than main memory. This setup can provide significant cost savings and provides a more graceful degradation rather than OOMing. Network-attached storage (like EBS volumes) can significantly degrade performance and is not supported. ### Swap support **New Terraform:** #### New Terraform The Materialize [Terraform module](https://github.com/MaterializeInc/materialize-terraform-self-managed/tree/main/gcp/examples/simple) supports configuring swap out of the box. **Legacy Terraform:** #### Legacy Terraform The Legacy Terraform provider, adds preliminary swap support in v0.6.1, via the [`swap_enabled`](https://github.com/MaterializeInc/terraform-google-materialize?tab=readme-ov-file#input_swap_enabled) variable. With this change, the Terraform: - Creates a node group for Materialize. - Configures NVMe instance store volumes as swap using a daemonset. - Enables swap at the Kubelet. See [Upgrade Notes](https://github.com/MaterializeInc/terraform-google-materialize?tab=readme-ov-file#v061). > **Note:** If deploying `v25.2`, Materialize clusters will not automatically use swap unless they are configured with a `memory_request` less than their `memory_limit`. In `v26`, this will be handled automatically. ## CPU affinity It is strongly recommended to enable the Kubernetes `static` [CPU management policy](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#static-policy). This ensures that each worker thread of Materialize is given exclusively access to a vCPU. Our benchmarks have shown this to substantially improve the performance of compute-bound workloads. ## TLS When running with TLS in production, run with certificates from an official Certificate Authority (CA) rather than self-signed certificates. ## Upgrading guideline

Whe upgrading: