Appendix: Azure deployment guidelines
Recommended instance types
As a general guideline, we recommend:
-
Processor Type: ARM-based CPU
-
Sizing: 2:1 disk-to-RAM ratio with spill-to-disk enabled.
Recommended Azure VM Types with Local NVMe Disks
When operating on Azure in production, we recommend Epdsv6 sizes Azure VM Types with Local NVMe Disk:
VM Size | vCPUs | Memory | Ephemeral Disk | Disk-to-RAM Ratio |
---|---|---|---|---|
Standard_E2pds_v6 | 2 | 16 GiB | 75 GiB | ~4.7:1 |
Standard_E4pds_v6 | 4 | 32 GiB | 150 GiB | ~4.7:1 |
Standard_E8pds_v6 | 8 | 64 GiB | 300 GiB | ~4.7:1 |
Standard_E16pds_v6 | 16 | 128 GiB | 600 GiB | ~4.7:1 |
Standard_E32pds_v6 | 32 | 256 GiB | 1,200 GiB | ~4.7:1 |
See also Locally attached NVMe storage.
Locally-attached NVMe storage
For optimal performance, Materialize requires fast, locally-attached NVMe storage. Having a locally-attached storage allows Materialize to spill to disk when operating on datasets larger than main memory as well as allows for a more graceful degradation rather than OOMing. Network-attached storage (like EBS volumes) can significantly degrade performance and is not supported.
Starting in v0.4.0 of Materialize on Azure Terraform, disk support (using OpenEBS and NVMe instance storage) is enabled, by default, for Materialize. With this change, the Terraform:
-
Installs OpenEBS via Helm;
-
Configures NVMe instance store volumes using a bootstrap script;
-
Creates appropriate storage classes for Materialize.
Associated with this change:
-
The following configuration options are available:
-
The default
aks_config.vm_size
has changed fromStandard_E8ps_v6
toStandard_E4pds_v6
. See Recommended instance types.
See also
CPU affinity
It is strongly recommended to enable the Kubernetes static
CPU management policy.
This ensures that each worker thread of Materialize is given exclusively access to a vCPU. Our benchmarks have shown this
to substantially improve the performance of compute-bound workloads.
TLS
When running with TLS in production, run with certificates from an official Certificate Authority (CA) rather than self-signed certificates.