# HAZL reference

Buoyant Enterprise for Linkerd's
[High Availability Zonal Load balancer (HAZL)](/buoyant-enterprise-linkerd/2.15/features/hazl/) is a dynamic
request-level load balancer that can dramatically reduce cloud spend by
minimizing cross-zone traffic. Unlike Kubernetes's native Topology Aware Routing
feature, HAZL never sacrifices reliability to achieve this cost reduction.

## Load band

HAZL's behavior is configured with a _load band_. This load band determines the
nominal load for a system. When system load is within this band, HAZL will not
change its behavior. When system performance falls outside this band, HAZL will
change its behavior:

- When system load exceeds the **high** value of the load band, HAZL will start
  adding endpoints from non-local zones.
- When system load falls below the **low** value of the load band, HAZL will
  start removing endpoints from non-local zones.

System load is defined as the _product of latency and throughput for a single
endpoint_. For example, given a client sending 50 requests per second to an
endpoint that has a median latency of 0.02 seconds, the load for that endpoint
is _50 \* 0.02 = 1.0_. If a Kubernetes service with 10 endpoints has an overall
median latency of 0.02 seconds, at 50 RPS we would expect load per endpoint to
be 0.1.

Note that the load corresponds roughly to the number of in-flight and queued
requests we would expect at any moment for a single endpoint. This can serve as
a useful shorthand when configuring HAZL.

Note also that factors such as retry configuration, circuit breaking, and
ongoing long-lived requests can all influence the exact sequence of changes to
the load balancing pool.

## Configuring HAZL

**Lifecycle operator:**

HAZL is configured within the `additionalEnv` block within the `proxy` block in
the control plane's `controlPlaneConfig` configuration. For example:

```yaml
controlPlaneConfig:
  destinationController:
    additionalArgs:
      - -ext-endpoint-zone-weights # Flag to enable HAZL
  proxy:
    additionalEnv:
      - name: BUOYANT_BALANCER_LOAD_LOW
        value: "0.8"
      - name: BUOYANT_BALANCER_LOAD_HIGH
        value: "2.0"
```

**Helm:**

HAZL is configured by setting `additionalEnv` values on the proxy configuration.
For example:

```bash
helm upgrade linkerd-control-plane -n linkerd --reuse-values \
  --set "destinationController.additionalArgs[0]=-ext-endpoint-zone-weights" \
  --set "proxy.additionalEnv[0].name=BUOYANT_BALANCER_LOAD_LOW" \
  --set-string "proxy.additionalEnv[0].value=0.8" \
  --set "proxy.additionalEnv[1].name=BUOYANT_BALANCER_LOAD_HIGH" \
  --set-string "proxy.additionalEnv[1].value=0.2" \
  linkerd-buoyant/linkerd-enterprise-control-plane
```

**CLI:**

HAZL is configured by setting `additionalEnv` values on the proxy configuration.
For example:

```bash
linkerd upgrade \
  --set "destinationController.additionalArgs[0]=-ext-endpoint-zone-weights" \
  --set "proxy.additionalEnv[0].name=BUOYANT_BALANCER_LOAD_LOW" \
  --set-string "proxy.additionalEnv[0].value=0.8" \
  --set "proxy.additionalEnv[1].name=BUOYANT_BALANCER_LOAD_HIGH" \
  --set-string "proxy.additionalEnv[1].value=0.2" | kubectl apply -f -
```

The values are defined as follows:

| Variable                     | Description                           | Default |
| ---------------------------- | ------------------------------------- | ------- |
| `BUOYANT_BALANCER_LOAD_LOW`  | The low value of the HAZL load band.  | 0.8     |
| `BUOYANT_BALANCER_LOAD_HIGH` | The high value of the HAZL load band. | 2.0     |

**Note:** If the default load bands are changed, you will have to restart your
application proxies for the new values to take effect.
