Configuring BEL's trust-rotation operator

BEL provides automated trust anchor rotation in the form of a Kubernetes operator that runs on the cluster and a companion linkerd-trust CLI extension.

This guide walks through installing the operator, bootstrapping the initial trust bundle, verifying everything is healthy, and performing a full trust rotation end to end.

Prerequisites

The trust-rotation operator depends on a cluster that already has cert-manager producing the Linkerd trust anchor and identity issuer. If you are unfamiliar with how cert-manager fits into Linkerd’s identity story, read the cert-manager and trust-manager concepts guide first.

In particular, the operator expects to find the following resources at installation time:

• An Issuer named linkerd-trust-root-issuer in the cert-manager namespace.
• A Certificate named linkerd-trust-anchor in the cert-manager namespace.
• A cluster-scoped ClusterIssuer named linkerd-identity-issuer.
• A Certificate named linkerd-identity-issuer in the Linkerd namespace.

The names can be overridden via flags on linkerd trust check, but matching the defaults keeps the workflow simple.

These resources can be deployed with the help of the linkerd-cert-manager chart, as explained below.

You will also need:

• Enterprise Linkerd 2.20 or later.
• The BEL CLI installed and configured.
• Cluster permissions sufficient to install Helm charts in the linkerd namespace and to create cluster-scoped CRDs.

Installing cert-manager and the Linkerd PKI

Install cert-manager:

helm repo add jetstack https://charts.jetstack.io --force-update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true \
  --wait

Install the linkerd-cert-manager chart. This bootstraps the cert-manager Issuer and Certificate resources that produce Linkerd’s trust-anchor and identity-issuer secrets:

helm install linkerd-cert-manager \
  -n linkerd --create-namespace \
  oci://ghcr.io/linkerd/charts/linkerd-cert-manager --wait

Installing the linkerd-trust CLI

The linkerd-trust CLI ships with the usual BEL install command, which installs BEL and the buoyant extension; that now also installs the linkerd-trust binary as another extension, accessible by invoking linkerd trust:

curl https://enterprise.buoyant.io/install | sh
linkerd trust --help

Bootstrapping the initial trust bundle

Linkerd reads its trust roots from the linkerd-identity-trust-roots ConfigMap. Before the operator is running, this ConfigMap must be seeded from cert-manager’s linkerd-trust-anchor Secret. The CLI’s bundle subcommand renders the right manifest:

linkerd trust bundle | kubectl apply -f -

Installing Linkerd

With the PKI and trust bundle in place, install the Linkerd CRDs and control plane. The relevant settings tell Linkerd to consume the cert-manager-managed secrets:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/experimental-install.yaml

helm repo add linkerd-buoyant https://helm.buoyant.cloud

helm repo update

helm install linkerd-crds \
  -n linkerd --create-namespace \
  linkerd-buoyant/linkerd-enterprise-crds

helm install linkerd-control-plane -n linkerd \
  --set license=$BUOYANT_LICENSE \
  --set identity.externalCA=true \
  --set identity.issuer.scheme=kubernetes.io/tls \
  linkerd-buoyant/linkerd-enterprise-control-plane

linkerd check

Installing the trust-rotation operator

The operator is published as two Helm charts: a CRD chart and the operator chart proper. Install the CRD chart first; the operator chart assumes the TrustAnchorRotation CRD already exists.

helm install -n linkerd trust-rotation-operator-crds \
    oci://ghcr.io/buoyantio/charts/trust-rotation-operator-crds --devel --wait

helm install -n linkerd trust-rotation-operator \
    oci://ghcr.io/buoyantio/charts/trust-rotation-operator --devel --wait

By default the operator chart creates a single TrustAnchorRotation resource named cluster. You can watch its status in real time (this requires yq):

watch -d -n 1 'kubectl get trustanchorrotations.trust.linkerd.io cluster \
    -o yaml | yq .status'

Verifying the installation

linkerd trust check validates that cert-manager, the cert-manager-managed Linkerd PKI resources, and the trust-rotation operator itself are all installed and healthy:

linkerd trust check

The output mirrors linkerd check’s formatting and ends with either Status check results are √ or Status check results are ×. You can also ask for JSON output for automation:

linkerd trust check -o json

Because the CLI registers as a linkerd extension, linkerd check will invoke linkerd trust check automatically and surface the same results alongside the core Linkerd checks.

If you only want to confirm that the prerequisites for the operator are in place (cert-manager and the cert-manager-managed resources), without checking whether the operator itself is installed, use --pre:

linkerd trust check --pre

Performing a trust-anchor rotation

With the operator running, rotating the trust anchor reduces to two manual cert-manager actions. This walkthrough uses cert-manager’s cmctl CLI, but any mechanism that triggers cert-manager renewal will work.

Step 1: Trigger trust-root renewal

cmctl renew -n cert-manager linkerd-trust-anchor

Shortly after this, the operator will detect the new trust root and proceed to:

1. Copy the old trust root into linkerd-previous-anchor.
2. Reconcile linkerd-identity-trust-roots to contain both the old and new roots.
3. Roll the Linkerd control plane so linkerd-proxy-injector observes the expanded bundle.

TrustAnchorRotation.status.phase will move through RefreshingControlPlane and into PropagatingTrustRoots.

Step 2: Restart your data plane

Meshed workloads need to be restarted so their newly injected proxies pick up the expanded bundle. For example, with emojivoto:

kubectl -n emojivoto rollout restart deploy

When every meshed pod has the expanded bundle, the operator advances to WaitingForIdentityRotation. You can confirm with:

linkerd trust inspect

which summarizes the current phase, convergence counts, and any pods that have not yet converged.

Step 3: Trigger identity-issuer renewal

This is the one intentional manual step that the operator does not automate.

cmctl renew -n linkerd linkerd-identity-issuer

Once the new issuer is in place, the operator advances to RenewingWorkloadCerts and publishes an accelerated workload-certificate refresh interval (5 minutes by default). When the data plane operator is also installed, it honors this interval and pushes it into each meshed pod via config.linkerd.io/proxy-additional-env, so workloads renew their identity certificates passively and no explicit restart is needed at this step. See Pairing with the data plane operator for the broader picture.

Without the data plane operator, the accelerated interval has no consumer. At this step you can either wait up to 24 hours for proxies to renew their identity certificates at the normal interval, or restart meshed workloads explicitly so their proxies pick up certificates signed by the new issuer right away:

kubectl -n emojivoto rollout restart deploy

Step 4: Bundle reduction

When workloads have converged onto the new issuer, the operator automatically:

1. Copies the current trust anchor into linkerd-previous-anchor, so the two anchors are realigned.
2. Reduces linkerd-identity-trust-roots to a single root.
3. Rolls the Linkerd control plane onto the reduced bundle.

Phase transitions through RefreshingControlPlane and RemovingOldTrustRoots. A final data plane restart completes the cleanup:

kubectl -n emojivoto rollout restart deploy

The status returns to Synced and linkerd trust inspect reports a clean state.

Inspecting cluster state

linkerd trust inspect is the day-to-day tool for understanding where a cluster is in its rotation lifecycle. It prints the current phase, convergence summary, blocking message (if any), and a list of meshed non-control-plane pods whose trust bundle annotation or live workload certificate has not converged.

linkerd trust inspect
linkerd trust inspect --output json
linkerd trust inspect --concurrency 50 --timeout 10s

The probe-based pod check connects to each pod’s proxy admin port over the mesh, so --concurrency and --timeout let you tune how aggressively the CLI fans out in large clusters.

Observability

The operator publishes Prometheus metrics on its admin endpoint, mirroring the TrustAnchorRotation.status surface so you can alert on rotations that stall or take too long:

• trust_rotation_phase{phase="..."}: 1 for the active phase, 0 for every other phase.
• trust_rotation_trust_bundle_converged_pods and trust_rotation_trust_bundle_target_pods: trust-bundle propagation progress.
• trust_rotation_workload_certificates_converged_pods and trust_rotation_workload_certificates_target_pods: workload-certificate convergence on the active issuer.
• trust_rotation_active_issuer_info{fingerprint="..."} and trust_rotation_previous_anchor_info{fingerprint="..."}: fingerprints of the current active issuer and previous anchor.
• trust_rotation_workload_certificate_refresh_interval_seconds: the accelerated refresh interval currently being pushed into the data plane; 0 when no override is in effect.

Recovery

The operator is designed to be resumable. Phase and progress live in TrustAnchorRotation.status, so an operator restart picks up where it left off without rewinding the workflow. If a rollout stalls, the operator parks in the current phase and surfaces a human-readable blockingMessage on the resource. Once the underlying issue is resolved (for example, a stuck pod is replaced), reconciliation continues from the same point on the next loop.

Learning more

• Feature: Trust anchor rotation
• Guide: cert-manager and trust-manager concepts