# Troubleshooting “linkerd-buoyant”

This page provides resolution steps for common problems reported with the
`linkerd check` command.

## The "linkerd-buoyant" checks

These checks only run when the Buoyant extension is installed. This check
validates that the `linkerd-buoyant` CLI and Buoyant extension are configured
correctly, including the Linkerd lifecycle automation operator and the Buoyant
Cloud agent when present.

### ✅ linkerd-buoyant can determine the latest version

##### ‼️ Warning

```plaintext
‼ linkerd-buoyant can determine the latest version
    Get "https://enterprise.buoyant.io/version.json": dial tcp: lookup enterprise.buoyant.io: no such host
```

##### 🛠️ Remediation

Ensure you can connect to the Buoyant version check endpoint from the
environment where the `linkerd` CLI is running:

```bash
$ curl https://enterprise.buoyant.io/version.json
{"linkerd-buoyant":"v0.27.0"}
```

### ✅ linkerd-buoyant CLI is up-to-date

##### ‼️ Warning

```plaintext
‼ linkerd-buoyant cli is up-to-date
    CLI version is v0.27.1 but the latest is v0.27.4
```

##### 🛠️ Remediation

To update to the latest version of the `linkerd-buoyant` CLI, run:

```bash
curl https://enterprise.buoyant.io/install | sh
```

### ✅ linkerd-buoyant Namespace exists

##### ⛔ Problem

```plaintext
× linkerd-buoyant Namespace exists
    namespaces "linkerd-buoyant" not found
```

##### 🛠️ Remediation

Ensure the `linkerd-buoyant` namespace exists:

```bash
kubectl get ns/linkerd-buoyant
```

If the namespace does not exist, the `linkerd-buoyant` installation may be
missing or incomplete. Follow the
[installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/) to fix.

### ✅ agent-metadata ConfigMap exists

##### ⛔ Problem

```plaintext
× agent-metadata ConfigMap exists
    configmaps "agent-metadata" not found
```

##### 🛠️ Remediation

Ensure that the config map exists:

```bash
$ kubectl -n linkerd-buoyant get cm/agent-metadata
NAME             DATA   AGE
agent-metadata   2      18d
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Visit the [Clusters](https://buoyant.cloud/clusters) page in your
Buoyant Cloud workspace. Select the "View install instructions" menu item for
this cluster and follow the instructions to reinstall.

### ✅ buoyant-cloud-org-credentials Secret exists

##### ⛔ Problem

```plaintext
× buoyant-cloud-org-credentials Secret exists
    secrets "buoyant-cloud-org-credentials" not found
```

##### 🛠️ Remediation

Ensure that the secret exists:

```bash
$ kubectl -n linkerd-buoyant get secret/buoyant-cloud-org-credentials
NAME                            TYPE     DATA   AGE
buoyant-cloud-org-credentials   Opaque   2      18d
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Visit the [Clusters](https://buoyant.cloud/clusters) page in your
Buoyant Cloud workspace. Select the "View install instructions" menu item for
this cluster and follow the instructions to reinstall.

### ✅ buoyant-cloud-org-credentials Secret has correct labels

##### ‼️ Warning

```plaintext
‼ buoyant-cloud-org-credentials Secret has correct labels
    missing app.kubernetes.io/part-of label
```

##### 🛠️ Remediation

If you've configured your agent to rely on an externally-provided credentials
secret, then it's safe to ignore this warning. If that's not the case, then the
`linkerd-buoyant` installation may be missing or incomplete. Visit the
[Clusters](https://buoyant.cloud/clusters) page in your Buoyant Cloud workspace.
Select the "View install instructions" menu item for this cluster and follow the
instructions to reinstall.

### ✅ buoyant-cloud-agent ClusterRole exists

##### ⛔ Problem

```plaintext
× buoyant-cloud-agent ClusterRole exists
    clusterroles.rbac.authorization.k8s.io "buoyant-cloud-agent" not found
```

##### 🛠️ Remediation

Ensure that the cluster role exists:

```bash
$ kubectl get clusterrole/buoyant-cloud-agent
NAME                  CREATED AT
buoyant-cloud-agent   2024-01-12T02:53:26Z
```

Also ensure you have permission to create ClusterRoles:

You should see:

```bash
$ kubectl auth can-i create ClusterRoles
yes
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Visit the [Clusters](https://buoyant.cloud/clusters) page in your
Buoyant Cloud workspace. Select the "View install instructions" menu item for
this cluster and follow the instructions to reinstall.

### ✅ buoyant-cloud-agent ClusterRoleBinding exists

##### ⛔ Problem

```plaintext
× buoyant-cloud-agent ClusterRoleBinding exists
    clusterrolebindings.rbac.authorization.k8s.io "buoyant-cloud-agent" not found
```

##### 🛠️ Remediation

Ensure that the cluster role binding exists:

```bash
$ kubectl get clusterrolebinding/buoyant-cloud-agent
NAME                  ROLE                              AGE
buoyant-cloud-agent   ClusterRole/buoyant-cloud-agent   65d
```

Also ensure you have permission to create ClusterRoleBindings:

You should see:

```bash
$ kubectl auth can-i create ClusterRoleBindings
yes
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Visit the [Clusters](https://buoyant.cloud/clusters) page in your
Buoyant Cloud workspace. Select the "View install instructions" menu item for
this cluster and follow the instructions to reinstall.

### ✅ buoyant-cloud-agent ServiceAccount exists

##### ⛔ Problem

```plaintext
× buoyant-cloud-agent ServiceAccount exists
    serviceaccounts "buoyant-cloud-agent" not found
```

##### 🛠️ Remediation

Ensure that the service account exists:

```bash
$ kubectl -n linkerd-buoyant get serviceaccount/buoyant-cloud-agent
NAME                  SECRETS   AGE
buoyant-cloud-agent   0         83d
```

Also ensure you have permission to create ServiceAccounts:

You should see:

```bash
$ kubectl -n linkerd-buoyant auth can-i create ServiceAccount
yes
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Visit the [Clusters](https://buoyant.cloud/clusters) page in your
Buoyant Cloud workspace. Select the "View install instructions" menu item for
this cluster and follow the instructions to reinstall.

### ✅ buoyant-cloud-agent Deployment exists

##### ⛔ Problem

```plaintext
× buoyant-cloud-agent Deployment exists
    deployments.apps "buoyant-cloud-agent" not found
```

##### 🛠️ Remediation

Ensure the `buoyant-cloud-agent` deployment exists:

```bash
$ kubectl -n linkerd-buoyant get deploy/buoyant-cloud-agent
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
buoyant-cloud-agent   1/1     1            1           83d
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Visit the [Clusters](https://buoyant.cloud/clusters) page in your
Buoyant Cloud workspace. Select the "View install instructions" menu item for
this cluster and follow the instructions to reinstall.

### ✅ buoyant-cloud-agent Deployment is running

##### ⛔ Problem

```plaintext
× buoyant-cloud-agent Deployment is running
    no running pods for buoyant-cloud-agent Deployment
```

##### 🛠️ Remediation

Note, it takes a little bit for pods to be scheduled, images to be pulled and
everything to start up. If this is a permanent error, you'll want to validate
the state of the `buoyant-cloud-agent` Deployment with:

```bash
$ kubectl -n linkerd-buoyant get po --selector app=buoyant-cloud-agent
NAME                                   READY   STATUS    RESTARTS   AGE
buoyant-cloud-agent-54b9476cdd-494c6   2/2     Running   0          13h
```

Check the agent's logs with:

```bash
kubectl -n linkerd-buoyant logs deploy/buoyant-cloud-agent buoyant-cloud-agent
```

### ✅ buoyant-cloud-agent Deployment is injected

##### ‼️ Warning

```plaintext
‼ buoyant-cloud-agent Deployment is injected
    could not find proxy container for buoyant-cloud-agent-54b9476cdd-494c6 pod
```

##### 🛠️ Remediation

Ensure the `buoyant-cloud-agent` pod is injected; the `READY` column should show
`2/2` containers:

```bash
$ kubectl -n linkerd-buoyant get pods --selector app=buoyant-cloud-agent
NAME                                   READY   STATUS    RESTARTS   AGE
buoyant-cloud-agent-54b9476cdd-494c6   2/2     Running   0          34h
```

Make sure that the `linkerd-proxy-injector` deployment is working correctly by
running `linkerd check`.

### ✅ buoyant-cloud-agent Deployment is up-to-date

##### ‼️ Warning

```plaintext
‼ buoyant-cloud-agent Deployment is up-to-date
    incorrect app.kubernetes.io/version label: v0.27.1, expected: v0.27.4
```

##### 🛠️ Remediation

Check the version with:

```bash
$ linkerd-buoyant version
CLI version:   v0.27.4
Agent version: v0.27.4
```

To update to the latest version, visit the
[Clusters](https://buoyant.cloud/clusters) page in your Buoyant Cloud workspace.
Click on the "Upgrade" link for this cluster and follow the instructions.

### ✅ buoyant-cloud-agent Deployment is running a single pod

##### ⛔ Problem

```plaintext
× buoyant-cloud-agent Deployment is running a single pod
    expected 1 buoyant-cloud-agent pod, found 2
```

##### 🛠️ Remediation

`buoyant-cloud-agent` should run as a singleton. Check for other pods:

```bash
$ kubectl get po -A --selector app=buoyant-cloud-agent
NAMESPACE         NAME                                   READY   STATUS    RESTARTS   AGE
linkerd-buoyant   buoyant-cloud-agent-54b9476cdd-494c6   2/2     Running   0          34h
```

### ✅ buoyant-cloud-metrics DaemonSet exists

##### ⛔ Problem

```plaintext
× buoyant-cloud-metrics DaemonSet exists
    daemonsets.apps "buoyant-cloud-metrics" not found
```

##### 🛠️ Remediation

Ensure the `buoyant-cloud-metrics` daemonset exists:

```bash
$ kubectl -n linkerd-buoyant get ds/buoyant-cloud-metrics
NAME                    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
buoyant-cloud-metrics   10        10        10      10           10          <none>          83d
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Visit the [Clusters](https://buoyant.cloud/clusters) page in your
Buoyant Cloud workspace. Select the "View install instructions" menu item for
this cluster and follow the instructions to reinstall.

### ✅ buoyant-cloud-metrics DaemonSet is running

##### ⛔ Problem

```plaintext
× buoyant-cloud-metrics DaemonSet is running
    no running pods for buoyant-cloud-metrics DaemonSet
```

##### 🛠️ Remediation

Note, it takes a little bit for pods to be scheduled, images to be pulled and
everything to start up. If this is a permanent error, you'll want to validate
the state of the `buoyant-cloud-metrics` DaemonSet with:

```bash
$ kubectl -n linkerd-buoyant get po --selector app=buoyant-cloud-metrics
NAME                          READY   STATUS    RESTARTS   AGE
buoyant-cloud-metrics-2lkk6   2/2     Running   0          35h
buoyant-cloud-metrics-5ptzn   2/2     Running   0          35h
buoyant-cloud-metrics-6466b   2/2     Running   0          35h
```

Check the metrics agent's logs with:

```bash
kubectl -n linkerd-buoyant logs ds/buoyant-cloud-metrics buoyant-cloud-metrics
```

### ✅ buoyant-cloud-metrics DaemonSet is injected

##### ‼️ Warning

```plaintext
‼ buoyant-cloud-metrics DaemonSet is injected
    could not find proxy container for buoyant-cloud-metrics-2lkk6 pod
```

##### 🛠️ Remediation

Ensure the `buoyant-cloud-metrics` pods are injected; the `READY` column should
show `2/2` containers:

```bash
$ kubectl -n linkerd-buoyant get po --selector app=buoyant-cloud-metrics
NAME                          READY   STATUS    RESTARTS   AGE
buoyant-cloud-metrics-2lkk6   2/2     Running   0          35h
buoyant-cloud-metrics-5ptzn   2/2     Running   0          35h
buoyant-cloud-metrics-6466b   2/2     Running   0          35h
```

Make sure that the `linkerd-proxy-injector` deployment is working correctly by
running `linkerd check`.

### ✅ buoyant-cloud-metrics DaemonSet is up-to-date

##### ‼️ Warning

```plaintext
‼ buoyant-cloud-metrics DaemonSet is up-to-date
    incorrect app.kubernetes.io/version label: v0.27.1, expected: v0.27.4
```

##### 🛠️ Remediation

Check the version with:

```bash
$ kubectl -n linkerd-buoyant get ds/buoyant-cloud-metrics -o jsonpath='{.metadata.labels}'
{"app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"metrics","app.kubernetes.io/part-of":"linkerd-buoyant","app.kubernetes.io/version":"v0.27.4","helm.sh/chart":"linkerd-buoyant-v0.27.4","linkerd.io/extension":"buoyant"}
```

To update to the latest version, visit the
[Clusters](https://buoyant.cloud/clusters) page in your Buoyant Cloud workspace.
Click on the "Upgrade" link for this cluster and follow the instructions.

### ✅ linkerd-control-plane-operator Deployment exists

##### ⛔ Problem

```plaintext
× linkerd-control-plane-operator Deployment exists
    deployments.apps "linkerd-control-plane-operator" not found
```

##### 🛠️ Remediation

Ensure the `linkerd-control-plane-operator` deployment exists:

```bash
$ kubectl -n linkerd-buoyant get deploy/linkerd-control-plane-operator
NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
linkerd-control-plane-operator   1/1     1            1           84d
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Follow the [installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/) to fix.

### ✅ linkerd-control-plane-operator Deployment is running

##### ⛔ Problem

```plaintext
× linkerd-control-plane-operator Deployment is running
    no running pods for linkerd-control-plane-operator Deployment
```

##### 🛠️ Remediation

Note, it takes a little bit for pods to be scheduled, images to be pulled and
everything to start up. If this is a permanent error, you'll want to validate
the state of the `linkerd-control-plane-operator` Deployment with:

```bash
$ kubectl -n linkerd-buoyant get po --selector app.kubernetes.io/component=linkerd-control-plane-operator
NAME                                            READY   STATUS    RESTARTS   AGE
linkerd-control-plane-operator-f594c6f4-n89s8   1/1     Running   0          36h
```

Check the operator's logs with:

```bash
kubectl -n linkerd-buoyant logs deploy/linkerd-control-plane-operator linkerd-control-plane-operator
```

### ✅ linkerd-control-plane-operator Deployment is up-to-date

##### ‼️ Warning

```plaintext
‼ linkerd-control-plane-operator Deployment is up-to-date
    incorrect app.kubernetes.io/version label: v0.27.1, expected: v0.27.4
```

##### 🛠️ Remediation

Check the version with:

```bash
$ kubectl -n linkerd-buoyant get deploy/linkerd-control-plane-operator  -o jsonpath='{.metadata.labels}'
{"app.kubernetes.io/component":"linkerd-control-plane-operator","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/part-of":"linkerd-buoyant","app.kubernetes.io/version":"v0.27.4","helm.sh/chart":"linkerd-buoyant-v0.27.4","linkerd.io/extension":"buoyant"}
```

To update to the latest version, follow the
[installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/).

### ✅ linkerd-control-plane-operator Deployment is running a single pod

##### ⛔ Problem

```plaintext
× linkerd-control-plane-operator Deployment is running a single pod
    expected 1 linkerd-control-plane-operator pod, found 2
```

##### 🛠️ Remediation

`linkerd-control-plane-operator` should run as a singleton. Check for other
pods:

```bash
$ kubectl get po -A --selector app.kubernetes.io/component=linkerd-control-plane-operator
NAMESPACE         NAME                                            READY   STATUS    RESTARTS   AGE
linkerd-buoyant   linkerd-control-plane-operator-f594c6f4-n89s8   1/1     Running   0          36h
```

### ✅ linkerd-control-plane-validator Deployment exists

##### ⛔ Problem

```plaintext
× linkerd-control-plane-validator Deployment exists
    deployments.apps "linkerd-control-plane-validator" not found
```

##### 🛠️ Remediation

Ensure the `linkerd-control-plane-validator` deployment exists:

```bash
$ kubectl -n linkerd-buoyant get deploy/linkerd-control-plane-validator
NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
linkerd-control-plane-validator   1/1     1            1           84d
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Follow the [installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/) to fix.

### ✅ linkerd-control-plane-validator Deployment is running

##### ⛔ Problem

```plaintext
× linkerd-control-plane-validator Deployment is running
    no running pods for linkerd-control-plane-validator Deployment
```

##### 🛠️ Remediation

Note, it takes a little bit for pods to be scheduled, images to be pulled and
everything to start up. If this is a permanent error, you'll want to validate
the state of the `linkerd-control-plane-validator` Deployment with:

```bash
$ kubectl -n linkerd-buoyant get po --selector app.kubernetes.io/component=linkerd-control-plane-validator
NAME                                              READY   STATUS    RESTARTS   AGE
linkerd-control-plane-validator-b977c7654-smj4k   1/1     Running   0          3d2h
```

Check the validator's logs with:

```bash
kubectl -n linkerd-buoyant logs deploy/linkerd-control-plane-validator linkerd-control-plane-validator
```

### ✅ linkerd-control-plane-validator Deployment is up-to-date

##### ‼️ Warning

```plaintext
‼ linkerd-control-plane-validator Deployment is up-to-date
    incorrect app.kubernetes.io/version label: v0.27.1, expected: v0.27.4
```

##### 🛠️ Remediation

Check the version with:

```bash
$ kubectl -n linkerd-buoyant get deploy/linkerd-control-plane-validator  -o jsonpath='{.metadata.labels}'
{"app.kubernetes.io/component":"linkerd-control-plane-validator","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/part-of":"linkerd-buoyant","app.kubernetes.io/version":"v0.27.4","helm.sh/chart":"linkerd-buoyant-v0.27.4","linkerd.io/extension":"buoyant"}
```

To update to the latest version, follow the
[installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/).

### ✅ linkerd-control-plane-validator Deployment is running a single pod

##### ⛔ Problem

```plaintext
× linkerd-control-plane-validator Deployment is running a single pod
    expected 1 linkerd-control-plane-validator pod, found 2
```

##### 🛠️ Remediation

`linkerd-control-plane-validator` should run as a singleton. Check for other
pods:

```bash
$ kubectl get po -A --selector app.kubernetes.io/component=linkerd-control-plane-validator
NAMESPACE         NAME                                              READY   STATUS    RESTARTS   AGE
linkerd-buoyant   linkerd-control-plane-validator-b977c7654-smj4k   1/1     Running   0          3d2h
```

### ✅ linkerd-data-plane-operator Deployment exists

##### ⛔ Problem

```plaintext
× linkerd-data-plane-operator Deployment exists
    deployments.apps "linkerd-data-plane-operator" not found
```

##### 🛠️ Remediation

Ensure the `linkerd-data-plane-operator` deployment exists:

```bash
$ kubectl -n linkerd-buoyant get deploy/linkerd-data-plane-operator
NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
linkerd-data-plane-operator   1/1     1            1           84d
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Follow the [installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/) to fix.

### ✅ linkerd-data-plane-operator Deployment is running

##### ⛔ Problem

```plaintext
× linkerd-data-plane-operator Deployment is running
    no running pods for linkerd-data-plane-operator Deployment
```

##### 🛠️ Remediation

Note, it takes a little bit for pods to be scheduled, images to be pulled and
everything to start up. If this is a permanent error, you'll want to validate
the state of the `linkerd-data-plane-operator` Deployment with:

```bash
$ kubectl -n linkerd-buoyant get po --selector app.kubernetes.io/component=linkerd-data-plane-operator
NAME                                           READY   STATUS    RESTARTS   AGE
linkerd-data-plane-operator-766d8fd469-hrj8p   1/1     Running   0          3d14h
```

Check the operator's logs with:

```bash
kubectl -n linkerd-buoyant logs deploy/linkerd-data-plane-operator linkerd-data-plane-operator
```

### ✅ linkerd-data-plane-operator Deployment is up-to-date

##### ‼️ Warning

```plaintext
‼ linkerd-data-plane-operator Deployment is up-to-date
    incorrect app.kubernetes.io/version label: v0.27.1, expected: v0.27.4
```

##### 🛠️ Remediation

Check the version with:

```bash
$ kubectl -n linkerd-buoyant get deploy/linkerd-data-plane-operator  -o jsonpath='{.metadata.labels}'
{"app.kubernetes.io/component":"linkerd-data-plane-operator","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/part-of":"linkerd-buoyant","app.kubernetes.io/version":"v0.27.4","helm.sh/chart":"linkerd-buoyant-v0.27.4","linkerd.io/extension":"buoyant"}
```

To update to the latest version, follow the
[installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/).

### ✅ linkerd-data-plane-operator Deployment is running a single pod

##### ⛔ Problem

```plaintext
× linkerd-data-plane-operator Deployment is running a single pod
    expected 1 linkerd-data-plane-operator pod, found 2
```

##### 🛠️ Remediation

`linkerd-data-plane-operator` should run as a singleton. Check for other pods:

```bash
$ kubectl get po -A --selector app.kubernetes.io/component=linkerd-data-plane-operator
NAMESPACE         NAME                                           READY   STATUS    RESTARTS   AGE
linkerd-buoyant   linkerd-data-plane-operator-766d8fd469-hrj8p   1/1     Running   0          3d14h
```

### ✅ controlplanes.linkerd.buoyant.io CRD exists

##### ⛔ Problem

```plaintext
× controlplanes.linkerd.buoyant.io CRD exists
    CRD controlplanes.linkerd.buoyant.io does not exist
```

##### 🛠️ Remediation

Ensure that the CRD exists:

```bash
$ kubectl get crd/controlplanes.linkerd.buoyant.io
NAME                               CREATED AT
controlplanes.linkerd.buoyant.io   2024-01-25T07:38:34Z
```

Also ensure you have permission to create CRDs:

You should see:

```bash
$ kubectl auth can-i create CustomResourceDefinitions
yes
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Follow the [installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/) to fix.

### ✅ dataplanes.linkerd.buoyant.io CRD exists

##### ⛔ Problem

```plaintext
× dataplanes.linkerd.buoyant.io CRD exists
    CRD dataplanes.linkerd.buoyant.io does not exist
```

##### 🛠️ Remediation

Ensure that the CRD exists:

```bash
$ kubectl get crd/dataplanes.linkerd.buoyant.io
NAME                            CREATED AT
dataplanes.linkerd.buoyant.io   2024-01-18T23:56:32Z
```

Also ensure you have permission to create CRDs:

You should see:

```bash
$ kubectl auth can-i create CustomResourceDefinitions
yes
```

If it does not exist, the `linkerd-buoyant` installation may be missing or
incomplete. Follow the [installation guide](/buoyant-enterprise-linkerd/2.15/installation/production/) to fix.
