Connect Kubernetes Clusters

This guide walks you through connecting a Kubernetes cluster to Onam — any CNCF-conformant distribution: EKS, AKS, GKE, OKE, ACK, IKS, or self-managed. You create a read-only ServiceAccount bound to a get/list/watch ClusterRole, then register the API server endpoint, CA certificate, and token in the Onam console. The connection is agentless and read-only: nothing is installed in the cluster, and Onam never modifies any object.

Time to complete: ~10 minutes per cluster. Kubernetes knowledge required: kubectl, RBAC basics.

The Onboarding view in the Onam console (demo account)
The Onboarding view in the Onam console (demo account)

How the connection works

Onam talks directly to the Kubernetes API server using a dedicated ServiceAccount whose ClusterRole allows exactly three verbs: get, list, and watch. At scan time, Onam enumerates cluster objects — workloads, RBAC bindings, network policies, secrets metadata, admission configuration — and evaluates them against 718 Kubernetes rules covering 51 resource kinds, plus 103 identity-focused CIEM rules for RBAC analysis.

  • Nothing runs in your cluster. No DaemonSet, no operator, no sidecars — Onam is a read-only API client, so there is zero workload overhead.
  • RBAC is the enforcement boundary. The ClusterRole you apply has no create, update, patch, or delete verb, and the Kubernetes API server rejects anything beyond it.
  • Revocation is one command. Delete the ServiceAccount (or its namespace) and access ends instantly.
How the connection stays read-only: the ClusterRole in this guide grants only get, list, and watch — the Kubernetes API server's admission path enforces that on every request, so Onam cannot create, modify, or delete any object even in error. The ServiceAccount token is stored in AWS Secrets Manager, encrypted with KMS, and kubectl delete ns onam-system severs access immediately.

Before you begin

  • kubectl access to the cluster with permission to create namespaces, ServiceAccounts, and ClusterRoleBindings (typically cluster-admin)
  • A cluster running Kubernetes 1.24 or later
  • Network reachability: the API server endpoint must be reachable from Onam — either a public endpoint restricted to Onam's egress IPs (shown on the onboarding screen), or private connectivity on Enterprise plans
  • An Onam account with at least the tenant_admin role

Step 1 — Create read-only cluster credentials

Apply this manifest. It creates a namespace, a ServiceAccount, a read-only ClusterRole, the binding, and a long-lived token Secret:

# onam-scanner.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: onam-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: onam-scanner
  namespace: onam-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: onam-scanner-readonly
rules:
  - apiGroups:
      - ""
      - "apps"
      - "batch"
      - "networking.k8s.io"
      - "rbac.authorization.k8s.io"
      - "policy"
      - "storage.k8s.io"
      - "autoscaling"
      - "apiextensions.k8s.io"
      - "admissionregistration.k8s.io"
    resources: ["*"]
    verbs: ["get", "list", "watch"]
  - nonResourceURLs: ["/version", "/healthz"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: onam-scanner-readonly
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: onam-scanner-readonly
subjects:
  - kind: ServiceAccount
    name: onam-scanner
    namespace: onam-system
---
apiVersion: v1
kind: Secret
metadata:
  name: onam-scanner-token
  namespace: onam-system
  annotations:
    kubernetes.io/service-account.name: onam-scanner
type: kubernetes.io/service-account-token
kubectl apply -f onam-scanner.yaml
The ClusterRole includes read access to Secrets so Onam can flag risky patterns — stale ServiceAccount tokens, secrets mounted into privileged pods, plaintext credentials in ConfigMaps. Onam evaluates secret metadata and never stores secret values. If your policy forbids this, replace resources: ["*"] in the core API group with an explicit list that omits secrets — the related checks are then skipped and marked as such in reports.

Retrieve the token, CA certificate, and endpoint

# API server endpoint
APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
echo "API server: $APISERVER"

# ServiceAccount token (long-lived, bound to the Secret above)
TOKEN=$(kubectl -n onam-system get secret onam-scanner-token \
  -o jsonpath='{.data.token}' | base64 -d)
echo "$TOKEN"

# Cluster CA certificate
kubectl -n onam-system get secret onam-scanner-token \
  -o jsonpath='{.data.ca\.crt}' | base64 -d > onam-ca.crt

Step 2 — Register the cluster in the Onam console

  1. In the Onam console, navigate to Onboarding → Connect Cloud Account
  2. Select Kubernetes
  3. Fill in the fields below
  4. Click Validate Connection — Onam performs a get /version and a namespace list to verify read access
FieldValue
Cluster Aliasany friendly name, e.g. prod-eks-us-east-1
API Server URLthe $APISERVER value from Step 1
DistributionEKS, AKS, GKE, OKE, ACK, IKS, or self-managed
CA Certificatecontents of onam-ca.crt
ServiceAccount Tokenthe $TOKEN value from Step 1

Run your first scan

  1. Navigate to Onboarding → Cloud Accounts
  2. Click the cluster you just added
  3. Click Run Scan Now — a cluster scan typically completes in 5–20 minutes depending on object count

Findings appear in the console graded Critical, High, Medium, Low, or Info, and flow into the Container Security views alongside image and runtime findings.

Managed clusters: EKS, AKS, and GKE

Cluster onboarding complements — not replaces — cloud account onboarding. The cloud connection sees the managed control plane from the outside; the cluster connection sees the objects inside it. Connect both for full coverage:

DistributionCloud account connection coversCluster connection adds
EKS (connect AWS)Cluster config, node groups, IRSA roles, security groupsWorkloads, RBAC bindings, NetworkPolicies, Pod Security admission
AKS (connect Azure)Cluster config, node pools, managed identities, NSGsWorkloads, RBAC bindings, NetworkPolicies, Pod Security admission
GKE (connect GCP)Cluster config, node pools, Workload Identity, firewall rulesWorkloads, RBAC bindings, NetworkPolicies, Pod Security admission

When both connections exist, Onam's Container Security engine correlates them — for example, linking an over-privileged IRSA role (cloud side) to the pod that mounts it (cluster side), which also feeds attack path analysis.

Enable audit-log ingestion for CDR

Onam's CDR engine ingests Kubernetes audit logs through the Discovery & Inventory pipeline and runs three detection tiers over them: single-event rules (L1), multi-event correlation scenarios (L2), and statistical behavior baselines (L3). Audit logging is off by default on most distributions — enable it per platform:

DistributionHow to enable audit logs
EKSEnable audit in control-plane logging (logs land in CloudWatch, read via the AWS connection)
AKSAdd a diagnostic setting for the kube-audit category to a storage account
GKECloud Audit Logs are on by default; confirm Data Access logs for the Kubernetes Engine API
Self-managedSet --audit-log-path or an audit webhook on the API server, shipping to object storage

Detections include anonymous API access, exec into production pods, ClusterRoleBinding escalation, and secret enumeration bursts.

What gets scanned

Onam evaluates 51 Kubernetes resource kinds against 718 posture rules, plus 103 CIEM rules focused on RBAC and identity — part of the 10,000+ rule registry spanning all seven supported clouds.

AreaResource kinds (examples)Example checks
WorkloadsPods, Deployments, StatefulSets, DaemonSets, Jobs, CronJobsPrivileged containers · hostPath mounts · missing resource limits · latest-tag images
RBAC & identityRoles, ClusterRoles, RoleBindings, ServiceAccountsWildcard verbs · cluster-admin bindings · auto-mounted tokens
NetworkServices, Ingresses, NetworkPoliciesNamespaces without default-deny · public LoadBalancers · missing TLS on Ingress
Secrets & configSecrets, ConfigMapsCredentials in ConfigMaps · stale ServiceAccount tokens · unencrypted etcd (self-managed)
Cluster & admissionNamespaces, Nodes, PodSecurity admission, webhooksAnonymous auth enabled · missing Pod Security standards · permissive admission webhooks

Troubleshooting

ErrorCauseFix
401 UnauthorizedToken invalid or Secret recreatedRe-extract the token from onam-scanner-token and update it in the console
Connection timed outAPI server not reachable from OnamAllowlist Onam's egress IPs on the endpoint, or use private connectivity
x509: certificate signed by unknown authorityWrong or truncated CA certificateRe-extract ca.crt from the token Secret and re-paste the full PEM
403 Forbidden on some kindsClusterRole missing an API groupRe-apply the manifest from Step 1 unchanged
Scan finds 0 workloadsClusterRoleBinding subject mismatchVerify the binding's subject namespace is onam-system
No audit-based CDR findingsAudit logging not enabledEnable audit logs per the CDR section above

Next steps

  • Container Security — image scanning, runtime posture, and K8s RBAC analysis on this cluster.
  • CIEM — effective-permission analysis across RBAC bindings and cloud IAM.
  • CDR — behavioral detection over the audit logs you just enabled.
  • Book a demo — walk through your first cluster scan with an Onam engineer.