This Kubernetes 101 A Foundational Guide provides readers with Shadow-Soft’s introduction to containers and Kubernetes.
Containers are singular instances of an isolated executable process (Ex. an instance of HTTPD, MariaDB, etc.) running on a Linux-based host operating system. These processes (and all their underlying required packages) are bundled together in an easy-to-manage and archivable format so they can be deployed between disparate systems. This idea isn’t groundbreaking or new and has existed under different names (i.e. Solaris Zones) for many years. However, it wasn’t until the Docker container runtime system was released in 2013 that containers became manageable enough for practical usage.
Unlike virtual machines, containers are built at the process level. Virtual machines can be large, cumbersome, and take a long time to execute in a rapid fashion; additionally, they require the overhead of a virtual hypervisor to emulate and isolate virtual machine instances from one another. Containers instead require only a shared Linux kernel and an OCI-compliant runtime (i.e., CRI-O, Docker, etc.) to run the process on top of the underlying operating system. The isolation is then applied via the usage of Linux Namespaces and CGroups.
The process of managing and deploying containers has been standardized over the last several years, making it safe, consistent, and easy to use. Regarding the usage of Containers, three main concepts need to be understood.
Despite containers providing a lightweight and highly distributable means of packaging and running workloads on individual systems, container runtimes (i.e. Docker, LXC, Podman, etc.) by themselves lack the additional capabilities necessary to orchestrate workloads across various interconnected systems. These missing capabilities include virtual networking, storage coordination, remote management, and scheduling of workloads across various systems while maintaining operational awareness and interconnected system dependency management.
This need warranted the creation of various orchestration platforms over time, including Kubernetes, Mesosphere, Docker Swarm, and many home-grown solutions. Founded by Google and maintained by various large IT organizations (including Red Hat), Kubernetes has long since become the de facto standard for container orchestration.
Abstracting away the underlying infrastructure, Kubernetes acts as a Software Defined Datacenter for containerized workloads. By providing a series of tightly coupled technology solutions, Kubernetes enables workloads to scale dynamically in a standardized way across various Linux systems. Various distributions of Kubernetes have been created, including Amazon EKS, Google GKE, Azure AKS, Red Hat OpenShift, SUSE Rancher, and more; however, the baseline functionality, system constructs, and API definitions have been standardized to create a consistent experience regardless of the selected distribution. The baseline functionality of Kubernetes includes (but is not limited to) the following:
The variations between distributions mostly affect the platform installation, upgrade, management experience, and pre-bundled add-on integrations frequently installed post-deployment. These pre-bundled add-ons vary between distributions but include solutions for functionality such as persistent storage, monitoring, service discovery, log aggregation, ingress networking, and more. These variations are similar to that of various distributions of Linux where the underlying kernel and baseline functions are the same, but the management experience can somewhat differ.
Kubernetes has various standardized object constructs & terms, many of which go beyond the scope of this introduction. However, a brief understanding of a subset of these terms is essential to understanding how Kubernetes orchestrates workloads.
Kubernetes is a Software Defined Datacenter for containerized workloads providing a pool of shared compute resources for all deployed workloads. Many configurable parameters exist that can impact how/where workloads get scheduled as well as actions that trigger deployment. As such it is important to understand the basic mechanics of how Kubernetes allocates & schedules deployments to ensure workloads are designed in a resilient and organized manner.
All deployments running in Kubernetes (including both user-defined and those provided) are orchestrated by the built-in scheduling system. The scheduler watches for new pod creation requests that have not been assigned a node. The scheduler then determines the best fit node for that pod to run on based on the pods individual different requirements. Nodes that meet the minimum scheduling requirements for a pod are considered “feasible” nodes which are scored against a set of predefined functions. Some of the factors taken into consideration include collective resource requirements, hardware/software/policy constraints, affinity / anti-affinity specifications, persistent storage availability, etc. If none of the nodes are considered suitable, the pod remains unscheduled until adequate placement is determined.
The following topics cover some of the most common considerations used during workload scheduling. Understanding these optional configuration settings aims to provide a greater deal of context for considerations when considering how workloads should be designed and deployed.
Many instances exist where workloads actively deployed will change. If a liveliness probe fails, a containerized workload will be considered unhealthy. At this point, the running pod will be terminated, and a new pod will be scheduled to replace the current instance. This new pod will go through the same scheduling process as a new workload, which means the instance will likely be deployed to a different node in the cluster.
Regardless of where the new instance is deployed, the new pod will be provisioned with a new IP address. This happens transparently and is automatically orchestrated behind any defined service object associated with the given deployment. This same behavior will automatically occur should any node go offline, whether due to a restart, failure, or update event.
Furthermore, pod autoscaling exists which can cause the number of pods to increase/decrease based on various metrics (i.e. mem, cpu, etc). It is important to understand that these changes can occur theoretically at any time to ensure deployments are designed to operate in this type of environment.
Kubernetes deployments have very distinct behavior characteristics that need to be considered during software development and deployment alike. Unlike traditional virtual machine workloads which are long-lived and traditionally point-solution specific, Kubernetes deployments are subject to the following:
1. Shared host operating system resources (i.e. Memory, CPU, Filesystem Resources)
2. Shared cluster resources (i.e. Persistent storage, IP allocation, cluster-wide mem/cpu, etc)
3. Frequent workload horizontal scaling events
4. Frequent node horizontal scaling events
5. Frequent changes to IP addresses
6. Frequent process reload/restart events
7. Ephemeral-based by default (including system logging and local storage)
Due to this variance in characteristics, user-defined workloads normally require partial or complete redevelopment to functionally and efficiently operate within a Kubernetes cluster.
Many organizations have the impression that deploying a workload into Kubernetes without any changes will automatically increase their ability to scale to meet the demands of their customers. Though this is possible, the method of scaling, deploying, and orchestrating workloads is so vastly different from traditional environments that frequently Kubernetes will bring to the surface existing issues arising from software development patterns that may have remained unknown previously.
Like many customer instances, a 10-year-old web application and a series of backend web services actively deployed in JBoss EAP 7.2 were set to be migrated to Kubernetes. These various components had gone through various iterations over the years, but the overall architecture remained the same. The front-end application required the usage of sticky sessions from a front-end load balancer to not interrupt user connections, and many of the backend services required stateful data sharing.
Without making any changes to the underlying application and backend services, migration began. Each component (i.e. front-end application, back-end services, messaging brokers, databases, etc) was converted into individual Kubernetes deployments with their own series of pods. Immediate challenges were noticed as user traffic would rotate to different deployed pods in the environment, and no shared caching mechanism was being leveraged. This led to inconsistent behavior (frequent request retries and re-logins required) when user requests were attempted against the frontend application. Furthermore, when an instance of a pod died, or additional pods were added, transactional information was frequently lost.
This additionally became problematic with backend SQL databases when deployment changes occurred as these systems were never designed to have the data remain consistent when frequent environment changes occurred. As such, as scaling events occurred rapidly, data was frequently lost due to synchronization challenges.
Eventually, the entire architecture was rebuilt on stateless microservices. Additionally, front-end shared caching mechanisms were implemented for authentication purposes, and the databases were swapped for systems built from the ground up to run in Kubernetes (i.e. CockroachDB). A Kubernetes-based persistent storage solution (i.e. Portworx) was added to the environment, and a backup solution (i.e. Kasten K10) was implemented to ensure workloads could be safely redeployed in the event of a total cluster failure. This is just one example of why it is crucial that the technology selected and application design be considered prior to beginning any migration efforts to this style of environment.
Various types of user-defined workloads can be deployed within Kubernetes. However, as we have seen in our previous examples, it is important to understand what is a good fit and a bad fit and where design considerations need to be made. The types of workloads that are traditionally a good fit include:
In comparison, various workload types exist that need additional considerations prior to migrating to Kubernetes. The workload types include:
Truly, anything requiring persistent storage needs to be thoroughly tested and evaluated prior to being considered production-ready in Kubernetes. Unlike a traditional hosting environment where storage usage is relatively static and relative to the running host, Kubernetes workloads scale and shift between hosts frequently. This means the underlying storage provider selected must be well understood to ensure that it can seamlessly migrate from one node to another instantaneously and that it is capable of real-time replication to ensure data consistency across all deployment instances.
Additionally, design considerations must be made considering how well these more traditional solutions function during scaling and deployment change events. If for example, a SQL instance has a sudden outage or scale event occurs, how is data integrity guaranteed, and does the data remain consistent across instances? Furthermore, in the event an entire cluster outage occurs, how will the workload be redeployed to a new environment? During these types of windows, will any data loss occur that is acceptable to the business?
Since Kubernetes workloads are subject to scaling events, node updates, networking changes, and more, the lifecycle of a given containerized workload is usually short-lived, lasting anywhere from minutes to hours. Additionally, despite system isolation, due to the unique nature of how containerized workloads are orchestrated, these shared resources can become overcommitted and potentially impact the stability of other workloads running on the same nodes.
For example, if workloads are deployed without the usage of requests/limits or a given workload has a major memory leak, there is a potential that other workloads running on the same system could run out of available resources and become unresponsive. As such, it is important that applications and services deployed on Kubernetes be designed from the ground up to be resilient to frequent system changes.
Because of the complexity of Kubernetes and its various capabilities, using Kubernetes effectively at scale requires a clear line of communication between various teams and a great deal of system governance through automation to reduce the potential for human error when running disparate workloads in production. Disappointingly, there is no one-size-fits-all approach to effectively implementing a governance strategy as the designed methodology will be greatly impacted by a number of factors, including:
1. Cultural / Organization Standards & Maturity
2. Workloads Variations (i.e. Data Science/Machine Learning, Databases, Web Applications, Web Services, etc.)
3. Underlying Infrastructure Features (Node Autoscalers, Availability Zones, etc.)
4. Persistent Storage Requirements
5. Security Requirements
6. Service Level Agreements
7. Available Automation Solutions
When moving to Kubernetes, many organizations not only migrate old workloads without first optimizing their design but continue to operate in very siloed working groups where Development and Operations are completely separated. However, operating in a Kubernetes environment without a clear governance strategy and well-defined communication patterns can often result in catastrophe.
Frequently, developers are given access to Kubernetes environments to administrate workloads they have built and maintained. On the surface, this may not seem like an issue, but because of the way Kubernetes orchestrates deployments, it is possible that systems can become unstable when proper operating procedures aren’t followed.
One situation that frequently comes up is around the usage of CICD as it relates to Kubernetes. In most organizations, the CICD solution (ex. Jenkins) is maintained by Operations, where the pipeline generated is built and maintained by Development teams. In a particular instance, an organization had a Kubernetes cluster running in AWS, with Jenkins contributing CICD jobs to the environment. The Development team created a pipeline that was responsible for deploying a workload, running a series of integration tests, and then cleaning up the workload when completed.
Unfortunately, this pipeline took numerous nodes in a given cluster offline, causing the cluster to be unstable.
The pipeline began by creating a temporary NFS instance, deploying the workload to be tested, mounting the storage and then running the integration test. The problem in this situation was the clean up job, which deleted the temporary NFS instance prior to deleting the workload. Since Kubernetes nodes are Linux instances, when the NFS server got removed, the nodes running the pipeline’s application workload had their file systems locked up in perpetuity, waiting for the NFS share to reconnect. The only way to fix the situation was to either login in as root on the host and forcibly remove the mount or restart the nodes.
The worst problem in this entire scenario is it took weeks to debug the issue because the Operations team was unaware this pipeline job was running in the first place to make the correlation between the job and the environment instability. This created a chaotic situation which crippled workloads at seemingly random times.
Regardless, a Kubernetes strategy should be derived from the perspective that the platform operates as a Software Defined Datacenter. Like all data centers, in the event a system failure or major security events occur, there is a large risk that all data and services deployed within the Kubernetes cluster will be lost (especially when persistent storage is a concern). As such, the organizational usage and governance strategy around Kubernetes should have a well thought and designed plan to ensure system-wide & individual workload stability alike. Additionally, a well-defined security posture and disaster recovery plan for individual workloads is critical.
Considering individual team responsibilities, user-defined workloads are co-located and scheduled amongst a series of shared cluster resources. It is crucial that limits and requests exist on all workloads and that affinity/anti-affinity rules are in place to ensure workloads operate in true HA at all times. This is frequently enforced by ensuring end-users do not have direct access to Production or Staging clusters beyond read access and that the configuration state of workloads is enforced through the usage of Continuous Delivery solutions (i.e. ArgoCD, Fleet, etc) via Service Accounts. These Continuous Delivery jobs should include lint testing rules for the required safety parameters, and a change review board should oversee every code committed to a release branch before it is accepted and deployed to an environment. Additionally, identical Staging environments should exist for testing purposes to not only test the functionality of the user-defined workload but to further understand the impact of these workloads against other workloads running within the same cluster.
Lower-end environments (i.e. Dev/Test environments) can potentially have more flexibility in their usage and governance depending on the organization's needs. However, it is important that shared lower environments maintain operational readiness to not interrupt development efforts. As such, it is frequently recommended that users are provisioned limited access to segmented Kubernetes namespaces and that workloads are isolated enough from other end-users to prevent memory leaks and thread usage issues from causing system-wide stability problems.
For other inquiries or support, please feel free to contact us here! We can help you think through your specific issues and use cases.