Content

MSP Deploys a Production-Ready Kubernetes Environment for ISV in Under 160 Hours

Written by Shadow-Soft Team | Apr 8, 2025 5:47:36 PM

Summary: When a managed services firm needed to migrate a client to the new version of a containerized ISV platform, Shadow-Soft delivered a full deployment, data migration, and upgrade-ready environment in under 160 hours—without the client needing to learn Kubernetes.

Client Overview

The client is a managed services provider that delivers IT infrastructure and platform support to government contractors across the U.S. Their services include hosting, maintenance, and application management for compliance-heavy environments. 

Challenge

The client needed to upgrade a government contractor from a legacy, Windows-based version of a business-critical platform to a new Kubernetes-only release.

However, their internal team had no experience with containerized infrastructure, and their customer expected minimal downtime, full compliance, and a seamless transition.

Without a solution, the client risked falling out of step with vendor requirements, delaying access to new features, and compromising their SLAs. 

Updated CMMC compliance requirements added further pressure to move quickly, increasing the risk of missteps without a proven deployment strategy.

“Even with Kubernetes experience, this would have been difficult. The old system was Windows-based. This was a complete shift. Figure it out on their own could’ve taken a year.”

 — Derrick Sutherland, Chief Architect Shadow-Soft

The Solution

Shadow-Soft delivered a production-grade Kubernetes environment tailored to the client’s infrastructure and compliance needs. 

The project included:

Deployment Automation and Infrastructure Setup

  • Provisioned Kubernetes clusters using Terraform and Ansible across the client’s preferred infrastructure (bare metal, VMware, and cloud).
  • Configured core components, including persistent storage, monitoring, log forwarding, ingress, and backup systems—aligned with CMMC compliance standards.

Migration and Application Readiness

  • Migrated application data and services from the legacy Windows environment to the new Kubernetes-based platform.
  • Coordinated with the client’s database team to align external services with Kubernetes backups for synchronized recovery.
  • Provided deployment options for key components (e.g., containerized database vs. external managed service), based on the client’s internal systems.

Operational Reliability and Customization

  • Enabled future version upgrades to be performed without downtime—demonstrated through multiple real-world updates during the project.
  • Delivered mid-project customization of the logging stack to meet evolving requirements without delaying delivery.

The deployment was completed in under 160 hours, allowing the client to meet compliance timelines and deliver a stable, modernized platform to their end customer—without needing to become Kubernetes experts.

“They need to have a bulletproof way of doing things that is not only tried and true but is simple and straightforward for them—and Kubernetes at scale a lot of times isn’t straightforward.”

Our Process

To deliver a secure, upgrade-ready deployment on a tight timeline, we followed a two-step process focused on infrastructure design and full production rollout. 

Each phase was structured to reduce risk, support compliance, and meet the client’s operational requirements.

Step 1: Deployment Design and Architecture

  • Configured DNS and load balancer settings (Route53, NLB) for managing environment endpoints
  • Assessed infrastructure capacity and hardware requirements to support latest version deployment
  • Defined cluster sizing and a growth plan aligned with the client’s internal objectives
  • Delivered a full project plan with technical documentation for the proposed environment

Step 2: Deployment to EKS

  • Worked alongside the ISV’s team to apply database schema updates and duplicate the existing instance
  • Used Terraform to provision the required infrastructure in the client’s AWS environment
  • Deployed environment and core Kubernetes components using Ansible, including:
    • Longhorn for persistent storage
    • NGINX Ingress Controller
    • MongoDB, Redis, Prometheus, Grafana, Elastic, Fluentd
  • Migrated existing files and MongoDB datasets to the new platform
  • Integrated backup tools, including Velero
  • Delivered a fully deployed EKS cluster running latest version in production

Roadblocks

Midway through the project, the client requested expanded logging capabilities beyond what was included in the original deployment scope. 

We built and integrated a Kubernetes-native logging stack, including Fluentd and Grafana, without disrupting the delivery timeline. Because of the modular nature of the deployment framework, this enhancement was added with minimal overhead or risk.

At the same time, the client was working against a moving compliance target. Updates to CMMC requirements had created pressure to complete the upgrade before new enforcement deadlines took effect. 

While the dates remained fluid, the risk of noncompliance pushed the client to act quickly—making speed and predictability critical throughout the engagement. With a structured rollout system, the project stayed on track and was delivered within the necessary window.

Tools

To support operational stability, visibility, and recovery across environments, we deployed a curated set of open-source and enterprise-grade tools:

  • Fluentd for log collection and forwarding
  • Grafana for visualization of logs and metrics
  • Prometheus for metrics gathering
  • NGINX Ingress Controller for routing and load balancing
  • Longhorn for persistent storage management
  • Velero for backup and recovery
  • Druva and PrismaCloud to meet internal backup and security requirements

These tools were deployed as part of the automated Kubernetes stack, giving the client real-time observability, reliable backups, and secure operations.

Results

Within two weeks (and under 160 hours of engineering time), the client had a fully deployed, production-ready Kubernetes environment running the latest version of their platform.

Because the rollout was automated, tested, and built on reusable components, the deployment met compliance timelines and avoided the operational risks of manual implementation. 

We also completed three live version upgrades during the engagement—all with zero downtime, including one where an end user remained connected throughout the process.

The client now has a stable, scalable system that meets CMMC requirements and can be updated without disruption, freeing their internal team from the burden of managing Kubernetes infrastructure directly.

We also provided complete technical documentation to support internal understanding, future updates, and operational continuity.

Key Results:

  • Deployed a production-ready Kubernetes environment in under 160 hours
  • Completed three real-time version upgrades with zero downtime
  • Met CMMC-driven compliance timelines without delaying project delivery
  • Integrated custom logging stack mid-project without scope creep
  • Provided full documentation to support future updates and internal maintenance

“They’ve had zero issues running in production. And we’ve already done three upgrades for them—all live. One of their users stayed connected the entire time.” — Sutherland

What’s Next

Following the success of this deployment, the client is preparing to onboard additional government customers to the updated platform. The tested rollout process and infrastructure design used in this engagement will serve as a model for future migrations.

While this project was delivered on behalf of a single ISV, the underlying challenge is becoming more common across the market. 

As SaaS providers release Kubernetes-based platforms, many find their customers lack the in-house expertise to deploy or maintain them on-prem. 

Shadow-Soft is now working with other ISVs facing the same constraints, helping them design and deliver stable, compliant environments their customers can confidently operate.