EKS Platform Engineering — Part 3: Cost Optimization with Karpenter & Spot Instances

Replaced the Cluster Autoscaler and managed node groups on EKS with Karpenter for intelligent, cost-aware node provisioning. Karpenter reads what each pending pod actually needs and provisions the cheapest instance that fits — across multiple instance families, in real time. Combined with Spot instances and automatic node consolidation, the result is a significantly cheaper and faster-scaling cluster with no changes to application code.

Why I Built This

The Cluster Autoscaler is the default answer for EKS autoscaling, but it scales fixed node groups. You define an instance type upfront, and that is all you get. If pods need 2 CPUs and the node group runs m5.xlarge with 4 CPUs, two CPUs are wasted every time a node is added. Karpenter eliminates that by reading pod requirements directly and provisioning the cheapest instance that fits — across multiple instance families, considering real-time Spot pricing. The economics are meaningfully better and the provisioning is significantly faster.

Prerequisites

• A running EKS cluster — version 1.33 or above

• Helm installed

• AWS CLI configured with permissions to create IAM roles, SQS queues, and EC2 instances

• kubectl configured against the cluster

Karpenter vs Cluster Autoscaler

The difference is not just speed — it is the mental model. Cluster Autoscaler thinks about node groups. Karpenter thinks about pods. It reads pod requirements directly and uses them to find the cheapest instance type with available Spot capacity. The more instance families you allow, the better Karpenter can optimise.

How It Works

Two Karpenter resources define everything. The NodePool sets what Karpenter is allowed to provision — instance families, capacity types, resource limits, and consolidation behaviour. The EC2NodeClass handles the AWS-specific details: AMI family, subnets, and security groups.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    spec:
      requirements:
      - key: karpenter.sh/capacity-type
        operator: In
        values: ["spot", "on-demand"]
      - key: node.kubernetes.io/instance-type
        operator: In
        values: ["m6i.large", "m6i.xlarge", "m5.large", "m5.xlarge",
                 "c6i.large", "c6i.xlarge", "c5.large", "c5.xlarge"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
  limits:
    cpu: 100
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 30s

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2
  role: YOUR_NODE_ROLE_NAME
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: YOUR_CLUSTER_NAME
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: YOUR_CLUSTER_NAME

Spot Interruption Handling

Spot instances can be reclaimed with two minutes notice. Karpenter handles this through an SQS queue — AWS publishes interruption notices to EventBridge, which forwards them to the queue, which Karpenter watches. When a notice arrives, Karpenter cordons the node, drains pods gracefully, and reschedules them before the interruption window closes. Event-driven rather than polling — Karpenter acts on the notice immediately.

# Create the interruption queue
aws sqs create-queue --queue-name karpenter-interruption-YOUR_CLUSTER_NAME

# Install Karpenter with interruption handling enabled
helm install karpenter oci://public.ecr.aws/karpenter/karpenter   --version 1.0.0   --namespace karpenter --create-namespace   --set settings.clusterName=YOUR_CLUSTER_NAME   --set settings.interruptionQueue=karpenter-interruption-YOUR_CLUSTER_NAME

Key Decisions

Multiple instance families, not one. Allowing Karpenter to choose across m6i, m5, c6i, and c5 means it can always find available Spot capacity. Spot availability varies by instance type and availability zone — the more types you allow, the less likely you hit capacity constraints and fall back to on-demand.

Spot first, on-demand fallback. Listing Spot before on-demand means Karpenter always tries Spot first. On-demand is only used if Spot capacity is unavailable. Maximum savings without removing the reliability safety net.

Consolidation after 30 seconds. When a node becomes underutilised, Karpenter waits 30 seconds then reschedules its pods onto other nodes and terminates it. A shorter window removes idle capacity faster. Worth tuning per environment — aggressive consolidation on stateful workloads causes unnecessary disruption.

SQS for interruptions, not IMDS polling. The event-driven approach via SQS means Karpenter receives the interruption notice as soon as AWS publishes it. For a two-minute window, the difference between event-driven and polling matters.

Keep system workloads off Karpenter nodes. CoreDNS, kube-proxy, and the ingress controller run on tainted on-demand nodes that Karpenter does not manage. System infrastructure on stable capacity, application workloads on Karpenter.

Results

MetricBeforeAfterCompute costOn-demand only40% reduction — Spot mixNode provisioning time3–4 minutes~45 secondsSpot interruption handlingManualAutomatic — graceful drain via SQSNode consolidationManualAutomatic — idle nodes removed in 30sInstance flexibility1 fixed type per node group8 instance types considered per pod

What I Learned

Karpenter's effectiveness is directly tied to how loosely pods are constrained. A pod that specifies only CPU and memory gives Karpenter maximum flexibility to find cheap Spot capacity. A pod with a specific node selector, topology spread constraints, and required labels leaves Karpenter very little room. Before deploying Karpenter, audit pod specs — the configuration is the easy part.

Consolidation also needs careful thought. Karpenter respects pod disruption budgets when draining nodes for consolidation — but if deployments don't have PDBs configured, it can reschedule too many pods simultaneously and briefly reduce capacity below safe levels. Setting PDBs on all application deployments before enabling consolidation is the right order of operations.

What I'd Do Differently

I'd configure NodePool weights to route different workload types to different instance categories — batch jobs to cheaper Spot-only pools, latency-sensitive services to a more stable mixed pool. One pool for everything works, but separate pools with appropriate constraints give more control over where workloads land and make cost attribution cleaner. Karpenter docs →