Skip to main content

Building EKS (Amazon hosted Kubernetes) clusters using eksctl

 


Disclaimer

I have published this post on my work blog https://reece.tech previously.

Overview

Eksctl acts as a wrapper around CloudFormation templates. Creating a cluster will add one stack for the control plane (EKS master servers) and one stack for each node group configured (a node group is a group of workers using the same networking and sizing as well as IAM permissions).

However, certain actions such as upgrading the Kubernetes master or worker version or scaling out the number of workers in a node group does not always update the CF stacks associated with it.

Preparation

Download and install the latest version of eksctl.

Follow the Weaveworks installation guide: https://eksctl.io/introduction/installation/.

Download eksctl (Linux)

curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp

Install eksctl (Linux)

sudo mv /tmp/eksctl /usr/local/bin

Provide AWS credentials

Ensure the AWS credentials are set for your current session. The easiest way is to run (aws configure) with the required region and access keys - however as things get more complicated (multiple accounts, assume role and mfa), additional scripts or applications may be required.

Planning

Our first EKS test cluster should be simple. Just running eksctl create cluster will create a new VPC, Internet Gateway, subnets and all other resources required for initial testing.

However, this cluster will be available externally (with EC2 instances / workers and the API server endpoint exposed to the internet). Ideally we would like to utilise existing networks and VPCs as well as adhering to existing security policies and regulations.

Assuming we decided on a setup with three internal subnets to be used for workers and (initially) an external EKS control plane / API server endpoint. We create a resource definition to be used with eksctl:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: myeks
  version: '1.13'
  region: ap-southeast-2
 
# Role used by the EKS control plane itself when managing resources
iam:
  serviceRoleARN: "arn:aws:iam::123412341234:role/basic/eks-cluster-service-role"
 
# Where to deploy the control plane endpoints, and the worker nodes
vpc:
  id: vpc-12341234123412345
  subnets:
    private:
      ap-southeast-2a: { id: subnet-12341234123412345 }
      ap-southeast-2b: { id: subnet-12341234123412346 }
      ap-southeast-2c: { id: subnet-12341234123412347 }

nodeGroups:
  - name: mynodes
    instanceType: r4.large
    desiredCapacity: 3
    privateNetworking: true
    securityGroups:
      withShared: true
      withLocal: true
    iam:
      instanceProfileARN: "arn:aws:iam::123412341234:instance-profile/basic/eks-cluster-iam-NodeInstanceProfile-1PSA1WKT5RP16"
      instanceRoleARN: "arn:aws:iam::123412341234:role/eks-cluster-node-instance-role"
    ssh:
      allow: true
      publicKeyPath: test

Creation

eksctl create cluster --config-file config.yaml --kubeconfig $HOME/.kube/config.eks

This takes around 12 minutes after which the worker nodes are running and showing as Ready in K8s:

$ kubectl get nodes -o wide
NAME                                               STATUS   ROLES    AGE   VERSION              INTERNAL-IP     EXTERNAL-IP     OS-IMAGE         KERNEL-VERSION                  CONTAINER-RUNTIME
ip-192-168-12-18.ap-southeast-2.compute.internal   Ready    <none>   4m    v1.13.7-eks-c57ff8   192.168.12.18   52.64.78.71     Amazon Linux 2   4.14.123-111.109.amzn2.x86_64   docker://18.6.1
ip-192-168-40-73.ap-southeast-2.compute.internal   Ready    <none>   4m    v1.13.7-eks-c57ff8   192.168.40.73   13.211.179.67   Amazon Linux 2   4.14.123-111.109.amzn2.x86_64   docker://18.6.1

Scaling

We can easily scale the cluster out and back in again.

eksctl scale nodegroup --cluster=myeks --name=mynodes --nodes=6
eksctl scale nodegroup --cluster=myeks --name=mynodes --nodes=2

Upgrades

Upgrading only upgrades to the next available higher version, so no version info is necessary.

eksctl update cluster --name=myeks --approve

To upgrade the workers / node pools, simply create a new pool and remove the existing one after.

eksctl create nodegroup --config-file config.yaml # create new node group
eksctl delete nodegroup --config-file config.yaml --only-missing # remove old node group

Deletion

eksctl delete cluster --name=myeks
Labels:

Comments

Post a Comment

Popular posts from this blog

Manual Kubernetes TLS certificate renewal procedure

Intro Kubernetes utilizes TLS certificates to secure different levels of internal and external cluster communication.  This includes internal services like the apiserver, kubelet, scheduler and controller-manager etc. These TLS certificates are created during the initial cluster installation and are usually valid for 12 months. The cluster internal certificate authority (CA) certificate is valid for ten years. There are options available to automate certificate renewals, but they are not always utilised and these certs can become out of date. Updating certain certificates may require restarts of K8s components, which may not be fully automated either. If any of these certificates is outdated or expired, it will stop parts or all of your cluster from functioning correctly. Obviously this scenario should be avoided - especially in production environments. This blog entry focuses on manual renewals / re-creation of Kubernetes certificates. For example, the api-server certificate below...
Post a Comment
Read more

Deprecating Networking Ingress API version in Kubernetes 1.22

  Intro Kubernetes deprecates API versions over time. Usually this affects alpha and beta versions and only requires changing the apiVersion: line in your resource file to make it work. However with this Ingress object version change, additional changes are necessary. Basics For this post I am quickly creating a new cluster via Kind (Kubernetes in Docker) . Once done, we can see which API versions are supported by this cluster (version v1.21.1). $ kubectl api-versions | grep networking networking.k8s.io/v1 networking.k8s.io/v1beta1 Kubernetes automatically converts existing resources internally into different supported API versions. So if we create a new Ingress object with version v1beta1 on a recent cluster version, you will receive a deprecation warning - and the same Ingress object will exist both in version v1beta1 and v1. Create $ cat ingress_beta.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: clusterpirate-ingress spec:   rules:  ...
Post a Comment
Read more

Analysing and replaying MySQL database queries using tcpdump

Why There are situations where you want to quickly enable query logging on a MySQL Database or trouble shoot queries hitting the Database server in real-time. Yes, you can enable the DB query log and there are other options available, however the script below has helped me in many cases as it is non intrusive and does not require changing the DB server, state or configuration in any way. Limitations The following only works if the DB traffic is not encrypted (no SSL/TLS transport enabled). Also this needs to be run directly on the DB server host (as root / admin). Please also be aware that this should be done on servers and data you own only. Script This script has been amended to suit my individual requirements. #!/bin/sh tcpdump -i any -s 0 -l -w - dst port 3306 | strings | perl -e ' while(<>) { chomp; next if /^[^ ]+[ ]*$/;   if(/^(ALTER|COMMIT|CREATE|DELETE|DROP|INSERT|SELECT|SET|UPDATE|ROLLBACK)/i) {     if (defined $q) { print "$q\n"; }     $q=$_; ...
Post a Comment
Read more