Although there are a lot of instructions available, I haven't found a straightforward way of deploying a container to Kubernetes cluster that is hosted in a private ECR registry. In this short article, I would like to share a sequence of steps that can be used to perform the deployment.
Prerequisites
Make sure that the machine that is going to be used to perform the deployment (whether it's your local or most likely CI/CD environment) does have aws-cli installed and configured with proper access key id and secret access key so that it has access to pull an image. More info in the official AWS article.
Overview
As a good practice, it's always best to deploy a different set of applications in separate namespaces. We would create namespace manually via kubectl create
command (for the reasons I explain later).
From the application standpoint, we would consider a minimalistic "health check application" that replies with status: ok
as an HTTP response. Kubernetes manifest file would consist out of the following objects:
- service (exposed via Nodeport)
- pod (containing an application)
manifest.yml
:
apiVersion: v1
kind: Service
metadata:
name: health-check-service
namespace: health-check
spec:
type: NodePort
ports:
- port: 3000
selector:
app: health-check-app
---
apiVersion: v1
kind: Pod
metadata:
name: health-check
namespace: health-check
labels:
app: health-check-app
spec:
containers:
- name: health-check
image: <aws-account-id>.dkr.ecr.<aws-region>.amazonaws.com/<image-name>:<tag>
imagePullPolicy: Always
ports:
- name: web
containerPort: 3000
imagePullSecrets:
- name: regcred
Besides a familiar look of service and pod definition, there are a couple of items that are needed to be highlighted:
- ECR Image Registry URL:
<aws-account-id>.dkr.ecr.aws-region.amazonaws.com/<image-name>:<tag>
<aws-account-id>
- your account id, e.g.e9ae3c220b23
<aws-region>
- aws region name (examples here)<image-name>
- image name<tag>
- image tag, usually defines a version or simply uselatest
- Image Pull Policy: Always enforce image force pull to avoid unexpected issues when k8s doesn't pull an image from a remote repository.
Deployment steps
Create namespace via
kubectl create
command:The reason we create namespace manually and not in the above manifest file is that in the next step we would have to create a secret within this namespace. This is super important since kubernetes secrets are scoped to a specific namespace.
Next, the secret is generated via a command line using
aws ecr
that is outside of "kubectl" ecosystem.Create a registry secret within the above namespace that would be used to pull an image from a private ECR repository:
kubectl create secret docker-registry regcred \ --docker-server=<aws-account-id>.dkr.ecr.<aws-region>.amazonaws.com \ --docker-username=AWS \ --docker-password=$(aws ecr get-login-password) \ --namespace=health-check
This command would utilize aws-cli
aws ecr get-login-password
and save the generated credentials in a specialdocker-registry
secret type. More info about it in the official kubernetes docs.Please note, that username is always set as
AWS
for all accounts.Deploy manifest file using
kubectl apply -f
command:
Using the http
command I can verify that my deployment is working:
$ http <cluster-ip>:<node-port>
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 15
Content-Type: application/json; charset=utf-8
Date: Mon, 15 Mar 2021 16:47:54 GMT
ETag: W/"f-VaSQ4oDUiZblZNAEkkN+sX+q3Sg"
Keep-Alive: timeout=5
X-Powered-By: Express
{
"status": "ok"
}
One-liner for CI/CD pipeline
If you need to automate the deployment via CI/CD or simply just would line to use one-line command, here it is:
NAMESPACE_NAME="health-check" && \
kubectl create namespace $NAMESPACE_NAME || true && \
kubectl create secret docker-registry regcred \
--docker-server=<aws-account-id>.dkr.ecr.<aws-region>.amazonaws.com \
--docker-username=AWS \
--docker-password=$(aws ecr get-login-password) \
--namespace=$NAMESPACE_NAME || true && \
kubectl apply -f manifest.yml
Clarification:
NAMESPACE_NAME
variable is pulled out for reuse. It's also scoped only to a current bash session....|| true
bash statement is added to ignore errors for already created space and secret when reusing the same command to re-apply changes from the manifest file.
Conclusion
This article provides a basic and production-ready approach of how to deploy a private-hosted application to a Kubernetes cluster. It can scale to any number of pods/replica-sets/deployments as long as they reside in the same namespace. It's also possible to repeat the same steps for each namespace.