CircleCI with Docker and Google Kubernetes Engine

I cannot start without a simple explanation about why my team decided to go with CircleCI, Docker Hub and Google Kubernetes Engine (GKE). Feel free to jump straight to the implementation part if you are not interested in this section.

So why CircleCI? There are many CI tools out there in the market such as Jenkins, Travis, Google build, etc and all of them do the job. In our case, since our current kubernetes cluster is hosted by GKE, we were tempted to use Google Cloud Build because of the benefits of having everything in a single platform plus it offers more free build minutes than CircleCI, among other advantages. However, the team decided to avoid the disadvantages of vendor lock-in with one single cloud provider. Besides, in our opinion, at this time, CircleCI posses a more mature declarative configuration and it is simpler to set up and trace compared to Google Build.

Moving forward with the docker registry, why Docker Hub? Here again, there are many options that do the work, we could have chosen Google registry to keep everything in the same platform and take advantage of it. However, Docker Hub is the most widely used Docker images repository for public images and we trust it. One basically does not need to make any extra configuration to pull or push docker images to a private repository, the experience for this purpose (which is our main purpose) is just smoother. However, if you look for a richer experience, there are other options designed to offer more enterprise-level features.

Finally, why GKE? Just to make sure that I was not biased, I have previously done a set up of kubernetes (K8s) with EC2 instances using KOPS and so on. My experience? I really felt it like a pain in the ass. Having an extra instance to access to the shell and configure ssh connections to the other instances, making extra configurations for ingress controllers with load balancers, having very poor visibility of the resources… all in all, it was not good user experience. Despite all the complications, I was able to finish the implementation. However, I was not completely happy with it, I just felt like “isn’t it an easier way to do this?”.

I moved to GKE and I did the same implementation in a third of the time and I really loved it. The integrated shell, the console app for iOS/Android, the interface to manage resources, the dashboards, the documentation, etc. Google did a great job when it comes to user experience to manage kubernetes clusters. We were lucky that we use just a few services in AWS, therefore it was a great opportunity to proceed with GKE.

The following diagram explains the overall picture of the design. The diagram can be read from left to right: (1) A developer pushes a commit to Github or does a pull request, (2) CircleCI detects this action, pull the code and builds a docker image based on the Dockerfile and some extra configuration done in CircleCI, (3) CircleCI pushes the built docker image(s) to Docker Hub, (4) CircleCI pushes a new K8s yaml to GKE with the new configuration, which will pull the newly built image(s).

Initial Setup

At the begining of the configuration file, under the key “docker”, we define the docker image we want CircleCI to use in order to run our job: “image: google/cloud-sdk”. In this case, it is google/cloud since it already contains the SDK to connect to Google Cloud and a variety of operations we will need to use later.

Next, I will just copy paste some lines from the original post that explain how to configure the service account:

“To ensure that the Service Account has permissions to interact with our cloud resources it should have at least the following permissions:

Note: make sure you select ‘Furnish a new private key’ and choose the key type as JSON. Save this file somewhere as we’ll use it again in a minute.”

Steps

Before moving to the next step it is important to understand that we handle environments in GKE by using namespaces. Therefore, we have to make sure that if a git push was done to a development branch, we need to use the development namespace. On the other hand, if a pull request to master was done, then we need to apply the changes in the production namespace.

For this reason, we use the environment variable called “NAMESPACE”, which will define which kubernetes namespace is going to be used. For instance, in this example, if an action is done on the master branch then the “production” namespace will be used, otherwise (“else”), the development namespace will be used. Here we are considering that we just have two branches, “master” and “development”. But if we would have more branches, it would be better to use an “elif” statement.

The next two instructions are quite straightforward, we just make sure that we are using the right project details while connecting to the GKE:

Finally, we get the credentials we need to interact with our k8s cluster through kubectl. “The get-credentials command will download and setup the necessary kubeconfig files”.

Applying the new configuration to k8s

Before jump into the last step of the job (Deploy to GKE), let’s take a look at our K8s configuration file used for a nodeJS app (which can be used for other kinds of apps too). There are different ways to manage K8s config files. However, this approach requires that at least our deployment configuration is located in the root of our project directory. In this case, we have included both the deployment and the service and we have called it “k8s.yml” which looks as follows:

your-docker-user/gateway:a78f061

Now that we have our new K8s config file, it is time to apply the changes to our GKE. However, we still need to define to which namespace are we going to apply the changes. For this purpose, we are just going to recall the ${NAMESPACE} environment variable previously calculated.

There is nothing else to do here. Your changes are now applied to the desired environment. Finally, if something goes wrong with the deployment in GKE, then we can consider the build as a failed build. For this purpose we add the following final line:

The last section of our configuration file is “workflows”. Here we have defined to filter the builds just for the commits done to the master and dev2019 branches. These are the ones we are going to use and these are the ones considered while building the conditionals for our namespaces.

There is a huge potential when it comes to workflows in Circleci and we could’ve accomplished this pipeline without the use of workflows. However, I thought it was a good thing to include it in the configuration from the begining in case the pipeline gets more complex in the future.

All in all, I am aware that there are many ways to improve this configuration in Circleci. However, I think this is a good start that goes beyond the basics and includes very interesting approaches to make the CI/CD pipeline very dynamic!

I am looking forward to seeing your comments, questions, constructive feedback, and suggestions.

Related posts:

At times human forget the reason why we are human,. “WHAT PATH” is published by Tito.

As a customer-focused organization, we listen carefully to how organizations are dealing with Digital Transformation, especially in regards to how those initiatives are both impacting their overall…

Coming back home for summer break after my first year of college, I was surprised to find a jar on my desk that reminded me of a habit I had in my senior year of high school. I used to take a moment…