By default, a user would use an ECS Instance when deploying applications on Alibaba Cloud. This usually is the go-to solution of most people. An ECS Instance in Alibaba Cloud is a Virtual Machine that give you a complete Linux/Windows machine for you to use. This is freedom as its best, and you are the sole responsible person for everything happening inside. Such freedom comes with a huge responsibility.

Why in this time of using Serverless functions we still use Virtual Machines to run Containers? A container only needs, at least, 2 things: CPU and Memory. And it only needs them only while its running. So… why not create type of instance that can grab some CPU from here and some Memory from there and run a Container? Welcome to ECI (Elastic Container Instance).

An ECI will mimic the lifecycle of a Container and will only exist when the Container is running and disappear when is not. This type of instance is perfect to run as a Node in a Kubernetes Cluster and is just what Alibaba Cloud is offering since 2018: Serverless Kubernetes or ASK.

How Alibaba Cloud leverages ECI for Kubernetes?

We mentioned before that an ECI Instance will mimic the lifecycle of a Container and this is where it comes very handy when combined with Kubernetes. Alibaba Cloud uses ECI like Nodes and they are dynamically created with the Pod’s needs.

Let’s say you setup a “Deployment” and then, for the container template, you set “1” in the cpu field and “8Gi” in the memory one. When the Pod (or Pods) comes to life, Alibaba Cloud will just provide an ECI with those requirements while the Pod exists. And everything within seconds.

Back to our problem… no Internet!

Everything sounds perfect but… by default an ECI does not come with Internet Access and everything inside the Serverless Cluster will be isolated from the outside world, it will only be capable of receiving requests and connecting to other resources within the VPC network or peered VPCs. If your application needs to send requests to an external API, well… it cannot. The common solution to this is to create a NAT Gateway inside the VPC where the Cluster lives. The problem? For most of deployments, using a NAT Gateway is overkill and expensive.

The solution, for cheap, is to use EIP (Elastic IP). At the end of the day, ECI follows the same network rules than ECS and, technically speaking, an EIP is also a NAT IP address that is provisioned in a public-facing gateway of Alibaba Cloud and is mapped to the associated cloud resource with NAT. After an EIP is associated with a cloud resource, the cloud resource can connect to the Internet by using this EIP.

Binding an EIP to a Serverless Kubernetes Pod

You can bind an EIP to a Pod in a serverless cluster running on a virtual node (ECI). In this case, you can deploy applications in a simplified manner. Just be sure the Security Group attached to the resulting ECI has the needed ports exposed.

For this, we will use the swiss knife for tricks in Kubernetes: Annotations. Annotations are those fields in a template used to store arbitrary non-identifying metadata in objects. In our case we will need to set an annotation called “k8s.aliyun.com/eci-with-eip” to “true” at the Pod level.

Another cool thing is that you can set “k8s.aliyun.com/eip-bandwidth” to specify the bandwidth of the ECI, by default that value will be 5 Mbit/s.

Remember, if you are launching Pods by using the Deployment object, you need to set the annotations inside the metadata field of the container template.

With Kubernetes default YAML

The default way to create templates in Kubernetes is by using YAML files, here you have the example:

annotations:
  "k8s.aliyun.com/eci-with-eip": "true"

With Terraform

For those of you, like me, who are using Terraform to manage their Kubernetes clusters, here is the config piece:

annotations = {
  "k8s.aliyun.com/eci-with-eip" = "true"
}

Conclusion

We learnt today how to get Internet access in a Serverless Kubernetes when our budget is limited and our scaling needs are not too crazy. Each Pod will get its own EIP, so be careful as cost can get high with many Pods.

Obviously, if your deployment has many Pods, this solution doesn’t scale well as a NAT Gateway eventually would be cheaper than assigning EIPs to all the Pods. In that case I don’t recommend you to use EIPs.