[{"slug":"aws-eks-high-density-pods","title":"AWS EKS high-density pods","date":"2021-11-23T00:00:00.000Z","excerpt":"AWS recently released a new version of the AWS-CNI that allows more Pods per node...","tags":["Cloud","AWS","Kubernetes","CNI"],"html":"

AWS recently released a new version of the AWS CNI that allows more Pods to be\ndeployed within each EC2 instance. Having an higher pod density results in more\nefficiency and a significative reduction in EC2 costs necessary to run the same\nnumber of pods. How does the solution work under the hood? And if it’s that\ngood, why has not been released before?

A bit of background

In order to understand the changes that the new version brings, it is necessary\na bit of background on how intra-node networking works in Kubernetes.

Each time a new pod is created, the kubelet agent delegates:

Creating the container to the Container Runtime Interface
Attaching the container to the network to the Container Network Interface
Mounting volumes to the Container Storage Interface

But for the scope of this post, let’s focus on the CNI part.

Each pod has its own isolated Linux network namespace and is attached to a\nbridge. The CNI is in charge of creating the bridge, assigning the IP and\nconnecting veth0 to the docker0.

AWS is doing it differently

This is how most of the CNIs works and what happens most of the time, but\ndifferent CNIs might use other ways to connect the container to the network.

As an example, there might not be a docker0 bridge; the AWS CNI is an\nexample of such CNI. In order to understand the reason of this choice, let’s\ninvestigate on how EC2 networking works.

In AWS, each EC2 instance can have multiple network interfaces (ENIs) and each\nof them can be assigned a limited number of IPs. For example, an m5.large can\nhave up to 10 IPs for ENI; out of those 10 IPs one of them has to be assigned to\nthe network interface itself while the remaining can be assigned freely.

In the past, these extra IPs were assign to Pods but there was a big limit: the\nnumber of assignable IP addresses for each node.

Let’s have a look at an example.

An m5.large EC2 instance can have up to 3 ENIs with 10 IP private addresses\neach. Since one IP is reserved for the ENI, remains left 9 addresses per ENI and\nthus 27 in total. As a conseuences the EC2 m5.large could run up to 27 Pods,\nnot a lot actually.

Let’s enter prefixes

In August 2021 AWS released a change\nto EC2 that allows prefixes to be assigned to network interfaces.

Prefixes?! In simple words, instead of supporting a single IP address, ENIs now\nsupport an IP range. Thus the same EC2 instance, which had 10 private IP\naddresses, now can have 10 ranges of IP addresses. And how big is the range?\nBy default, 16 IP addresses; so with 10 ranges you could have\nup to 160 IP addresses.

Let’s have a look at another example.

An m5.large can have 3 ENIs with 10 slots each for IPs or prefixes; since\none IP is reserved for the ENI, you’re left with 9 prefixes. Each prefix is 16\nIPs, so 9*16=144 IPs. Since there are 3 ENIs, 144x3=432 IPs.

The same EC2 can now runs up to 432 Pods, not\nonly 27 as before!

AWS CNI 1.9 and Pod-per-node limit

The AWS CNI 1.9.0 includes support to prefixes and\ncaps the max number of Pods to 110 or 250 so you won’t be able to run 432 Pods\non an m5.large; the latter in comòiance with the official Kubernetes large clusters\nguide recommendation. The CNI is also tunable and will prewarm IPs and prefixes\nso that your Pods start instantly.

So… in order to answer the question at the beginning of this post, her’s why\nthe AWS-CNI 1.9.0 release (hopefully) makes more sense now!

","css":{"code":"picture.svelte-1gy5t1o.svelte-1gy5t1o{position:relative;width:100%;height:100%}picture.svelte-1gy5t1o img.svelte-1gy5t1o{width:100%;height:100%}\n.wrapper.svelte-1sgfl7c{background:var(--highlight-marker-url)}","map":null},"head":"","readingTime":"4 min read","relatedPosts":[{"slug":"vmworld-2021-as-speaker","title":"VMworld 2021 as speaker","date":"2021-10-11T00:00:00.000Z","excerpt":"VMworld 2021 has just ended... the first VMworld I took part in... as speaker!","tags":["Events","Cloud","Communities","VMware"],"html":"

VMworld 2021 has just ended! Due to the health worldwide emergency, the event\ntook place virtually. There were three days, from October 5th to 7th, full of\nkeynotes, sessions, round-tables and other social moments like were available\nonline and watchable directly from the people sofa!

Among the other events, my good friend Lino Telera\nand me took a speech about Tanzu and how it can be used on different automation\nscenarios, both on infrastructure and application side.

My first time VMworld… as speaker

Even though it was not the first world tech conference I took part at, it has\nbeen the first time at a VMworld! For sure the priceless experience on\nconference is the possibility of networking (that’s also the most difficult part\nto replicate in a remote experience), the conference surprised me with the\ncompleteness of content for all the technical fields.

Moreover being speaker at a world-wide event, gave me the right excitement\nnecessary to enjoy the conference at best and pushed me to perform better on the\n“virtual stage”! I don’t want to hide my head under the sand: the tension was\nvery high and I felt like the so-called demo effect was hunting for me…

In the end all want for the best; In the next few lines I want to give you a\ngeneral overview of the speech.

Automate DevOps in a Cloud Native Way with VMware Tanzu

The recording of the speech is available on the VMworld On-Demand Video Library.

As a Cloud Native Platform Engineer, automation takes a big part of my\neveryday work, therefore I’ve spent a lot of time researching, designing,\ndeveloping and maintaining systems aimed at automating the work of both\ninfrastructure engineers and software developers.

During the last years, I had the great opportunity to work with Lino Telera and\nbring all my experience on the field to realize a hybrid cloud infrastructure\nintegration and pipelines for continuous integrations &\ndelivery; with the aim to automate the deployment of the\ninfrastructures and the whole applications lifecycle.

In our session we have shown how VMware Tanzu and Tanzu Kubernetes Grid can take\nan important role in the automation field and how they can be integrated with a\nlot of DevOps technologies like Jenkins, Gitlab, Tekton, Flux, and Terraform.

The presentation was oriented not only to infrastructure automation people but\nalso to DevOps pipeline because contained also application development scenarios\nand real-case demos.

What’s next?

That was not all! Another great experience happened during the VMworld days; it\nhas been such a great experience that it’s worth a dedicated blog post… so\nstay tuned!

","css":{"code":"picture.svelte-1gy5t1o.svelte-1gy5t1o{position:relative;width:100%;height:100%}picture.svelte-1gy5t1o img.svelte-1gy5t1o{width:100%;height:100%}\n.wrapper.svelte-1sgfl7c{background:var(--highlight-marker-url)}","map":null},"head":"","readingTime":"3 min read"}]},{"slug":"vmware-code-hackathon-at-vmworld-2021","title":"VMware {code} Hackathon @ VMworld 2021","date":"2021-10-13T00:00:00.000Z","excerpt":"Imagine That! We wan to see what unique projects your creativity leads to!","tags":["Hackathon","Events","Communities","VMware"],"html":"

Imagine That! We wan to see what unique\nprojects your creativity leads to!

This was the theme of the VMware code Hackathon @ VMworld 2021,\nthus practically it was an open theme to allow participants to hack on and work\non whatever they chose to!

At first I was a bit disappointed about taking part in an online-only hackaton;\nI strongly believe that the most exciting part of such events is spending an\nevening meeting new people and staying awake overnight to contribute and make\nyour team reach the expected goal!

However my team mates Lino and\nScott, convinced me that was going to be fun\nanyway, they told me about the idea they had in mind and at that point I could\nnot refuse to partecipate.

So the TanzuFormers team got ready for the battle!

The project

Our project aimed at fully automating the infrastructure setup of a Tanzu\nkubernetes cluster, deploying all the recommended addons for logging,\nmonitonring, etc… and having it automaticlly registered on the Tanzu Mission\nControl dashboard.

The goal was to leverage Terraform to automate the whole process, thus we had to\nimplement several terraform modules each one in charge of deploying a different\naddon to the newly provisioned cluster.

My main task goal was the interaction with Tanzu Mission Control to join the\nfresh new cluster on it; thus I decided to brush again my poor GoLang coding\nskills and decided to develop a custom terraform provider for TMC.

The code for all the modules and the provider is publicy available on our team’s\nGithub organization

And the winner is…

The judges decide to prize our contribution because involved many different\nVMware products and aimed at the integration towards a common goal.

In the end, I enjoyed taking part in this experience more that I expected… the\nfact that it was only online did not reduce its effective value at my eyes.\nOn the contrary, the opportunity to keep in touch with new pepole was extremely\nvalualbe.

With the experience of this yeas online hackaton, I really really can’t wait for\nthe next year hackaton that, hopefully 🤞, will take place in presence at the\nnext VMwolrd!

","css":{"code":".wrapper.svelte-1sgfl7c{background:var(--highlight-marker-url)}\npicture.svelte-1gy5t1o.svelte-1gy5t1o{position:relative;width:100%;height:100%}picture.svelte-1gy5t1o img.svelte-1gy5t1o{width:100%;height:100%}","map":null},"head":"","readingTime":"3 min read","relatedPosts":[{"slug":"vmworld-2021-as-speaker","title":"VMworld 2021 as speaker","date":"2021-10-11T00:00:00.000Z","excerpt":"VMworld 2021 has just ended... the first VMworld I took part in... as speaker!","tags":["Events","Cloud","Communities","VMware"],"html":"

Among the other events, my good friend Lino Telera\nand me took a speech about Tanzu and how it can be used on different automation\nscenarios, both on infrastructure and application side.

My first time VMworld… as speaker

In the end all want for the best; In the next few lines I want to give you a\ngeneral overview of the speech.

Automate DevOps in a Cloud Native Way with VMware Tanzu

The recording of the speech is available on the VMworld On-Demand Video Library.

The presentation was oriented not only to infrastructure automation people but\nalso to DevOps pipeline because contained also application development scenarios\nand real-case demos.

What’s next?

That was not all! Another great experience happened during the VMworld days; it\nhas been such a great experience that it’s worth a dedicated blog post… so\nstay tuned!

","css":{"code":"picture.svelte-1gy5t1o.svelte-1gy5t1o{position:relative;width:100%;height:100%}picture.svelte-1gy5t1o img.svelte-1gy5t1o{width:100%;height:100%}\n.wrapper.svelte-1sgfl7c{background:var(--highlight-marker-url)}","map":null},"head":"","readingTime":"3 min read"}]},{"slug":"vmworld-2021-as-speaker","title":"VMworld 2021 as speaker","date":"2021-10-11T00:00:00.000Z","excerpt":"VMworld 2021 has just ended... the first VMworld I took part in... as speaker!","tags":["Events","Cloud","Communities","VMware"],"html":"

Among the other events, my good friend Lino Telera\nand me took a speech about Tanzu and how it can be used on different automation\nscenarios, both on infrastructure and application side.

My first time VMworld… as speaker

In the end all want for the best; In the next few lines I want to give you a\ngeneral overview of the speech.

Automate DevOps in a Cloud Native Way with VMware Tanzu

The recording of the speech is available on the VMworld On-Demand Video Library.

The presentation was oriented not only to infrastructure automation people but\nalso to DevOps pipeline because contained also application development scenarios\nand real-case demos.

What’s next?

That was not all! Another great experience happened during the VMworld days; it\nhas been such a great experience that it’s worth a dedicated blog post… so\nstay tuned!

","css":{"code":"picture.svelte-1gy5t1o.svelte-1gy5t1o{position:relative;width:100%;height:100%}picture.svelte-1gy5t1o img.svelte-1gy5t1o{width:100%;height:100%}\n.wrapper.svelte-1sgfl7c{background:var(--highlight-marker-url)}","map":null},"head":"","readingTime":"3 min read","relatedPosts":[{"slug":"vmware-code-hackathon-at-vmworld-2021","title":"VMware {code} Hackathon @ VMworld 2021","date":"2021-10-13T00:00:00.000Z","excerpt":"Imagine That! We wan to see what unique projects your creativity leads to!","tags":["Hackathon","Events","Communities","VMware"],"html":"

Imagine That! We wan to see what unique\nprojects your creativity leads to!

This was the theme of the VMware code Hackathon @ VMworld 2021,\nthus practically it was an open theme to allow participants to hack on and work\non whatever they chose to!

However my team mates Lino and\nScott, convinced me that was going to be fun\nanyway, they told me about the idea they had in mind and at that point I could\nnot refuse to partecipate.

So the TanzuFormers team got ready for the battle!

The project

The code for all the modules and the provider is publicy available on our team’s\nGithub organization

And the winner is…

The judges decide to prize our contribution because involved many different\nVMware products and aimed at the integration towards a common goal.

With the experience of this yeas online hackaton, I really really can’t wait for\nthe next year hackaton that, hopefully 🤞, will take place in presence at the\nnext VMwolrd!

","css":{"code":".wrapper.svelte-1sgfl7c{background:var(--highlight-marker-url)}\npicture.svelte-1gy5t1o.svelte-1gy5t1o{position:relative;width:100%;height:100%}picture.svelte-1gy5t1o img.svelte-1gy5t1o{width:100%;height:100%}","map":null},"head":"","readingTime":"3 min read"},{"slug":"aws-eks-high-density-pods","title":"AWS EKS high-density pods","date":"2021-11-23T00:00:00.000Z","excerpt":"AWS recently released a new version of the AWS-CNI that allows more Pods per node...","tags":["Cloud","AWS","Kubernetes","CNI"],"html":"

A bit of background

In order to understand the changes that the new version brings, it is necessary\na bit of background on how intra-node networking works in Kubernetes.

Each time a new pod is created, the kubelet agent delegates:

Creating the container to the Container Runtime Interface
Attaching the container to the network to the Container Network Interface
Mounting volumes to the Container Storage Interface

But for the scope of this post, let’s focus on the CNI part.

Each pod has its own isolated Linux network namespace and is attached to a\nbridge. The CNI is in charge of creating the bridge, assigning the IP and\nconnecting veth0 to the docker0.

AWS is doing it differently

This is how most of the CNIs works and what happens most of the time, but\ndifferent CNIs might use other ways to connect the container to the network.

As an example, there might not be a docker0 bridge; the AWS CNI is an\nexample of such CNI. In order to understand the reason of this choice, let’s\ninvestigate on how EC2 networking works.

In the past, these extra IPs were assign to Pods but there was a big limit: the\nnumber of assignable IP addresses for each node.

Let’s have a look at an example.

Let’s enter prefixes

In August 2021 AWS released a change\nto EC2 that allows prefixes to be assigned to network interfaces.

Let’s have a look at another example.

The same EC2 can now runs up to 432 Pods, not\nonly 27 as before!

AWS CNI 1.9 and Pod-per-node limit

So… in order to answer the question at the beginning of this post, her’s why\nthe AWS-CNI 1.9.0 release (hopefully) makes more sense now!

All posts