Tag: datacenter

Google Jupiter Data Center Network Fabric – New Way of Building Data Center Network Underlay

Google’s Datacenter Optical Circuit Switches and Jupiter network fabric

Google’s data centers are unlike any other. It seems they have windows like normal houses because as from the last SIGCOMM’22 presentation, they took their SPINE switches and threw them right out of that window.

Google worked on the Micro Electro Mechanical Systems (MEMS) for years in order to build an Optical Circuit Switch (OCS) that would enable dynamic reconfiguration of optical connections between switches in the data center. Optical Circuit Switch enables on-the-fly data center fabric aggregation block switch connections reconfiguration without the need for physical rewiring. And most interestingly, the usage of Software-Defined Networking (SDN) traffic engineering, enables aggregation block switches to be directly connected and completely removes the need for those bulky Spine switches that were connecting aggregation block switches in CLOS topologies.

OCS MEMS mirror array can redirect light from any input port to any output port by slightly moving one of its mirrors and changing dynamically optical connections between aggregation blocks

Spine switch roles are basically replaced with OCS devices for smart, dynamic and direct interconnections of Leafs. SDN is used for BUM traffic handling (Broadcast, unknown-unicast and multicast traffic) that was the other important spine role.

Google paper (Jupiter Evolving: Transforming Google’s Datacenter Network via Optical Circuit Switches and Software-Defined Networking) that was presented at the SIGCOMM’22 conference describes how getting rid of Spines, and smart traffic handling with SDN, enabled the Jupiter fabric to get up to 5x higher speed and capacity and 40% reduction in power consumption in relation to similar CLOS topology datacenter fabrics.

VMware NSX-T Install Tips & Tricks

UPDATE on 13 Feb 2021:
There were some changes and improvements with version NSX-T 3.1, so some tips are no longer needed. I’m in the process of proving those notes myself, but it seems NSX EDGE VMs can be migrated now and EDGE VTEPs don’t need a separate subnet from HOST VTEPs anymore.

Intro

It’s a shortlist of things that you should probably know when installing VMware NSX-T. Of course, installing NSX-T should be done by following the official documentation. This here is just a few additional points that could help. It’s for your peace of mind afterward.

This is an article from the VMware from Scratch series

NSX Manager is a Cluster of three VMs

You should end up having three NSX-Manager VMs in a cluster when you finish NSX-T installation. The first one will be deployed via .ovf file from vCenter, the other two direct from first NSX Manager GUI as soon as you connect it to vCenter (aka. adding the Fabric -> Compute Manager)

VMware NSX-T Managers cluster

NSX Manager VMs should not run on the same ESXi host

Use vCenter datacenter configuration VM/host rules (affinity rules) to automatically keep manager VMs running on different hosts on the VMware environment. It’s about the host failing and you still having most of the managers running.

VMware TKGI – Deployment of Harbor Container Registry fails with error

This is an article from the VMware from Scratch series

During the process of preparation to Install Tanzu Kubernetes Grid Integrated Edition (TKGI v1.8) on vSphere with NSX-T Data Center (v3.0.2) one of the steps is to use Ops Manager to deploy Harbor Container Registry (in this case v2.1.0).

The process of deployment ended with Harbor error several times so I’m sharing here my solution in order to ease things out for you giving the fact that I didn’t come across any solution googling around.

VMware NSX TKGI K8S

Image from VMware website https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid-Integrated-Edition/index.html

In the process, the Harbor Registry product tile is downloaded from the VMware Tanzu network portal, imported in the Ops Manager installation dashboard, and selected to be configured and prepared for deployment into the VMware environment.

Software-defined data center and what’s the way to do it

SDDC – Software-Defined Data Centers

Times of Software Defined everything has long since arrived, the need to implement many appliances, two or more for each network function, is not so popular anymore. The possibility to manage packet forwarding, load balancing and security of network traffic inside the datacenter from one simple web console is showing finally that things can be managed in a simpler way after all. All vendors in the networking world tried to come up with their own way of centralizing data center management, as it ends up, all of them did it, some better than the others. As always, it’s not a surprise that some vendors are better in creating hardware-based forwarding solutions and some others in software solutions (in this case, software for packet forwarding).

Requirements

It seems that we have basically only a few good options when wanting to select a complete SDDC solution. The data center needs to provide a large number of server access ports in the form of networking devices that are configured and managed as simply and promptly as possible. Datacenter network needs to be configured in a way to provide robustness and stability of packet forwarding at almost line rate and all that at 10-100, even 400Gbps speeds.

Cisco ACI – Configuring by POSTing JSON

If you are configuring Cisco ACI datacenter fabric it will sooner or later get to the point that you need to configure multiple objects inside the GUI which will, by using the click-n-click method, take a huge amount of time.

While using POSTMAN to create multiple objects of the same type is the preferred method that everybody is speaking about (because you can generate REST API calls using Python or something similar), the quickest way to do it is using POST of JSON configuration file directly through the GUI.

POSTing JSON config example

As described above, the POST of JSON for some simple yet repetitive configuration is the way to go. Let’s see how it’s done:

Creating multiple BDs inside a tenant in Cisco ACI: