VMware NSX-T Install Tips & Tricks

UPDATE on 13 Feb 2021:
There were some changes and improvements with version NSX-T 3.1, so some tips are no longer needed. I’m in the process of proving those notes myself, but it seems NSX EDGE VMs can be migrated now and EDGE VTEPs don’t need a separate subnet from HOST VTEPs anymore.

Intro

It’s a shortlist of things that you should probably know when installing VMware NSX-T. Of course, installing NSX-T should be done by following the official documentation. This here is just a few additional points that could help. It’s for your peace of mind afterward.

This is an article from the VMware from Scratch series

NSX Manager is a Cluster of three VMs

You should end up having three NSX-Manager VMs in a cluster when you finish NSX-T installation. The first one will be deployed via .ovf file from vCenter, the other two direct from first NSX Manager GUI as soon as you connect it to vCenter (aka. adding the Fabric -> Compute Manager)

VMware NSX-T Managers cluster

NSX Manager VMs should not run on the same ESXi host

Use vCenter datacenter configuration VM/host rules (affinity rules) to automatically keep manager VMs running on different hosts on the VMware environment. It’s about the host failing and you still having most of the managers running.

VMware Affinity rules

VMware Affinity rules 2

NSX EDGE VMs should not run on the same ESXi host

Same as above, after you deploy your first EDGE service cluster it is important for redundancy and robustness of the system that two EDGE cluster members (two EDGE VMs) are separated via affinity rules from vCenter so that you don’t lose both of them if one of your ESXi hosts from the cluster fails (just the right one running both EDGE VMs).

NSX EDGE VMs should not be vMotioned around

VMware prefers to suggest that this should be avoided. It actually works if you try it but it can temporarily break the connection from the EDGE to the network and overlay when moved which can be tricky if you then need to wait for BGP peering to get back up and running. Use affinity rules if needed.

NSX Manager administration credentials tend to expire

Check NSX-T manager administrator credentials expiration. It is 90 days by default and afterward, you will not be able to login to the manager GUI or anywhere. It will happen, particularly in stale LAB environments where you will get back to your LAB after a while and find it locked up.

This command will disable password expiration and it’s great for LABs:

clear user admin password-expiration
get user admin password-expiration

In production, you should keep password expiration, but you can extend expiration time:

This one will expand the password validity period to 180 days:

set user admin password-expiration 180
set user root password-expiration 180

Transport Nodes and EDGE VTEPS need separate subnets

Update: If you are going with NSX-T version 3.1 this is no longer a requirement.

Transport Nodes (ESXi hosts) VTEPs need to be from a separate IP subnet from EDGE VMs VTEPs IP subnet. This is true for collapsed transport/compute node clusters where you run all your compute and NSX-T functions on the same cluster.

It’s the architecture of virtual EDGE which, if running on the transport node as a VM, needs a different subnet for its internal virtual N-VDS distributed switch from the subnet used for transport node (NSX enabled ESXi hosts) VTEPs. Then, overlay traffic (Geneve encapsulated L2 or L3 traffic) arriving at the host could be distinguished if it is destined for VMware host virtual kernel adapter for overlay (vmk10-11-12-…) or it needs to be forwarded across host VDS towards N-VDS inside EDGE VM.

VMware NSX-T VTEP pools

 

Leave a Reply