Table of Contents
Intro
It’s a shortlist of things that you should probably know when installing VMware NSX-T. Of course, installing NSX-T should be done by following the official documentation. This here is just a few additional points that could help. It’s for your peace of mind afterward.
This is an article from the VMware from Scratch series
NSX Manager is a Cluster of three VMs
You should end up having three NSX-Manager VMs in a cluster when you finish NSX-T installation. The first one will be deployed via .ovf file from vCenter, the other two direct from first NSX Manager GUI as soon as you connect it to vCenter (aka. adding the Fabric -> Compute Manager)
NSX Manager VMs should not run on the same ESXi host
Use vCenter datacenter configuration VM/host rules (affinity rules) to automatically keep manager VMs running on different hosts on the VMware environment. It’s about the host failing and you still having most of the managers running.
NSX EDGE VMs should not run on the same ESXi host
Same as above, after you deploy your first EDGE service cluster it is important for redundancy and robustness of the system that two EDGE cluster members (two EDGE VMs) are separated via affinity rules from vCenter so that you don’t lose both of them if one of your ESXi hosts from the cluster fails (just the right one running both EDGE VMs).
NSX EDGE VMs should not be vMotioned around
VMware prefers to suggest that this should be avoided. It actually works if you try it but it can temporarily break the connection from the EDGE to the network and overlay when moved which can be tricky if you then need to wait for BGP peering to get back up and running. Use affinity rules if needed.
NSX Manager administration credentials tend to expire
Check NSX-T manager administrator credentials expiration. It is 90 days by default and afterward, you will not be able to login to the manager GUI or anywhere. It will happen, particularly in stale LAB environments where you will get back to your LAB after a while and find it locked up.
This command will disable password expiration and it’s great for LABs:
clear user admin password-expiration get user admin password-expiration
In production, you should keep password expiration, but you can extend expiration time:
This one will expand the password validity period to 180 days:
set user admin password-expiration 180 set user root password-expiration 180
Transport Nodes and EDGE VTEPS need separate subnets
Update: If you are going with NSX-T version 3.1 this is no longer a requirement.
Transport Nodes (ESXi hosts) VTEPs need to be from a separate IP subnet from EDGE VMs VTEPs IP subnet. This is true for collapsed transport/compute node clusters where you run all your compute and NSX-T functions on the same cluster.
It’s the architecture of virtual EDGE which, if running on the transport node as a VM, needs a different subnet for its internal virtual N-VDS distributed switch from the subnet used for transport node (NSX enabled ESXi hosts) VTEPs. Then, overlay traffic (Geneve encapsulated L2 or L3 traffic) arriving at the host could be distinguished if it is destined for VMware host virtual kernel adapter for overlay (vmk10-11-12-…) or it needs to be forwarded across host VDS towards N-VDS inside EDGE VM.
