I came across to troubleshoot NSX-T connectivity issues between host transport nodes. My first idea was to check if the NSX-T kernel ports are able to “reach” each other, so I decided to start with a simple ping test. As we’re all familiar, in that case vmkping on the command line is a good command to start with.Initial procedure:
- Login to vCenter Server
- Select the host for the tests
- Navigate to Configure -> Services and start SSH
After logging into the ESXi host, I checked the names of the kernel ports in the NSX-T UI and afterwards did the same on the command line. Of course, we have two options. In the first case I’ll demo both, but later on I’ll use only one.
esxcli network ip interface ipv4 get
In my case the VMKernel port I am looking for is vmk10. Additional please note, that the esxcli command is not showing the netstack, but that is some information we need.
So, this is my infrastructure I know, that the second host has 172.20.11.152 as an ip address on vmk10.
OK, back to start troubleshooting. As I wrote, I am going to use vmkping to ping the Kernel port for NSX-T on the second transport node as I was used to verify features like vMotion. Unfortunately, the following screenshot shows what’s happening
vmk10 is reported as an invalid argument, even though “-I” is used to provide the outgoing VMKernel port (referring to the official VMware knowledge base, since ESXi 5.1). That was unexpected because it always worked perfectly fine, but it seems that the NSX-T configured/created VMKernel ports are a bit different. Checking a bit further and using the help of vmkping, I thought to add the netstack to the command to get it working. Therefor we’ve two options to select the appropriate netstack:
vmkping ++netstack=vxlan <dst IP>
vmkping <dst IP> -S vxlan
Both options are valid and implicitly choosing the correct vmkXX as it can be seen with the TTL.
Still this is not a valid test, because the larger frame size is not specified, but is needed for Geneve traffic. Now we’re going to specify the frame size used for the vmkping command:
vmkping <dst IP> -S vxlan -s 1600
OK, looks quite OK, but there is one more option which is -d. That disallows fragmentation, just to make sure the data doesn’t get fragmented.
vmkping <dst IP> -S vxlan -s 1600 -d
Now it seems that something might not be ideally configured, but I need to go back and read the RFC about Geneve, but this is still a draft version (14) speaking of today.
I’ll update this article later after some additional investigation about DF on IPv4 and if Geneve supports fragmentation or not.
I hope this article was helpful
Thanks for reading!!