EDP Standard: Architecture walkthrough of the ESXi datapath under NSX

EDP Standard is the packet-processing engine described for an ESXi host running as an NSX Transport Node. The architecture divides traffic handling into clear layers: ingress (workloads and host services), a virtual switching fabric, a datapath core with a fast path and fallback, thread scheduling, NUMA-aware queueing, and the physical driver/hardware boundary.

High-performance fast-path processing is available only to paravirtualized VMXNET3 adapters. Virtual machines and containerized pods that expose vNIC0 or Eth0 via VMXNET3 can reach the EDP fast path. Workloads using emulated or legacy adapters (for example E1000 or E1000E) cannot use EDP's fast path and instead continue on the IOChain slow path.

vSphere host services (vmk adapters) are also processed by the EDP engine. The VMkernel adapters (vmk0–vmk4) that carry management, vMotion, vSAN, replication, and TEP overlay encapsulation are part of the same EDP-managed domain.

Datapath core: fast path and IOChain fallback

Engine is the primary pipeline for forwarding: it accepts VLAN and overlay packets, uses a compact 128-byte Mbuf framework (cutting memory footprint compared to previous structures), and looks up flows in an EDP Flow Cache. Once a flow is learned on its initial packet, subsequent packets that match the signature bypass the full network stack.

If a packet represents an uncached flow, requires first-packet setup, needs control-plane resolution, or is a diagnostic packet, EDP routes it to the legacy IOChain fallback. IOChain resolves forwarding rules and the result is used to populate the Flow Cache so later packets use the fast path.

Thread scheduling and load balancing

EDP assigns dedicated ESXi execution threads (referred to as EnsNetWorlds) to poll and process network queues. A Thread Load Balancer monitors thread utilization and redistributes active flows across CPU cores to reduce contention and avoid hotspots.

Memory, queuing, and NUMA alignment

EDP Queue Manager maps packet queues to system hardware with strict NUMA awareness. Receive Side Scaling (RSS) queues are bound to local NUMA sockets so packets arriving on NIC ports tied to NUMA 0 are processed using memory and threads on socket 0. This reduces cross-socket interconnect latency. A default queue RSS pool serves as a fallback when dedicated hardware queues are saturated.

Physical boundary: drivers and hardware offloads

• To get EDP performance, use VMXNET3 for VMs and ensure pods use paravirtualized interfaces where supported. Emulated adapters will remain on the slow path.
• Verify pNIC drivers listed as EDP-capable for your hardware to ensure hardware offloads and queue binding work as intended.
• NUMA alignment matters: NIC-to-socket binding and RSS configuration affect latency and CPU utilization.