The NVIDIA Mellanox MCX651105A-EDAT SmartNIC is a high-performance network interface card that offers unparalleled performance for network-intensive workloads. This SmartNIC leverages advanced network acceleration techniques such as hardware offload and RDMA to provide extremely low latency and low CPU utilization. The firmware version 20.28.4000 enhances the device's performance and adds new features. This post will detail the specifications and features of this device, as well as common issues and resolutions.

Features

• ConnectX-6 Dx SmartNIC
• 200Gb/s Ethernet
• Enhanced Hardware offload engine
• Quick assist for IPsec, SSL/TLS and AES-XTS
• Secure boot and firmware signature validation*
• 8MB Flash (SPI)
• RoHS-R6 complied
• Vision-enabled remote acceleration
• Using NVIDIA SmartNIC accelerator engines (Precision Time Protocol, Universal RDMA, IPsec transport offload, and more.)

Specifications

• Device Type: Network adapter
• Interface (Bus) Type PCI Express Gen 4.0 x16
• Ports: 1x QSFP56-DD
• Data Transfer Rate: 200Gbps bidirectional
• Network / Transport Protocol: TCP/IP, UDP/IP, SMB, HTTP, SMB2, SMB3, NFS, RDMA over Converged Ethernet (RoCE)
• Remote Management Protocol: SNMP 1, SNMP 2, SNMP 3, HTTP, HTTPS
• Features: LSO, FCoE support, RDMA compatibility, Virtual Protocol Interconnect (VPI), Mellanox peer-direct (MPD) architecture

Technologies Used

The NVIDIA Mellanox MCX651105A-EDAT SmartNIC uses different technologies. Some of these technologies include:

• Remote Direct Memory Access (RDMA)
• User Direct Access Memory (UDMA)
• Explicit Congestion Notification (ECN)
• Mellanox Spectrum and ConnectX families of Ethernet switches and adapters

Known Issues and Troubleshooting

The following are common issues that users have encountered with the NVIDIA Mellanox MCX651105A-EDAT SmartNIC:

• Firmware upgrade failures if first stage max size is larger than 3660 bytes.
• Flashing an upgraded firmware with two distinct enclosure names failing.

Users can try the following troubleshooting measures for the issues above:

• Firmware Upgrade
1. Engage the NVIDIA Mellanox technical support team to obtain a firmware image with an improved first-only size that will support image.
2. Use firmware images with v3.00 or newer BootWare in fallback eflash bank by adding 'bfallbkup' line to device tree kernel command line.
• Flashing Failure
1. Ensure that the enclosure name is identical before upgrading the firmware on two different locations in the enclosure to avoid flashing the firmware twice

Device Driver

The NVIDIA Mellanox SmartNIC device driver is mandatory for ensuring optimal performance of the SmartNIC. The driver for this device is supported by a variety of operating systems including Red Hat Linux, SUSE Linux, and VMware ESXi. It is recommended that users always verify compatibility before installing the driver. The driver can be downloaded from the NVIDIA Mellanox website or various linux distribution repositories.