HCA Firmware Upgrade

The Host Channel Adapter (HCA) is the network interface card that connects a host to the InfiniBand fabric. It implements all software verbs defined in the IB specification.

Firmware Overview

HCA firmware provides the low-level instructions for the adapter to perform its functions. Firmware is typically installed or upgraded automatically as part of a DOCA-OFED installation, but there are cases where a separate upgrade may be needed:

1. The DOCA-OFED version and the firmware version are out of sync.
2. An issue was observed during the firmware installation process.
3. The firmware is corrupted or has known bugs that need to be resolved.

Nvidia MFT Package

The Mellanox Firmware Tools (MFT) package is used to manage HCA firmware. It can be installed as part of MLNX_OFED / DOCA-OFED or separately.

Key Tools

Tool	Purpose
MST (Nvidia Software Tools Service)	Provides the device interface for firmware operations.
flint (Flash Interface)	Burns firmware images and queries firmware attributes.
mlxfwreset	Resets firmware on the device to activate a new image.

Flint Capabilities

• Burns a binary firmware image to flash storage on adapters, bridges, or switches.
• Burns expansion ROM images to ConnectX family devices.
• Queries firmware attributes (version, GUIDs, MACs, PSID).
• Executes various operations on flash memory from the command line.
• Enables/disables access to hardware registers.

For the full MFT documentation, see the Nvidia MFT User Manual.

Firmware Upgrade Procedure

Step 1: Identify the HCA Type

Determine which HCA(s) are installed on the system.

lspci | grep Mellanox

This returns the model and type of the adapter (e.g., ConnectX-6, ConnectX-7).

Step 2: Gather Device Information

Query the HCA for details needed to select the correct firmware.

ibv_devinfo

Key fields to note:

• hca_id: Identifies which HCA you are upgrading.
• fw_ver: Current firmware version.
• vendor_part_id: Used to match the HCA to its firmware download.
• board_id (PSID): The Parameter-Set Identification. Critical for selecting the correct firmware image.

Step 3: Download the Firmware

1. Navigate to the Nvidia Firmware Downloads page.
2. Select the HCA type (e.g., ConnectX-6).
3. Select the firmware version.
4. Download the .zip file.

Step 4: Burn the Firmware

Unzip the downloaded firmware image.

unzip <firmware-filename>.zip

Identify the full device path of the HCA using mst status.

sudo mst status

Use the vendor_part_id from Step 2 to match the correct device in the mst status output.

Burn the firmware image to the HCA using flint.

sudo flint -d <full-path-hca> -i <firmware-bin-file> b

• -d <full-path-hca>: The full device path from mst status (e.g., /dev/mst/mt4123_pciconf0).
• -i <firmware-bin-file>: The unzipped .bin firmware file.
• b: The “burn” command.

Step 5: Finalize the Upgrade

Reset the HCA firmware to activate the new image.

mlxfwreset -d <hca-id> reset

Verify the new firmware version is active.

ibstat

The fw_ver field should now reflect the updated firmware version.