RDMA Performance Tests

Performance testing tools are essential for verifying RDMA functionality and optimizing network performance. The perftest package provides several utilities for measuring latency and bandwidth.

Prerequisites

• Two Nodes: Must run the commands between 2 nodes at the same time.
• Clock Synchronization: Both nodes should ideally have the same CPU clock frequency.
• OFED Utilities: Ensure the perftest package is installed.

Latency Tests

These tests measure the latency of RDMA operations.

Server Side

Run the following on the server node:

# Read latency test
ib_read_lat

# Write latency test
ib_write_lat

# Send latency test
ib_send_lat

Client Side

Run the following on the client node, replacing <Hostname/LID> with the server’s hostname or Local Identifier (LID):

ib_read_lat <Hostname/LID>
ib_write_lat <Hostname/LID>
ib_send_lat <Hostname/LID>

Note: Using the LID directly can avoid potential hostname resolution overhead.

Bandwidth Tests

These tests measure the bandwidth throughput of RDMA operations.

Server Side

Run the following on the server node:

ib_read_bw
ib_write_bw
ib_send_bw

Client Side

Run the following on the client node:

ib_read_bw <Hostname/LID>
ib_write_bw <Hostname/LID>
ib_send_bw <Hostname/LID>