Partitions

Partitions in InfiniBand allow you to isolate nodes into sub-clusters. You can ensure that only certain nodes can communicate with each other, similar to VLANs in Ethernet networks.

Overview

• P_KEY: Each partition is identified by a 16-bit Partition Key (P_Key).
• BTH: The P_Key is contained within the Base Transport Header (BTH). The HCA sets all configured P_Keys in this header.
• Subnet Manager (SM): For a node to join a partition, the Subnet Manager must assign one of its ports a relevant P_Key.
• Switch Enforcement: If an HCA sends a P_Key that isn’t set by the SM, the switch will drop the traffic.
• Port Membership: Ports can be members of multiple partitions. Ports in different partitions are unaware of each other.
• Use Cases: Multi-tenancy, storage/network management, QoS.

Membership Types

A partition contains a group of members. There are two membership types:

1. Full Membership: Can communicate with all other partition members (both full and limited).
2. Limited Membership: Can ONLY communicate with a Full Member. Cannot communicate with other Limited Members.

Use Case Example

Imagine a storage network where you want all servers to talk to the storage array but NOT to each other. You would configure the storage array as a Full Member and the servers as Limited Members.

Determining Membership

Membership type can be determined by examining the leftmost bit of the P_KEY value:

• 0: Limited Member
• 1: Full Member

Example: P_KEY value 65535 (0xFFFF) is 1111 1111 1111 1111 in binary. The leftmost bit is 1, so it indicates Full Membership.

Default Partition

The Subnet Manager always configures a Default Partition.

• P_Key: 0x7fff (32767).
• Membership: All connected ports are Full Members by default.
• Value: Since all ports are Full Members, the effective P_Key value is 0xFFFF (65535) (0x8000 | 0x7FFF).
• Constraints: The default partition cannot be altered or deleted as it controls the configuration pushed to all switches.