System Administration Commands cfgadmib(1M)
NAME
cfgadmib - InfiniBand hardware specific commands for cfgadm
SYNOPSIS
/usr/sbin/cfgadm -f [-y -n] [-v] -c function apid...
/usr/sbin/cfgadm [-f] [-y -n] [-v] [-o hardwareoptions]
-x hardwarefunction apid...
/usr/sbin/cfgadm -v [-a] [-s listingoption] [-] [apid aptype...]
/usr/sbin/cfgadm -v -h [apid]...
DESCRIPTION
The InfiniBand hardware specific library
/usr/lib/cfgadm/ib.so.1 provides the functionality for
administering its fabric through the cfgadm(1M) utility.
cfgadm operates on attachment points. See cfgadm(1M).
An InfiniBand (IB) device is enumerated by the IB nexus
driver, ib(7D), based on the services from the IB Device
Manager (IBDM).
The IB nexus driver creates and initializes five types of
child device nodes:
o IB Port devices
o IB HCA service (HCASVC) devices
o IB Virtual Physical Point of Attachment (VPA) dev-
ices
o I/O Controller (IOC)
o IB Pseudo devices
See ib(7D) for details on enumeration of IB Port, IB VPA,
and IB HCASVC devices. For additional information on IBDM,
see ibdm(7D). See ib(4) for details on IB Pseudo devices.
For IB administration, two types of static attachment point
are created for the fabric administration as seen by the
given host. There is one static attachment point ib and all
SunOS 5.11 Last change: 2 Jul 2008 1
System Administration Commands cfgadmib(1M)
IB devices (either an IOC, Port, VPA, HCASVC, or a Pseudo
device) in the fabric are represented as dynamic attachment
points based off of it. There is another static attachment
point for each Host Channel Adapter (HCA) in the host based
on its node Globally Unique Identifier (GUID) value.
Attachment points are named through apids. There are two
types of apids: logical and physical. The physical apid is
based on the physical path name. For the IB fabric it is
/devices/ib:fabric. The logical apid is a shorter, and has
a more user friendly name.
The static apid for the IB fabric is ib. The IB devices are
dynamic attachment points and have no physical apid. The
logical apid of an IOC contains its GUID, ib::IOC-GUID. An
example of an IOC apid is ib::80020123456789a. The logical
apid of a Pseudo device, see ib(4) for details, is of the
format ib::drivername,unit-address. An example of a pseudo
apid would be ib::sdp,0 where "sdp" is the driver name and
"0" is its unit-address property. The logical apid of Port,
VPA and HCASVC device contains its Partition Key (PKey),
Port GUID / Node GUID and a communication service-name. The
format of apid is as below:
Port device
ib::PORTGUID,0,service-name
VPA device
ib::PORTGUID,PKey,service-name
HCASVC device
ib::HCAGUID,0,servicename
The Partition Key (PKey) is 0 for Port and HCASVC devices.
The PKey helps determine the partition to which this port
belongs for a VPA device node. A port might have more than
one PKey. An example of a VPA device logical apid point
is ib::80245678,ffff,ipib. The port-GUID is 80245678, the
PKey is 0xffff, and the service name is ipib. The service-
name information is obtained from the file
/kernel/drv/ib.conf which contains service-name strings. The
HCA's logical apid contains its node GUID value, hca:HCA-
GUID. An example is hca:21346543210a987.
SunOS 5.11 Last change: 2 Jul 2008 2
System Administration Commands cfgadmib(1M)
A listing of the IB attachment points includes information
on all IB devices (IOC, VPA, HCASVC, Pseudo, and Port dev-
ices seen by the IBDM and the IB nexus driver) in the fabric
even if they are not seen by the host and configured for
use.
The following shows a listing of five IB devices (two IOC,
one VPA, one Port, one HCASVC) and one HCA:
example# cfgadm -al
ApId Type Receptacle Occupant Condition
hca:21346543210a987 IB-HCA connected configured ok
ib IB-FABRIC connected configured ok
ib::80020123456789a IB-IOC connected configured ok
ib::802abc9876543 IB-IOC connected unconfigured unknown
ib::80245678,ffff,ipib IB-VPA connected configured ok
ib::12245678,0,nfs IB-PORT connected configured ok
ib::21346543,0,hnfs IB-HCASVC connected configured ok
ib::sdp,0 IB-PSEUDO connected configured ok
The apid ib::802abc9876543 shows an IOC device that is not
yet configured by the host for use or had been previously
offlined by an explicit
cfgadm -c unconfigure
operation. The distinction was made by the information
displayed under the Condition column. The IB device with a
zero PKey and HCA GUID is a HCASVC device. Refer to
cfgadm(1M) for more information regarding listing attachment
points.
The receptacle state for attachment points have the follow-
ing meanings:
connected
For an IOC/VPA/Port/Pseudo/HCASVC device, connected
implies that it has been seen by the host. The device
might not have been configured for use by Solaris.
For a HCA attachment point, connected implies that it
has been configured and is in use.
SunOS 5.11 Last change: 2 Jul 2008 3
System Administration Commands cfgadmib(1M)
All IB apids are always shown as connected.
The occupant state for attachment points have the following
meanings:
configured
The IB device, and the HCA apid, are configured and
usable by Solaris.
unconfigured
The IB device at the apid was explicitly offlined using
cfgadm -c unconfigure, was not successfully configured.
This could be because it wasn not successfully confi-
guref for use with Solaris (no driver, or a device prob-
lem), or because it was never configured for use by the
IB nexus driver.
The unconfigured operation is not supported for the HCA
attachment point. The IB static apid, ib, is shown
unconfigured if the system has no IB hardware.
The attachment point conditions are:
failed
Not used.
failing
Not used.
ok
Normal state. Ready for use.
unknown
This state is only valid for IB device that have been
probed by IBDM but not yet configured for use by
Solaris. It is also shown for devices that have been
explicitly offlined by a cfgadm -c unconfigure opera-
tion. This condition does not apply to a HCA attachment
SunOS 5.11 Last change: 2 Jul 2008 4
System Administration Commands cfgadmib(1M)
point.
unusable
Not used.
OPTIONS
The following options are supported:
-c function
The IB hardware specific library supports two generic
commands (functions). These commands are not supported
on the static attachment points (that is, the HCA apids
and the IB static ib apid).
The following generic commands are supported:
configure
Configure the IB device to be used by Solaris.
unconfigure
Unconfigure the IB device. If successful, cfgadm
reports the condition of this apid as unknown.
-f
Not supported.
-h apid
Obtain IB specific help for an IB attachment point.
-l
List the state and condition of IB attachment points.
The -l option works as described in cfgadm(1M).
When paired with the -a option, displays the dynamic
attachment points as well (IOC, VPA, Port, Pseudo, and
HCASVC devices).
When paired with -v option, displays verbose data about
SunOS 5.11 Last change: 2 Jul 2008 5
System Administration Commands cfgadmib(1M)
the apids. For an IOC, the Info field in the
cfgadm -avl
output displays the following information: VendorID,
IOCDeviceID, DeviceVersion, SubsystemVendorID, Subsys-
temID, Class, Subclass, Protocol, ProtocolVersion and
IDString from the IOControllerProfile. If the ID string
isn't provided then nothing is displayed in its place.
These fields are defined in the InfiniBand Specification
Volume 1 (http:/www.infinibandta.org).
For a VPA, Port, or HCASVC device the Info field in
the cfgadm -lav display shows the service name informa-
tion to which this device is bound. If no such informa-
tion exists, nothing is displayed.
For a Pseudo device cfgadm -alv displays the driver name
and its unit-address information. For a HCA the verbose
listing displays the VendorID, ProductID of the HCA,
number of ports it has, and the PortGUID value of its
ports. See EXAMPLES.
-o hardwareoption
This option is not currently defined.
-s listingoption
Attachment points of class ib can be listed by using the
select sub-option. Refer to the cfgadm(1M) man page for
more information.
-x hardwarefunction
Perform a hardware specific function. Note that the name
can not be more than 4 characters long.
The following hardware specific functions are supported:
addservice -ocomm=[portvppahcasvc],service=name
This hardware specific function is supported on the
static IB attachment point. It can be used to add a
new service to /kernel/drv/ib.conf file and to
update the ib(7D) driver.
You must use the service=name option to indicate the
SunOS 5.11 Last change: 2 Jul 2008 6
System Administration Commands cfgadmib(1M)
new service to be added. You must use the option
comm=[portvppahcasvc] option to add the name ser-
vice to either port-svc-list or to the hca-svc-list
in the /kernel/drv/ib.conf file. See EXAMPLES.
deleteservice -ocomm=[portvppahcasvc],service=name
This hardware specific function is supported on the
static IB attachment point only. It can be used to
delete an existing service from the
/kernel/drv/ib.conf file and also from the ib(7D)
driver's data base. You must use the service=name
option to indicate which service to delete. You must
use the comm=[portvppahcasvc] option to delete
this service from the port-svc-list, vppa-svc-list,
or vppa-svc-list of the /kernel/drv/ib.conf file.
See EXAMPLES.
listclients
Supported on HCA attachment points. Displays all the
kernel IB clients using this HCA. It also displays
the respective apids of these kernel IB clients and
if they have opened an alternate HCA device. See
EXAMPLES.
.
If a given kernel IB client does not have a valid
apid then a - is displayed in that column.
listservices
This hardware specific function is supported on the
static IB attachment point only. It lists all the
Port and VPA services as read from the
/kernel/drv/ib.conf file. See EXAMPLES.
unconfigclients
This hardware specific function is supported on the
static HCA attachment point only. It can be used to
unconfigure all IB kernel clients of this given HCA.
Only IB kernel clients that do not have an alternate
HCA are unconfigured. See EXAMPLES.
SunOS 5.11 Last change: 2 Jul 2008 7
System Administration Commands cfgadmib(1M)
updateiocconfig
This hardware specific function is supported on
static ib attachment point and the IOC attachment
points. For the ib APID, this function updates pro-
perties of all the IOC device nodes. For the IOC
APID, this function updates the properties of speci-
fied IOC device node. This command updates the
port-list, port-entries, service-id, and service-
name IOC node properties .
See ib(7D).
updatepkeytbls
Supported on the static ib attachment point. Updates
the PKEY information inside IBTL. IBTL re-reads the
PKey tables for all the ports on each HCA present
on the host.
See ibtl(7D).
EXAMPLES
Example 1 Listing the State and Condition of IB Devices
The following command lists the state and condition of IB
devices on the system. It only shows the static attachment
points.
example# cfgadm
hca:21346543210a987 IB-HCA connected configured ok
ib IB-FABRIC connected configured ok
The -a option lists all attachment points. The following
example uses the -a option and lists all attachment points:
example# cfgadm -a
hca:21346543210a987 IB-HCA connected configured ok
ib IB-FABRIC connected configured ok
ib::80020123456789a IB-IOC connected unconfigured ok
ib::80245678,ffff,ipib IB-VPA connected configured ok
ib::21346543,0,hnfs IB-HCASVC connected configured ok
ib::12245678,0,nfs IB-PORT connected configured ok
SunOS 5.11 Last change: 2 Jul 2008 8
System Administration Commands cfgadmib(1M)
ib::sdp,0 IB-PSEUDO connected configured ok
Example 2 Listing the Verbose Status of a IB VPA Device
The following command lists the verbose status of a IB VPA
device:
example# cfgadm -alv ib::80245678,ffff,ipib
ApId Receptacle Occupant Condition Information
When Type Busy PhysId
ib::80245678,ffff,ipib connected configured ok ipib
unavailable IB-VPA n /devices/ib:fabric::80245678,ffff,ipib
A verbose listing of an IOC shows additional information.
The following command shows a verbose listing:
example# cfgadm -alv ib::80020123456789a
ApId Receptacle Occupant Condition Information
When Type Busy PhysId
ib::80020123456789a connected configured ok VID: 0xeaea
DEVID: 0xeaea VER: 0x5 SUBSYSVID: 0x0 SUBSYSID: 0x0 CLAS: 0xffff
SUBCLAS: 0xff PROTO: 0xff PROTOVER: 0x1 IDSTRING: Sample Host Adapter
unavailable IB-IOC n /devices/ib:fabric::80020123456789a
A verbose listing of a Pseudo device shows:
example# cfgadm -alv ib::sdp,0
ApId Receptacle Occupant Condition Information
When Type Busy PhysId
ib::sdp,0 connected configured ok Driver = "sd
p" Unit-address = "0"
unavailable IB-PSEUDO n /devices/ib:fabric::sdp,0
A verbose listing of a HCA shows:
example# cfgadm -alv hca:21346543210a987
SunOS 5.11 Last change: 2 Jul 2008 9
System Administration Commands cfgadmib(1M)
ApId Receptacle Occupant Condition Information
When Type Busy PhysId
hca:21346543210a987 connected configured ok VID: 0x15b3,
PID: 0x5a44, #ports: 0x2, port1 GUID: 0x80245678, port2 GUID: 0x80245679
unavailable IB-HCA n /devices/ib:21346543210a987
You can obtain more user-friendly output if you specify
these following cfgadm class and field selection options: -s
"select=class(ib),cols=apid:info"
The following command displays only IB apids. The output
only includes the apid and Information fields.
# cfgadm -al -s "cols=apid:info" ib::80245678,ffff,ipib
ApId Information
ib::80245678,ffff,ipib ipib
Example 3 Unconfiguring an Existing IB IOC
The following command unconfigures the IB IOC attached to
ib::80020123456789a, then displays the status of the apid:
# cfgadm -c unconfigure ib::80020123456789a
Unconfigure the device: /devices/ib:fabric::80020123456789a
This operation will suspend activity on the IB device
Continue (yes/no)?
Enter: y
IB device unconfigured successfully.
# cfgadm -al ib::80020123456789a
ApId Type Receptacle Occupant Condition
ib::80020123456789 IB-IOC connected unconfigured unknown
#
SunOS 5.11 Last change: 2 Jul 2008 10
System Administration Commands cfgadmib(1M)
The condition unknown implies that the device node doesn't
exist anymore and this IB device's existence is known only
to the IB Device Manager.
Example 4 Configuring an IB IOC
The following series of commands configures an IB device
attached to ib::80020123456789a:
# cfgadm -yc configure ib::80020123456789a
# cfgadm -al ib::80020123456789a
ApId Type Receptacle Occupant Condition
ib::80020123456789a IB-IOC connected configured ok
Example 5 Listing All Kernel IB Clients of a HCA
The following command lists all kernel IB clients of an HCA
attached to hca:21346543210a987:
# cfgadm -x listclients hca:21346543210a987
Attachment Point Clients Alternate HCA
ib::80020123456789a ioc1 Yes
ib::80245678,ffff,ipib ipib No
ib::21346543,0,hnfs hnfs No
- ibdm No
- ibmf No
Example 6 Adding a Port Service
The following command adds a new Port service called srp:
# cfgadm -o comm=port,service=srp -x addservice ib
Example 7 Deleting a VPA Service
The following command deletes the ibd VPA service ibd:
SunOS 5.11 Last change: 2 Jul 2008 11
System Administration Commands cfgadmib(1M)
# cfgadm -o comm=vppa,service=ipib -x deleteservice ib
Example 8 Listing Port, VPA, HCASVC Services
The following command lists all Port, VPA, and HCASVC ser-
vices:
# cfgadm -x listservices ib
Port communication services:
srp
VPA communication services:
ipib
nfs
HCASVC communication services:
hnfs
Example 9 Reprobing IOC Devices
The following command reprobes all IOC device nodes.
# cfgadm -x updateiocconfig ib
This operation can update properties of IOC devices.
Continue (yes/no)?
Enter: y
#
Example 10 Unconfiguring All Kernel Clients of a HCA
The following command unconfigures all kernel clients of a
HCA
# cfgadm -x unconfigclients hca:21346543
This operation will unconfigure clients of this HCA.
Continue (yes/no)?
Enter: y
SunOS 5.11 Last change: 2 Jul 2008 12
System Administration Commands cfgadmib(1M)
FILES
/usr/lib/cfgadm/ib.so.1
Hardware-specific library for generic InfiniBand device
administration
ATRIBUTES
See attributes(5) for descriptions of the following attri-
butes:
ATRIBUTE TYPE ATRIBUTE VALUE
Availability SUNWcsl
SEE ALSO
cfgadm(1M), configadmin(3CFGADM), libcfgadm(3LIB), ib(4),
attributes(5), ib(7D), ibdm(7D), ibtl(7D)
InfiniBand Specification Volume 1
(http:/www.infinibandta.org)
NOTES
Apart from the listing (cfgadm -l or cfgadm -x
listclients), only the superuser can execute any functions
on an attachment point.
SunOS 5.11 Last change: 2 Jul 2008 13
|
