Page 1
Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Corporate Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 526-4100 Text Part Number: 78-17838-01...
Page 2
OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. CCSP, the Cisco Square Bridge logo, Follow Me Browsing, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.;...
IBM Support Contact Information Before You Call Determining the Software Version Downloading the Latest Software Release Cisco Systems Support Contact Information Cisco Technical Support & Documentation Website Submitting a Service Request Definitions of Service Request Severity IBM Statement of Limited Warranty Z125-4753-08 04/2004...
Page 4
FRANCE GERMANY HUNGARY IRELAND SLOVAKIA SOUTH AFRICA, NAMIBIA, BOTSWANA, LESOTHO AND SWAZILAND UNITED KINGDOM Part 3 - Warranty Information Warranty Period Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xvii xvii xviii xviii xviii xviii xxii xxii xxiii...
Page 5
Installing the Server Switch Module Downloading Operating System Software from Cisco.com Configuring the Server Switch Module 78-17838-01 xxvii xxviii xxxi xxxi xxxi xxxii xxxiii xxxiv xxxv Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxii xxxiii xxxiii xxxiii xxxiii xxxiv Contents...
Page 6
Changing the Default Password Configuring the System Clock Connecting Beyond BladeCenter Connecting InfiniBand Cables Specifications C H A P T E R Electrical Requirements Environmental Requirements N D E X Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
Preface Audience The intended audience is the administrator responsible for installing, configuring, and managing Server Switch equipment. This administrator should have experience administering similar networking or storage equipment. Organization This publication is organized as follows: Chapter Chapter 1 Chapter 2...
Cisco Boot over IB User Guide for Linux • Cisco Boot over IB User Guide for Windows • Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Description Series indicate a pop-up menu sequence to open a form or execute a desired function.
IBM and Cisco are both available for more information. For support, contact IBM before Cisco. Note 78-17838-01 Table 1 EU Country Telephone List Italy—9-02-482-9202 Luxembourg—352-360385-1 Netherlands—020-514-5770 Portugal—351-21-7915-147 Spain—34-91-662-4916 Sweden—46-8-477-4420 United Kingdom—01475-555-055 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Contacting Technical Support Table...
Cisco support website at http://www.cisco.com/cgi-bin/tablebuild.pl/sfs-linux. After registering your product, you should have received a username and password to give you access to this site. Cisco Systems Support Contact Information Cisco Technical Support provides 24-hour-a-day award-winning technical assistance. The Cisco Technical Support &...
Severity 1 (S1)—Your network is “down,” or there is a critical impact to your business operations. You and Cisco will commit all necessary resources around the clock to resolve the situation. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Contacting Technical Support...
Page 12
You and Cisco will commit resources during normal business hours to restore service to satisfactory levels. Severity 4 (S4)—You require information or assistance with Cisco product capabilities, installation, or configuration. There is little or no effect on your business operations. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
THE WARRANTY PERIOD. NO WARRANTIES APPLY AFTER THAT PERIOD. SOME STATES OR JURISDICTIONS DO NOT ALLOW LIMITATIONS ON HOW LONG AN IMPLIED WARRANTY LASTS, SO THE ABOVE LIMITATION MAY NOT APPLY TO YOU. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide...
IBM or your reseller will also manage and install selected engineering changes that apply to the Machine. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
IBM is liable for no more than damages for bodily injury (including death) and damage to real property and tangible personal property; and 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 1 - General Terms xiii...
Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the Ordinary Commercial Court of the city of Buenos Aires. BOLIVIA Jurisdiction: The following is added after the first sentence: Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
In accordance with Article 1328 of the Peruvian Civil Code the limitations and exclusions specified in this section will not apply to damages caused by IBM's willful misconduct (“dolo”) or gross negligence (“culpa inexcusable”). 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
The warranties specified in this Section are in addition to any rights you may have under the Trade Practices Act 1974 or other similar legislation and are only limited to the extent permitted by the applicable legislation. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
Governing Law: The following replaces “laws of the country in which you acquired the Machine” in the first sentence: laws of Hong Kong Special Administrative Region of China. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms xvii...
Where Machines are not acquired for the purposes of a business as defined in the Consumer Guarantees Act 1993, the limitations in this section are subject to the limitations in that Act. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xviii...
The warranty for Machines acquired in Western Europe shall be valid and applicable in all Western Europe countries provided the Machines have been announced and made available in such countries. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
Page 22
Commercial Court of Paris; 5) in Russia, all disputes arising out of or in relation to the interpretation, the violation, the termination, the nullity of the Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
Information. You may contact IBM at the following address: IBM Warranty & Service Quality Dept. PO Box 30 Spango Valley Greenock Scotland PA16 0AH 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
AS AN IMMEDIATE CONSEQUENCE OF THE EVENT THAT GENERATED THE DAMAGES; OR 4) LOSS OF BUSINESS, REVENUE, GOODWILL, OR ANTICIPATED SAVINGS. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxii IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
In such instances, regardless of the basis on which you are entitled to claim damages from IBM, IBM is liable for no more than: (items 1 and 2 unchanged). 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms xxiii...
Default. Circumstances may arise where, because of a Default, you are entitled to recover damages from IBM. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxiv IBM Statement of Limited Warranty Z125-4753-08 04/2004...
IBM will accept unlimited liability, subject always to the Items for Which IBM is Not Liable below, for physical damage to your tangible property resulting from the negligence of IBM. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
Country or Region of Purchase Worldwide * See “Types of Warranty Service” for the legend and explanations of warranty-service types. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxvi IBM Statement of Limited Warranty Z125-4753-08 04/2004 Table 1...
CRU and Customer Carry-In or Mail-In Service This type of Warranty Service is a combination of Type 1 and Type 4 (see above). 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 3 - Warranty Information xxvii...
IBM product, and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxix...
Red Hat, the Red Hat “Shadow Man” logo, and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc., in the United States and other countries. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Predictive Failure Analysis...
Recycling facilities may not be available in your area. For information on disposal of batteries outside the United States, go to http://www.ibm.com/ibm/environment/products/batteryrecycle.shtml or contact your local waste disposal facility. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Important Notes xxxi...
This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) this device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxii Notices...
This is a Class A product. In a domestic environment this product may cause radio interference in which case the user may be required to take adequate measures. Taiwanese Class A warning statement 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Electronic Emission Notices xxxiii...
Notices Electronic Emission Notices Chinese Class A warning statement Japanese Voluntary Control Council for Interference (VCCI) statement Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxiv 78-17838-01...
European Union. This label is applied to various products to indicate that the product is not to be thrown away, but rather reclaimed upon end of life per this Directive. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxv 78-17838-01...
Page 38
WEEE Notice Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxvi 78-17838-01...
Server Switch Modules to your BladeCenter chassis and add HCA expansion cards to your BladeCenter hosts, your hosts can communicate to one another over IB within the chassis. When you connect the Server Switch Module to an outside IB fabric, BladeCenter hosts can communicate with all nodes that connect to the IB network.
Cisco 4x InfiniBand Switch Module for IBM BladeCenter Each Server Switch Module includes 16 4x ports to the backplane and 8 4x ports (in the form of 2 4x connectors and 2 12x connectors) on the front panel. The Server Switch Module provides full non-blocking switching for all 24 ports.
With the Server Switch Module and HCA expansion cards, you can create a non-redundant, single-switch topology or a redundant, dual-switch topology. There are four I/O module bays in the switch, labelled 7-10. Bay 7 is for the first InfiniBand interface, Note and bay 9 is for the second InfiniBand interface.
- OK Server Switch Module Status LEDs Module Status LEDs provide an at-a-glance indication of the health of the Server Switch Module. The tables list and describe the states of the Module Status LEDs. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide...
TX/RX External InfiniBand Ports The Server Switch Module has both 4x IB and 12x IB external connectors. The 12x IB connectors support three 4x connections using an "octopus cable." This cable begins as one 12x connector (which connects to your Server Switch Module), then branches into three 4x connectors that you can attach to 4x IB interfaces.
Page 48
Chapter 1 About the Server Switch Module External InfiniBand Ports Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
Ensure that you are properly grounded by attaching the ESD wrist strap to bare skin and attaching the clip on the other end to unpainted metal. Choose the I/O slot into which you want to install the Server Switch Module. You must install the module Step 2 in I/O slot 7 or slot 9.
Step 5 A heading appears, where X is the number of the slot. Step 6 Enter an IP address for the Server Switch Module in the IP address field of the New Static IP Configuration section. Note The IP address of the Server Switch Module must fall in the same subnet as the modules in the other slots.
Chapter 2 Installing and Configuring the Server Switch Open a Telnet session to the Server Switch Module with the IP address that you configured in Step 9 Log in to your Server Switch Module, and then proceed to your Server Switch Module. Log in with the default user name of USERID and password of PASSW0RD, where the 0 in PASSW0RD is a zero.
Page 52
The example that follows sets the date to 1:45 am on June 3rd, 2004: Cisco BladeCenterH# clock set 01:45:00 03 06 04 To configure primary, secondary, and tertiary NTP servers to synchronize the Server Switch with the network, follow these steps: Step 1 Switch to Privileged Exec mode by entering the enable command.
To extend your InfiniBand fabric beyond your BladeCenter cluster, you must configure the priority of the subnet manager on your Server Switch Module, then connect one or more of the external interfaces on your Server Switch Module to an outside InfiniBand fabric.
Page 54
Chapter 2 Installing and Configuring the Server Switch Connecting Beyond BladeCenter Figure 2-1 Removing a Pull Connector Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
0 to 11,000 meters 5% to 85% non-condensing 5% to 95% non-condensing 3.5G for 3 mSec. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide C H A P T E R Requirement 12.2 Volts nominal 6.0 Amps maximum...
Page 56
Storage Vibration Note In the case of non-uniform air flow, the velocity at any point must meet or exceed 150 LFM. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Requirement 0.27G at 5Hz to 500Hz for 30 minutes 50G with a velocity change of 3454 mm/sec squared 0.5G at 2Hz to 200Hz for 15 minutes...
LEDs server switch module initial configuration management ports status LEDs status LEDs subnet mask switch module about switch module status LEDs system clock, setting Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide I N D E X IN-1...
Page 58
Index topology dual-switch single-switch Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IN-2 78-17838-01...