Page 1
IBM System Storage SAN768B Installation, Service, and User Guide Service information: 2499-384 Read Before Using This product contains software that is licensed under written license agreements. Your use of such software is subject to the license agreements under which they are provided.
Page 3
IBM System Storage SAN768B Installation, Service, and User Guide Service information: 2499-384 GA32-0574-05...
Read this first Summary of changes This is the sixth edition of the IBM System Storage SAN768B Installation, Service, and User Guide. The content changes since the last edition of this publication are noted by a vertical line placed in the left margin beside each change. Minor edits are not identified by this mark.
Web site: www.redbooks.ibm.com/. For more information about IBM SAN products, see the following Web site: www.ibm.com/servers/storage/san/ For support information for this and other IBM products, see the IBM Support Portal, www.ibm.com/supportportal. Search for the product Machine type or product name.
Your feedback is important in helping us provide the most accurate and high-quality information. If you have comments or suggestions for improving this document, send us your comments by e-mail to starpubs@us.ibm.com . Be sure to include the following: v Exact publication title...
Page 8
SAN768B Installation, Service, and User Guide...
Tables ....xiii Chapter 3. Starting and configuring the Safety and environmental notices . . . xv SAN768B ....31 Safety notices and labels . . xv Providing power to the SAN768B .
Page 10
Removing and installing a cable management comb 70 Preparing for a chassis replacement . . 110 Time and items required . . 70 Disconnecting the SAN768B from the network Removing a cable management comb. . 70 and fabric . 113 Installing a cable management comb .
Port side of the SAN768B (sample Filler panel removal and installation . . 76 configuration) . Removing the control processor (CP8) blade Blower (non-port) side of the SAN768B (sample Removing the core switch blade. . 83 configuration) . FR4-18i router blade and components .
Page 14
SAN768B Installation, Service, and User Guide...
Critical information checklist example Blades available for the SAN768B Sample cable routing table for SAN768B Security features . Cable routing table for SAN768B (64 ports Items supplied with the 14U rack mount kit shown). . 120 Port blades available on the SAN768B.
Page 16
SAN768B Installation, Service, and User Guide...
IDs, which are displayed in parentheses, for example (D004), at the end of each notice. Use this ID to locate the translation of these danger and caution notices in the IBM System Storage b-type Switch and Router Safety Notices publication, which is shipped with this product.
Page 18
(D004) A general electrical danger notice provides instructions on how to avoid shock hazards when servicing equipment. Unless instructed otherwise, follow the procedures in the following danger notice. SAN768B Installation, Service, and User Guide...
Page 19
Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard: v Connect power to this unit only with the IBM provided power cord. Do not use the IBM provided power cord for any other product.
Delivery and subsequent transportation of the equipment The customer should prepare his environment to accept the new product based on the installation planning information provided, with assistance from an IBM Installation Planning Representative (IPR) or IBM authorized service provider. In...
Page 21
Read and comply with the following caution notices before installing or servicing this device. CAUTION: Energy hazard present. Shorting may result in system outage and possible physical injury. Remove all metallic jewelry before servicing. (C001) CAUTION: The weight of this part or unit is more than 55 kg (121.2 lb). It takes specially trained persons, a lifting device, or both to safely lift this part or unit.
CAUTION: For CA residents only: IBM recommends installing this product in a room size of 62 cubic meters (2190 cubic feet) or larger at 0.4 ACH ventilation rate to reduce the concentrations of any chemicals emitted by the product. Safety labels As an added precaution, safety labels are often installed directly on products or product components to warn of potential hazards.
Attention: Many of the field replaceable units (FRUs) are sensitive to electrostatic discharge (ESD), and can potentially be damaged by improper handling. Wear a wrist grounding strap connected to chassis ground (if the SAN768B is plugged in) or a bench ground. Store all ESD-sensitive components in antistatic packaging.
Attempting to move the drawer partially or completely out of the rack might cause the rack to become unstable or cause the drawer to fall out of the rack. (R001 part 2 of 2) xxii SAN768B Installation, Service, and User Guide...
Rack relocation (19" rack) CAUTION: Removing components from the upper positions in the rack cabinet improves rack stability during relocation. Follow these general guidelines whenever you relocate a populated rack cabinet within a room or building: v Reduce the weight of the rack cabinet by removing equipment starting at the top of the rack cabinet.
Internal machine checks Perform the following internal machine checks: 1. Check for any non-IBM changes that might have been made to the machine. If any are present, obtain the “Non-IBM Alteration Attachment Survey” form, number R009, from the IBM branch office. Complete the form and return it to the branch office.
Product recycling and disposal Refer to the IBM Systems Environmental Notices and User Guide (Z125-5823) on the product documentation CD for translated environmental statements and information regarding product recycling and disposal. This document may be provided either in printed version or on the product documentation CD. See “Removing the batteries”...
Page 28
SAN768B Installation, Service, and User Guide...
The following documents contain information related to this product. The documentation may be printed material, on the documentation CD that is shipped with the product, or available on the web through the IBM Support Portal or IBM Publications Center. v IBM System Storage SAN768B Installation, Service, and User Guide, GA32-0574 (this document, which is also available in accessible HTML format on the documentation CD).
When you use any of the Brocade documents, you will notice that the model numbers reflect the original Brocade products. Table 1 provides a product matrix for you to use to correlate the Brocade products and models to the IBM product names and machine types and model numbers. Products withdrawn from marketing are not listed.
Page 31
Vendor software The SAN768B includes certain vendor software that is not covered under the IBM license agreement. IBM makes no representation about the accessibility features of these products. Contact the vendor for the accessibility information about its products. Related accessibility information You can view the publications for the SAN768B in Adobe Portable Document Format (PDF) using the Adobe Acrobat Reader.
Page 32
SAN768B Installation, Service, and User Guide...
This chapter introduces the features and components of the IBM System Storage SAN768B fabric backbone. Throughout the remainder of this document, the product will be referred to as the SAN768B, or more generically as system, device, or chassis, where appropriate. This chapter contains the following information: v “Overview of the SAN768B”...
Two slots for control processor blades (CP8): – A single active CP8 blade can control all 512 ports in the chassis. – The standby CP8 blade assumes control of the SAN768B if the active CP fails. v Two slots for core switch blades (CR8): –...
Note: Airflow in the SAN768B is from the non-port (non-cable) side to the port (cable) side and out the exhaust vent. Figure 1 shows a sample configuration of the port side of the SAN768B, with the chassis door removed. Figure 1. Port side of the SAN768B (sample configuration)
Figure 2. Blower (non-port) side of the SAN768B (sample configuration) WWN bezel Blower assembly Power supply SAN768B blades Table 2 summarizes the port, application, control processor, and core switch blades that are available for the SAN768B. Table 2. Blades available for the SAN768B...
Page 37
Table 2. Blades available for the SAN768B (continued) Description Feature Code Name Function Core switch blade The CR8 blade contains the ASICs for switching between port blades. Every port blade connects to every core switch blade. There are 384 total ports for port blades.
Table 2. Blades available for the SAN768B (continued) Description Feature Code Name Function Fibre Channel router FC3850 FR4-18i The FR4-18i blade blade has 16 physical 4-Gbps Fibre Channel SFP ports supporting Fibre Channel Routing Services and two physical Gigabit Ethernet (GbE) SFP...
Reliability The SAN768B uses the following error detection and correction mechanisms to ensure reliability of data: v Data is protected by the Error Detection and Correction mechanism, which...
Guide. The Brocade Adapters Administrator’s Guide provides additional information that may be useful when setting up SAO. Security Table 3 highlights some of the key security features available for the SAN768B and for other switches running Fabric OS 5.2.0 or newer. Table 3. Security features...
E_port disable Network Manageability The SAN768B has a single domain and is managed as a single element with the Data Center Fabric Manager, Enterprise Fabric Connectivity Manager (EFCM), or Fabric Manager. The SAN768B responds to its own IP address and appears as a separate entity to the Telnet protocol and SNMP.
Page 42
SAN768B Installation, Service, and User Guide...
IBM service representatives. The SAN768B can be ordered pre-installed in the cabinet, or the SAN768B can be added to an existing C36 cabinet. The cabinet is designed to support a total of two SAN256Bs or SAN768Bs. The SAN384B can also be installed in the C36 cabinet.
– Routing the cables through the cable channels on the sides of the cabinet – Using patch panels v Ensure that the airflow in the vicinity of the SAN768B is a minimum of 595 cubic meters per hour (350 cubic feet per minute).
1. Provide a space that is 14 rack units (14U) high, 61.29 cm (24.09 in.) deep, and 43.74 cm (17.22 in.). 1U is equal to 4.45 cm (1.75 in.). 2. Plan to install the SAN768B with the non-port side facing the air-intake aisle. The SAN768B can be installed facing either direction, if serviceability and cooling requirements are met.
159.2 kg (351 lb) and requires the specified lift tool to install it. Installing a SAN768B in a cabinet This section describes how to install the SAN768B in the Model C36 cabinet. These procedures use parts that are included in the 14U rack mount kit. Time required Allow approximately 3 hours to complete this procedure.
1/4-20 x 1/2 in. (1.27 cm) panhead Phillips screw, with lock washer For cabinets that have vertical rails with square holes 10-32 retainer nut 1/4-20 x 1/2 in. (1.27 cm) panhead Phillips screw, with glue 0.375 in. (0.953 cm) alignment washer Chapter 2. Installing and removing a SAN768B...
Figure 3. Parts in the 14U rack mount kit Unpacking the SAN768B To unpack the SAN768B, follow these steps: 1. Cut the bands that encircle the packaging. 2. Remove the lid, the 14U rack mount kit, the accessory kit, and the packing foam from the top of the chassis.
1U space below the brackets for cable management. 4. Tighten the adjusting screws on the shelf brackets to a torque of 37 cm-kg (32 in.-lb). Chapter 2. Installing and removing a SAN768B...
See Figure 7 on page 20 for approximate positions. Note: Do not align the clip or retainer nuts with the top or bottom holes of the SAN768B mounting brackets because the door will interfere with the screw heads. Attaching clip nuts...
Step 1 on page 20 through step 9 on page 21 describe how to use the lift tool (PN 09P2481) and the 24–inch load plate (PN 11P4369) to install a SAN768B in the cabinet. Step 10 on page 21 through step 23 on page 23 describe how to install the bridge tool (PN 18P5855) in the cabinet for the lower SAN768B or the upper SAN768B.
SAN768B to reduce the weight. This can reduce the weight by up to 27 kg (60 lb). 4. Move the lift tool next to the pallet that contains the SAN768B. a. Adjust the load platform to the same height.
Page 53
Raise the platform to allow enough room to attach the anti-tip bracket. d. Secure the anti-tip bars by using the anti-tip bracket and bolt. 10. If you are installing the lower SAN768B, go to step 11. If you are installing the upper SAN768B, go to step 12 on page 22.
14. Set the wheel brake and place the wheel chocks around the wheels. 15. Remove the retention straps. 16. Slide the SAN768B into the cabinet until it rests securely on the shelf brackets. 17. Remove the wheel chocks, release the wheel brake, and move the lift tool away from the cabinet.
37 cm-kg (32 in-lb). See Figure 9 for the approximate location of the screws. Note: Do not use the top or bottom holes of the SAN768B mounting brackets because the screw heads will interfere with the chassis door.
Note: For turning on the power and for the configuration instructions, see Chapter 3, “Starting and configuring the SAN768B,” on page 31. Attention: Do not connect the SAN768B to the network until the IP addresses are correctly set. Port numbering Table 5 has a list of available port blades for the SAN768B.
Slots are numbered 1 through 12, from left to right when facing the port side of the SAN768B. Control processor blades (CP8) can be installed only in slots 6 and 7. Core switch blades (CR8) can be installed only in slots 5 and 8. Port, application, or encryption blades can be installed in slots 1–4 and 9–12.
FC8-64 port blade. Narrower OM-3 LC cables are used to connect the FC8-64. These cables are offered by several major manufacturers. Contact your IBM representative for options regarding different cable and patch panel configurations to simplify cable management with higher density FC8-64 port blades.
Removing a SAN768B from the cabinet If you need to remove a SAN768B chassis from a cabinet to move it to a new location, follow the instructions below to remove the chassis, and then follow the installation instructions (“Installing a SAN768B in a cabinet” on page 14) to install the chassis in the new cabinet.
Page 61
A pinch point exists between the load plate and the bridge tool. P/N 18P5850-B SJ000752 14. Slide the SAN768B from the cabinet onto the 24–inch load plate. Center the SAN768B on the platform. 15. Install the retention straps. 16. Remove the wheel chocks, release the wheel brake, and move the lift tool away from the cabinet.
Page 62
SAN768B Installation, Service, and User Guide...
On the displayed product page, click the Interoperability Matrix link in the Learn more section. The SAN768B must be configured before it is connected to the fabric, and all of the configuration commands must be entered through the active CP blade. The...
2. Specify a switch name. See “Customize a switch name” on page 36. 3. Specify a domain ID for the SAN768B. See “Setting the Domain ID” on page 4. Verify the PID mode and connect the SAN768B to the fabric. See “Verifying the PID mode and connecting to the fabric”...
IP address of the control processor. To establish a serial connection and log on to the SAN768B: 1. Verify that the SAN768B is powered on and that POST is complete by verifying that all power LED indicators on the port, control processor, and core switch blades display a steady green light.
To log in to the SAN768B through the serial connection, follow these steps. 1. Log in to the SAN768B as admin. The default password is "password." At the initial login, you are prompted to change the default passwords. Record the new passwords and keep this information in a secure location.
Page 67
Follow these steps to configure the IP addresses for both CP blades (from the active CP blade): 1. Log in to the active CP as admin using the serial connection. 2. Set up the SAN768B IP address by entering the ipaddrset -chassis command at the prompt. swDir:admin> ipaddrset -chassis Enter the information at the prompts.
Customize a switch name The switch name of the SAN768B can be up to 30 characters long as of Fabric OS release 6.3.0; can include letters, numbers, and underscore characters; and must begin with a letter. Fabric OS releases earlier than 6.3.0 require switch names of 15 characters or less.
Setting the date To set the date, follow these steps. 1. If necessary, log on to the SAN768B by Telnet, using the admin account. 2. Enter the date command, using the following syntax:date "mmddHHMMyy" Where: v mm is the month;...
The principal or primary FCS switch synchronizes its time with the NTP server every 64 seconds. switch:admin> tsclockserver LOCL switch:admin> tsclockserver "132.163.135.131" switch:admin> tsclockserver 132.163.135.131 switch:admin> The following example shows how to set up more than one NTP server using a DNS name: SAN768B Installation, Service, and User Guide...
In the example above, the license key is AAbbccDDeeFFeeGG. Make sure you keep a copy of your license key in a secure location for reference. The 64-bit chassis ID is required to obtain and activate licenses for the SAN768B. The chassis ID is available through the licenseIdShow command.
5. Verify the SAN768B and port status using the switchShow command. 6. Verify fabric connectivity using the fabricShow command. Note: The chassis door must be installed to ensure the SAN768B meets EMI and other regulatory certifications. Once all transceivers and cables are attached and arranged, and configuration is complete, reinstall the chassis door.
(ICL) cables” on page 98. Planning for encryption (optional) Encryption is an optional feature available on the SAN768B through the installation and use of the FS8-18 encryption blade. Advance planning is essential to achieve a successful implementation of encryption.
There is no support of Cisco switches at this time by IBM. The section in the Fabric OS Encryption Administrator’s Guide Supporting Tivoli Key Lifecycle Manager (TKLM) Environments related to Cisco Fabric connectivity does not currently apply. v The use of Smart Cards provides additional encryption security management, and is highly recommended.
1. Check the LEDs to verify that all components are functional. For information about LED patterns, see Chapter 4, “Monitoring system components,” on page 2. If necessary, log on to the switch by Telnet, using the admin account. Chapter 3. Starting and configuring the SAN768B...
This command provides information about switch and port status. 4. Verify the correct operation of the SAN768B in the fabric by entering the fabricShow command from the workstation. This command provides general information about the fabric.
Click the Release Firmware link and follow the online prompts to navigate to the Brocade Downloads for IBM End Users page. v Follow the links and instructions on the Brocade pages. Note that associated FOS documentation can also be downloaded from this site.
Page 78
SAN768B Installation, Service, and User Guide...
Chapter 4. Monitoring system components The SAN768B is engineered for reliability and requires no routine operational steps or maintenance. This chapter provides information about determining the status of the following components using LEDs and CLI commands. Refer to the Web Tools Administrator’s Guide and the Fabric OS Administrator’s Guide for additional...
Figure 17. FC8-48 port blade Blade power LED Fibre Channel port Blade status LED Port status LED SAN768B Installation, Service, and User Guide...
Page 83
Figure 18. FC8-64 port blade Blade status LED Fibre Channel port Blade power LED Port status LED Note: The FC8-64 port blade requires narrower OM-3 LC cables. Chapter 4. Monitoring system components...
Figure 19. FC10-6 port blade Blade power LED Fibre Channel port Blade status LED Port status LED SAN768B Installation, Service, and User Guide...
Page 85
Figure 20. FR4-18i routing blade Blade power LED Fibre Channel port Blade status LED Port status LED Chapter 4. Monitoring system components...
Figure 21. FS8-18 encryption blade | | | | | Blade power LED Fibre Channel port | | | | Blade status LED Port status LED SAN768B Installation, Service, and User Guide...
Page 87
Figure 22. FX8-24 extension blade Blade power LED GbE port 6 LED Port map Blade status LED 10 GbE port 0 GbE port 6 10 GbE port 0 LED Chapter 4. Monitoring system components...
Page 88
See “FX8-24 blade” on page 144 for information about trunking groups on this blade. Figure 23. FCOE10-24 blade Blade power LED 10 GbE FCoE port 9 LED Blade status LED 10 GbE FCoE port 19 10 GbE FCoE port 9 10 GbE FCoE port 19 LED SAN768B Installation, Service, and User Guide...
See “FCOE10-24 blade” on page 145 for limitations. If LED remains amber, contact IBM. Slow-flashing amber (on 2 Blade is not seated correctly or Pull blade out and reseat it. If seconds, then off 2 seconds) is faulty.
Page 90
Port is online, with traffic No action required. flowing through port. Fast-flashing amber (on 1/4 Transceiver or port is faulty. Change the transceiver or reset second, then off 1/4 second) the switch from the workstation. SAN768B Installation, Service, and User Guide...
Table 6. Port, application, and encryption blade LED descriptions (continued) LED purpose Color Status Recommended action 10GbE port status (FCOE10-24) No light (LED is off) Port is offline. Verify that the power LED is on, check the transceiver and cable. Steady green Port is online but has no No action required.
LED purpose Color Status Action Blade power Steady green CP blade is on. No action required. No light (LED is off) CP blade is not on. Ensure blade is firmly seated and has power. SAN768B Installation, Service, and User Guide...
If LED remains amber, contact IBM. Slow-flashing amber CP blade is not Pull blade out and (on 2 seconds; then seated correctly or is reseat it. If LED off 2 seconds) faulty.
Color Status Action Blade power Steady green CP blade is on No action required. No light (LED is off) CP blade is not on Ensure that the blade is firmly seated and has power. SAN768B Installation, Service, and User Guide...
See “FCOE10-24 blade” on page 145 for limitations.If LED remains amber, contact IBM. Slow-flashing amber CP blade is not Pull blade out and (on 2 seconds; then seated correctly or is reset it. If LED off 2 seconds) faulty.
2. Check the blower assembly status using the fanShow command. The status for each blower assembly displays OK, Absent, or Faulty. The RPM of each fan in the assembly is also provided. If a blower assembly is absent or faulty, contact IBM. SAN768B Installation, Service, and User Guide...
Figure 27. Blower assembly LEDs Item Description Power LED Fault LED Table 10. Blower assembly LED descriptions Recommended LED purpose Color Status action Power No Light (LED is off) Blower assembly Ensure that the does not have power. blower assembly is firmly seated and has power.
0x24c (fabos): Switch: switchname, Error accessible. EM-WWN_ABSENT, 2, WWN #1 not present Writing to the FRU history log 0x24c (fabos): Switch: switchname, Error (hilSetFruHistory) has failed. EM-HIL_FAIL, 2, HIL Error: hilSetFruHistory failed, rc=-3 for SLOT SAN768B Installation, Service, and User Guide...
Figure 28 displays the WWN bezel. Figure 28. LEDs on the WWN card and bezel CP blade Status (above) and Power Port blade Status (above) and (below) LEDs Power (below) LEDs CR blade Status (above) and Power Slot numbers (below) LEDs The WWN card LED patterns and the recommended actions for those patterns are described in Table 12.
Page 100
SAN768B Installation, Service, and User Guide...
“Removing the batteries” on page 107 Removing and installing the chassis door Note: The chassis door must be installed to ensure the SAN768B meets EMI and other regulatory certifications. Additionally, if ICL cables are not used, EMI plugs must be inserted in the ICL cable ports to meet certification standards.
Figure 29. Chassis door Removing and installing a cable management comb The SAN768B can continue to operate during the replacement of the cable management comb. Time and items required The replacement procedure for the cable management comb takes less than 5 minutes.
Figure 30. Cable management comb Installing a cable management comb To install a cable management comb, do the following: 1. Align the comb with the four screw holes on the chassis. 2. Position and tighten the four screws to secure the cable management comb to the chassis.
Slots are numbered from 1 through 12, from left to right when facing the port side of the SAN768B. Port, application, and encryption blades can be installed in slots 1 through 4 and 9 through 12. Attention: The FCOE10-24 application blade is not compatible with the FC8-64 high-density port blade and the other application blades (FR4-18i and FX8-24, , and FS8-18) in the same chassis.
Page 105
7. Disconnect all cables and the SFP, SFP+, or XFP transceivers from the blade. For mSFP transceivers(FC8-64 only), it is recommended that you use the pull tab to remove the transceiver from the blade before removing the cable from the transceiver. 8.
Attention: Follow ESD precautions (see “ESD precautions” on page xxi). Note: An FA4-18i routing blade and the SAN768B must have the same version of firmware. To upgrade firmware versions, refer to the Fabric OS Administrator’s Guide.
Adjust the ejectors to the open position, align the flat side of the blade inside the upper and lower rail guides in the slot, and slide the blade into the slot until it is firmly seated. 3. Perform the appropriate following action based on the type of blade: v For FC8-16, FC8-32, FC8-48, FC8-64 port blades, FX8-24 and FCOE10-24 application blades, and FS8-18 encryption blades: Adjust the ejectors to the closed position by pulling them away from the center of the blade.
This section describes how to remove and install a control processor (CP8) blade. Each SAN768B has two CP8 blades located in slot 6 and slot 7. Note: If the new CP blade does not have the same firmware as the active CP blade, you must upgrade the new blade to the same firmware version.
For more information about error messages, refer to the Fabric OS Message Reference. Recording critical SAN768B information Back up the SAN768B configuration before you replace a CP blade. Refer to the Fabric OS Administrator’s Guide for backup information. To record critical SAN768B information, do the following: 1.
Removing a control processor blade (CP8) The SAN768B continues to operate while a CP blade is being installed if the redundant CP blade is active and a failover does not occur. You can prevent failover by entering the haDisable command.
Figure 33. Removing the control processor (CP8) blade Installing a control processor blade (CP8) Attention: Follow ESD precautions (“ESD precautions” on page xxi). To install a CP blade (slot 6 or 7): 1. Open the ejector handles to approximately 45 degrees. Orient the CP blade so that the handles are toward you and the flat metal side is on your left.
From the Release notes page, click the Release Firmware link and follow the online prompts to navigate to the Brocade Downloads for IBM End Users page. f. Follow the instructions on the Brocade pages.
Page 113
11. Type version or firmwareShow to verify that the firmware version has been updated. 12. Pack the faulty CP blade in the packaging provided with the new CP blade for return to IBM, following local procedures. 13. Install the chassis door (See “Removing and installing the chassis door” on page 69.) Downloading firmware from a USB drive Firmware can be downloaded from a Brocade USB drive.
This section describes how to remove and install a core switch blade. Each SAN768B has two core switch blades: one in slot 5 and one in slot 8. Note that the blade in Figure 34 on page 83 is shown with EMI plugs installed.
2. Power off the blade by sliding the slider switch in the top ejector down to the off position (see Figure 34). Do not eject the blade until the power LED has gone off and you have completed the next two steps. 3.
This section describes how to remove and install an FR4-18i router blade in the SAN768B. You can install up to two router blades in any of slots 1-4 and 9-12. To ensure proper cooling, install only one router blade in each slot grouping. The physical removal of a router blade is the same as for port blades.
2. Establish a Telnet or console session before removing an router blade to determine a failure and verify operation before removal. Use the switchShow command to view the status of blades on the SAN768B. 3. Check for adequate cable slack.
5, this step is complete. If not, enter the chassisConfig 5 command. Attention: Changing the chassisConfig mode requires a reboot, and is disruptive to SAN768B operation. 2. Select an empty slot from slots 1-4 or 9-12 to install the router blade. SAN768B Installation, Service, and User Guide...
CLI, see the Fabric OS Administrator's Guide, or the Fabric OS Command Reference Manual , which are located on the product documentation CD. 11. Log into the SAN768B as the admin user. Configuring FCIP and Fibre Channel routing services and enabling the ports Refer to the Fabric OS Administrator's Guide or the Web Tools Administrator's Guide for detailed information on configuring the router blade.
SAN768B command prompt. This command provides information about switch and port status. Once the FR4-18i router blade is installed and fully configured in a SAN768B, the switchShow command displays 32 Fibre Channel ports (port numbers 0 through 31) and 2 GbE ports.
Removing a power supply To remove a power supply, do the following: 1. If the SAN768B will not be operating during the replacement procedure, go to step 2. If the SAN768B will continue operating during the replacement, check the power LEDs to verify that the minimum number of power supplies is functioning.
3. Tighten the thumbscrew. 4. Replace the power cord. 5. Turn on the power switch. 6. Verify that the power LED on the power supply displays a steady green light. SAN768B Installation, Service, and User Guide...
Removing and installing a blower assembly This procedure provides instructions for removing and installing a blower assembly. Attention: The BSAN768B can continue operating during the replacement of one blower assembly if the other two blower assemblies are operating, To ensure continuous adequate cooling, maintain three operating blower assemblies at all times except for the brief period when replacing a blower assembly.
Allow approximately 20 minutes to replace a WWN card. The following items are needed to install a WWN card: v ESD grounding strap v #2 Phillips screwdriver (required only for some versions of the WWN card) SAN768B Installation, Service, and User Guide...
v If a serial console session is used: serial cable and a workstation computer with a terminal emulator application (such as HyperTerminal for Windows systems or TIP for Solaris systems) Verifying necessity of installation Before installing a WWN card, verify that the installation is necessary. Any of the following events can indicate that the card requires installation: v Status LEDs on the WWN bezel do not reflect the actual status of the components...
To remove the WWN bezel and WWN card, complete the following steps. Attention: Follow ESD precautions. 1. Open a Telnet session to the SAN768B and log in to the active CP as admin. The default password is password. 2. Verify you are logged into the active CP. Run the haShow command to determine the active CP.
Figure 39. WWN bezel and card removal and installation Installing the WWN bezel and WWN card To install the WWN bezel and WWN card, do the following: Attention: Follow ESD precautions. 1. Unpack the new WWN card and save the packaging. 2.
Removing and replacing an SFP, SFP+, or XFP optical transceiver The SAN768B comes with a transceiver extraction tool (see Figure 40) and holster. The extraction tool is designed to remove transceivers from blades where the space is limited. To use the extraction tool, slide the tool on to the bale of the transceiver and pull it out.
Note: This tool is part of the SAN768B ship group and should be kept with the SAN768B. Attention: mSFP optical transceivers should not be inserted into ports intended for SFP or SFP+ transceivers. They will be faulted on power-up for Fabric OS 6.4.0 and later.
Note: The mSFP transceivers are used only with the FC8-64 port blade. Narrower OM-3 LC cables are used to connect the FC8-64. These cables are offered by several major manufacturers. Contact your IBM representative for options regarding different cable and patch panel configurations to simplify cable management with higher density FC8-64 port blades.
ICL cables in one of the configurations shown. The cables can connect ICL connector ports on the SAN768B core switch blades (CR8) or SAN384B core switch blades (CR4S-8). Figure 48 on page 105 and Figure 49 on page 106 show 3-way ICL connections between SAN768B and SAN384B models.
3. Using the latch release tabs, pull each plug on the cable from its respective socket on the SAN768B core switch blades (CR8) or SAN384B core switch blades (CR4S-8). 4. Repeat for each cable that requires replacement.
3 Chassis 2 (SAN768B) 6 ICL connector (ICL 0) For Fabric OS 6.4.0 and later, you can use three SAN768B chassis is an ICL configuration if the third chassis is in an immediately adjacent rack. Note: For clarity, the two sets of cables are drawn differently (solid and dashed) in Figure 48 on page 105 throughFigure 50.
Repair or disassemble Exchange only with the IBM-approved part. Recycle or discard the battery as instructed by local regulations. In the United States, IBM has a process for the collection of this battery. For information, call 1-800-426-4333. Have the IBM part number for the battery unit available when you call.
Removing and replacing the chassis The SAN768B chassis is highly reliable and unlikely to fail. This chapter describes how to remove and replace a chassis in the unlikely event of a failure. The SAN768B must be removed from the fabric and powered off to perform this procedure.
Note: The SAN768B must be removed from the fabric and powered off to perform this procedure. Contact IBM Support if you have any questions about whether the chassis requires replacement. Verifying necessity of replacement Before replacing the chassis, verify that the replacement is necessary. Ensure that the components are firmly seated when troubleshooting.
Page 143
New serial number (if available): To record critical switch and SAN information: 1. Open a Telnet session and login to the SAN768B as admin. The default password is password. Refer to your records for current passwords. Enable the logging function on your Telnet or serial console connection.
Page 144
1." If the current WWN cards are the original cards, the factory serial number listed is the same as the chassis serial number. If a "Serial Num" value displays, contact IBM to determine whether this serial number should be modified after the new WWN card is installed.
Copy the command output into a text file named SANbefor.txt. After the SAN768B is restored to the fabric, this information can be used to verify that no unintentional changes have occurred to the fabric. switch:admin> nsshow Enter Pid COS PortName NodeName TTL <output truncated>...
Although the empty chassis is lighter than a fully populated chassis, a lifting device is still required to move it. Refer to “Removing a SAN768B from the cabinet” on page 28 for more detailed instructions on removing a SAN768B from a cabinet.
19 for detailed steps. Installing components into the new chassis Install the components into the new SAN768B chassis in the order listed below. Refer to Chapter 5, “Removing and installing components,” on page 69 for more specific installation information.
SW BLADE 51 FC8-48 ENABLED UNKNOWN VACANT switch:admin> 3. Verify that the SAN768B is functioning correctly by entering switchShow or switchStatusShow. The switchShow command displays chassis and port status information. switch:admin> switchshow switchName: switch switchType: 62.1 SAN768B Installation, Service, and User Guide...
Refer to “Managing cables” on page 40 for information on how to Attention: physically manage the arrangement of the cables within the SAN768B chassis, and cautions on handling and routing the cables. Refer to the cable routing information recorded in “Cable routing tables” on page 119 to ensure the correct cable connections in the steps below.
Note: The SAN768B can be accessed by remote connection using any of the available management tools, such as Telnet or Web Tools. Ensure that the SAN768B is not modified from other connections during the rest of this procedure. 2. Reconnect the transceivers and cables to the port blades, as required (see “Removing and installing transceivers”...
3. Resolve any issues or unintentional changes to the SAN768B or fabric. v If there are any mechanical problems, try reseating the associated component. v If the configuration information is not correct for the SAN768B, modify as required. v If other issues exist, contact technical support.
Table 18. Cable routing table for SAN768B (64 ports shown) Slot/Port Cable labels Connected Slot/Port of Slot Port Switch end Device end device device SAN768B Installation, Service, and User Guide...
Page 153
Table 18. Cable routing table for SAN768B (64 ports shown) (continued) Slot/Port Cable labels Connected Slot/Port of Slot Port Switch end Device end device device Chapter 5. Removing and installing components...
Page 154
SAN768B Installation, Service, and User Guide...
Refer to “Safety notices and labels” on page xv before performing any service or installation procedures. The SAN768B can be upgraded with any of the following hot-pluggable features while the SAN768B is operational. Refer to the individual feature installation instructions for any possible exceptions.
CLI, see the Fabric OS Administrator's Guide, which is located on the product documentation CD. 10. Reinstall the chassis door. See “Removing and installing the chassis door” on page 69. SAN768B Installation, Service, and User Guide...
Note: This 8-Gb 64-port blade (FC3864) cannot be installed in the same chassis as a FCOE10-24 blade (FC3880) These instructions cover installing a new FC3864 in the SAN768B chassis. Time required 20 minutes or more per port blade due to port and cable density...
Page 158
FC8-64 port blade. Narrower OM-3 LC cables are used to connect the FC8-64. These cables are offered by several major manufacturers. Contact your IBM representative for options regarding different cable and patch panel configurations to simplify cable management with higher density FC8-64 port blades.
3. Verify which slot to install the blade. Slots are numbered from one through ten, from left to right, when facing the port side of the SAN768B. A maximum of two router blades can be installed in a single SAN768B chassis. Router blades can be installed in any slot 1-4 and 9-12.
11. Reinstall the chassis door. See “Removing and installing the chassis door” on page 69. FC3870 - Installing an FC10-6 10 Gb blade This feature can be installed while the SAN768B is operational. Attention: Follow these precautions for all procedures in this section to avoid damaging the port blades or chassis: v Wear a grounded ESD strap when handling a port blade.
69. FC3880 - Installing an FCOE10-24 blade This feature can be installed while the SAN768B is operational. Attention: This blade cannot be used in the same chassis with an FC8-64 high density port blade or the FR4-18i, FX8-24, or FS8-18 application blades.
CLI, see the Fabric OS Administrator's Guide, or the Fabric OS Command Reference Manual , which are located on the product documentation CD. 11. Reinstall the chassis door. See “Removing and installing the chassis door” on page 69. SAN768B Installation, Service, and User Guide...
FC3890 - Installing an FX8-24 extension blade This feature can be installed while the SAN768B is operational. Attention: Follow these precautions for all procedures in this section to avoid damaging the port blades or chassis: v Wear a grounded ESD strap when handling a port blade (see “ESD precautions”...
Encryption Administrator’s Guide Supporting Tivoli Key Lifecycle Manager (TKLM) Environments to successfully complete installation, configuration, and management of the encryption functionality. This feature can be installed while the SAN768B is operational. Attention: This blade cannot be used in the same chassis with an FCOE10-24 blade.
Time required Less than 20 minutes. Items required v ESD (electrostatic discharge) grounding strap v Workstation computer v Encryption blade v Phillips screwdriver v Transceivers (as needed) v Optical cables (as needed) Procedures Follow the links in the steps below to complete this installation. 1.
FC7870 - Installing an inter-chassis cable kit This feature can be installed while the SAN768B is operational. Time required Less than 10 minutes, not including installation of the ICL license or any needed configuration.
Appendix A. Product specifications This appendix provides product specifications for the SAN768B for the following: v “System architecture” v “System size and weight” on page 138 v “Facility requirements” on page 139 v “Power specifications” on page 139 v “Environmental requirements” on page 140 v “Fibre Channel port specifications”...
Page 168
CEE to CEE and FCOE to FC, <5 msec any port to any port at 10 Gbps, cut-through routing Maximum frame size 2112-byte payload Frame buffers 2048 per ASIC, dynamically allocated. Classes of service Class 2, Class 3, Class F (interswitch frames) SAN768B Installation, Service, and User Guide...
Page 169
Media types Note: For a listing of SFPs, SFP+, mSFPs and XFPs compatible with this product: v Go to the IBM SAN web page www.ibm.com/ systems/storage/san v Select the SAN768B product link v On the SAN768B product page, click the Interoperability Matrix link.
System size and weight The weight of the SAN768B can vary considerably depending on the combination of blades installed. Use Table 21 and Table 22 to determine the weight of the system with your combination of port and application blades.
Facility requirements The facility where the SAN768B is in use must meet the following requirements to provide for correct operation: v Adequate supply circuit, line fusing, and wire size, as specified by the electrical rating on the chassis nameplate v An air flow of at least 595 cubic meters per hour (350 cubic feet per minute),...
Fibre Channel port specifications The Fibre channel ports in the SAN768B support full duplex link speeds at 1, 2, 4, 8, or 10 Gbps inbound and outbound, automatically negotiating to the highest common speed of all devices connected to the port.
The ports are universal and self-configuring, capable of becoming F_Ports (fabric ports), FL_Ports (fabric loop-enabled), or E_Ports (expansion ports). The ports meet all required safety standards. Data transmission ranges Table 25 provides the data transmission ranges for different transceivers, cable types, and port speeds.
(Feature Code 7880). See the IBM TotalStorage SAN Cabinet 2109 Model C36 Installation and Service Guide for information on connecting the cabinet to the facility power source. If the stand-alone feature is ordered for the SAN768B, then you must also order the country-specific power cords. Contact your IBM representative for more information on these power cords.
Support for 200 ms RTT (on a limited number of GbE ports) v Adaptive Rate Limiting (licensable) v TCP performance graphing in Web Tools v FCIP Tunnels – A maximum of 10 FCIP Tunnels for all GbE ports SAN768B Installation, Service, and User Guide...
– Four tunnels maximum per GbE port – Two 10 GbE ports can support up to ten FCIP tunnels each – Each FCIP tunnel is represented and managed as a virtual Fibre Channel E_Port – Fibre Channel Routing Services can be used over the FCIP link –...
Page 178
SAN768B Installation, Service, and User Guide...
Fabric Manager, Web Tools, Fabric Watch, and Advanced Performance Monitoring. If the SAN768B does not operate as expected, the following steps can be taken to diagnose the problem: v Check the LEDs and refer to the LED tables (see Chapter 4, “Monitoring system components,”...
Interpreting POST and boot results The SAN768B performs power on self test (POST) by default each time the system is powered on, rebooted, or reset. The system can be rebooted using the reboot (to reboot each CP individually), or fastBoot commands.
1. Universal port configuration is performed. 2. Links are initialized. 3. Fabric is analyzed. If any ports are connected to other switches, the SAN768B participates in a fabric configuration. 4. The SAN768B obtains a domain ID and assigns port addresses.
Page 182
9600 bits per second, 8 databits, no parity, 1 stop bit, no flow control. Serial port might be incompatible Ensure that the SAN768B is connected to an (only RS–232 is supported). RS–232 port. RS–423 serial ports might experience difficulties due to corner-case incompatibilities of the standards.
Page 183
Table 27. Troubleshooting (continued) None of the LEDs on an Component might not be seated Ensure that the SAN768B has power and individual component are correctly. component is firmly seated. If problem continues, run the sensorShow command to determine component status. If component is a CP blade or port blade, enter the slotShow command to determine status.
Page 184
SAN768B Installation, Service, and User Guide...
Appendix D. Blade port numbering The illustrations in this appendix show the port numbering for the different SAN768B blades. See “Port numbering” on page 25 for details on the port numbering pattern for the different blades. v Figure 53 on page 154 shows an FC8-16 port blade...
Figure 54. FC8-32 port blade Blade power LED FC ports 16-31 (bottom to top) Blade status LED FC ports 0-15 (bottom to top) Appendix D. Blade port numbering...
Figure 55. FC8-48 port blade Blade power LED FC ports 24-47 (bottom to top) Blade status LED FC ports 0-23 (bottom to top) SAN768B Installation, Service, and User Guide...
Page 189
Figure 56. FC8-64 port blade Blade power LED port 61 LED Blade status LED port 29 LED Port 63 FC ports 32-63 (bottom to top) Port 30 FC ports 0-31 (bottom to top) Appendix D. Blade port numbering...
10-6 Figure 57. FC10-6 port blade, ports 0-5 (bottom to top) SAN768B Installation, Service, and User Guide...
Page 191
Figure 58. FR4-18i routing blade FC ports 0-15 (bottom to top) GbE ports (GE0 bottom, GE1 top) Appendix D. Blade port numbering...
Page 192
Figure 59. FS8-18 encryption blade | | | Key management 1000Base media FC ports 0-15 (bottom to top) interfaces Key management smart card interface SAN768B Installation, Service, and User Guide...
Page 193
Figure 60. FX8-24 extension blade Status LED FC ports 0-5 (numbered bottom to top) Power LED FC ports 6-11 (numbered bottom to top) 1 GbE ports 0-3 (numbered bottom to top) 1 GbE ports 4-9 (numbered bottom to top) 10 GbE ports 0-1 (numbered bottom to top) Appendix D.
Page 194
Figure 61. FCOE10-24 blade Status LED 10 GbE FCoE ports 0-11 (numbered bottom to top) Power LED 10 GbE FCoE ports 12-23 (numbered bottom to top) SAN768B Installation, Service, and User Guide...
The materials at those web sites are not part of the materials for this IBM product and use of those web sites is at your own risk.
Page 196
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products.
International Business Machines Corporation in the United States, other countries, or both. A complete and current list of other IBM trademarks is available on the Web at www.ibm.com/legal/copytrade.shtml Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
Properly shielded and grounded cables and connectors must be used in order to meet FCC emission limits. IBM is not responsible for any radio or television interference caused by using other than recommended cables and connectors or by unauthorized changes or modifications to this equipment.
Klasse A ein. Um dieses sicherzustellen, sind die Geräte wie in den Handbüchern beschrieben zu installieren und zu betreiben. Des Weiteren dürfen auch nur von der IBM empfohlene Kabel angeschlossen werden. IBM übernimmt keine Verantwortung für die Einhaltung der Schutzanforderungen, wenn das Produkt ohne Zustimmung der IBM verändert bzw.
Korea Communications Commission (KCC) Class A Statement Please note that this equipment has obtained EMC registration for commercial use. In the event that it has been mistakenly sold or purchased, please exchange it for equipment certified for home use. SAN768B Installation, Service, and User Guide...
Russia Electromagnetic Interference (EMI) Class A Statement Australia and New Zealand Class A Statement Attention: This is a Class A product. In a domestic environment this product might cause radio interference in which case the user might be required to take adequate measures.
Page 202
SAN768B Installation, Service, and User Guide...
Page 204
55 SFPs 39 status 47, 57 WWN bezel and card 92 FC8-64 port blades 125 XFP 96 FCIP extension blade XFPs 39 FX8-24 47 installing components 69 FCOE10-24 blade 56, 145 getting help iv SAN768B Installation, Service, and User Guide...
Page 205
40 specifications 135 overview 1 product documents xxvii labels, safety xx product specifications 135 LEDs providing power to the SAN768B 33 encryption blades 57 port blades 57 parameters routing blades 57 configuration 33 license, for patents 163 parameters, initial configuration 34...
Page 206
34 time Server Application Optimization setting 37 (SAO) 8 time required serviceability 7 installing a SAN768B 14 setting time server date 37 NTP 38 Domain ID 37 time zone time 37 setting 38 SAN768B Installation, Service, and User Guide...
Page 208
Part Number: 45W8666 Printed in USA GA32-0574-05...