AIX fibre channel error - FCS_ERR6

This error can occur if the fiber channel adapter is extremely busy. The AIX FC adapter driver is trying to map an I/O buffer for DMA access, so the FC adapter can read or write into the buffer. The DMA mapping is done by making a request to the PCI bus device driver.

The PCI bus device driver is saying that it can't satisfy the request right now. There was simply too much IO at that moment, and the adapter couldn't handle them all. When the FC adapter is configured, we tell the PCI bus driver how much resource to set aside for us, and it may have gone over the limit. It is therefore recommended to increase the max_xfer_size on the fibre channel devices.

It depends on the type of fibre channel adapter, but usually the possible sizes are:

0x100000, 0x200000, 0x400000, 0x800000, 0x1000000

To view the current setting type the following command:

# lsattr -El fcsX -a max_xfer_size

Replace the X with the fibre channel adapter number.

You should get an output similar to the following:

max_xfer_size 0x100000 Maximum Transfer Size True

The value can be changed as follows, after which the server needs to be rebooted:

# chdev -l fcsX -a max_xfer_size=0x1000000 -P

Read More

How-to replace a failing HBA using SDD storage

This is a procedure how to replace a failing HBA or fibre channel adapter, when used in combination with SDD storage:

1. Determine which adapter is failing (0, 1, 2, etcetera):
   # datapath query adapter
2. Check if there are dead paths for any vpaths:
   # datapath query device
3. Try to set a "degraded" adapter back to online using:
   # datapath set adapter 1 offline

   # datapath set adapter 1 online
   (that is, if adapter "1" is failing, replace it with the correct adapter number).
4. If the adapter is still in a "degraded" status, open a call with IBM. They most likely require you to take a snap from the system, and send the snap file to IBM for them to analyze and they will conclude if the adapter needs to be replaced or not.
5. Involve the SAN storage team if the adapter needs to be replaced. They will have to update the WWN of the failing adapter when the adapter is replaced for a new one with a new WWN.
6. If the adapter needs to be replaced, wait for the IBM CE to be onsite with the new HBA adapter. Note the new WWN and supply that to the SAN storage team.
7. Remove the adapter:
   # datapath remove adapter 1
   (replace the "1" with the correct adapter that is failing).
8. Check if the vpaths now all have one less path:
   # datapath query device | more
9. De-configure the adapter (this will also de-configure all the child devices, so you won't have to do this manually), by running: diag, choose Task Selection, Hot Plug Task, PCI Hot Plug manager, Unconfigure a Device. Select the correct adapter, e.g. fcs1, set "Unconfigure any Child Devices" to "yes", and "KEEP definition in database" to "no". Hit ENTER.
10. Replace the adapter: Run diag and choose Task Selection, Hot Plug Task, PCI Hot Plug manager, Replace/Remove a PCI Hot Plug Adapter. Choose the correct device (be careful, you won't see the adapter name here, but only "Unknown", because the device was unconfigured).
11. Have the IBM CE replace the adapter.
12. Close any events on the failing adapter on the HMC.
13. Validate that the notification LED is now off on the system, if not, go back into diag, choose Task Selection, Hot Plug Task, PCI Hot Plug Manager, and Disable the attention LED.
14. Check the adapter firmware level using:
    # lscfg -vl fcs1
    (replace this with the actual adapter name).
    And if required, update the adapter firmware microcode. Validate if the adapter is still functioning correctly by running:
    # errpt

    # lsdev -Cc adapter
15. Have the SAN admin update the WWN.
16. Run:
    # cfgmgr -S
17. Check the adapter and the child devices:
    # lsdev -Cc adapter

    # lsdev -p fcs1

    # lsdev -p fscsi1
    (replace this with the correct adapter name).
18. Add the paths to the device:
    # addpaths
19. Check if the vpaths have all paths again:
    # datapath query device | more

Read More

Raid Levels

The different raid levels available today

Raid 0 - Stripping data across the disks. This stripes the data across all the disks present in the array. This improves the read and write performance. Eg. Reading a large file takes a long time in comparison to reading the same file from a Raid 0 system.They is no data redundancy in this case.

Raid 1 - Mirroring. In case of Raid 0 it was observed that there was no redundancy,i.e if one disk fails then the data is lost. Raid 1 overcomes that problem by mirroring the data. So if one disk fails the data is still accessible through the other disk.

Raid 2 - RAID level that does not use one or more of the "standard" techniques of mirroring,striping and/or parity. It is implemented by splitting data at bit level and spreading it across the data disks and redundant disk. It uses a special algorithm called as ECC (error correction code) which is accompanied across each data block. These are tallied when the data is read from the disk to maintain data integrity.

Raid 3 - data is striped across multiple disks at a byte level. The data is stripped with parity and the parity is maintained in a separate disk. So if that disk goes off , it results in a data loss.

Raid 4 - Similar to Raid 3 the only difference is that the data is striped across multiple disks at block level.

Raid 5 - Block-level striping with distributed parity. The data and parity is stripped across all disks thus increasing the data redundancy. Minimum three disks are required and if  any one disk goes off the data is still secure.

Raid 6 - Block-level striping with dual distributed parity. Its stripes blocks of data and parity across all disks in the Raid except that it maintains two sets of parity information for each parcel of data thus increasing the data redundancy. So if two disk go off the data is still intact.

Raid 7 - Asynchronous, cached striping with dedicated parity. This level is not a open industry standard. It is based on the concepts of Raid 3 and 4 and a great deal of cache is included across multiple levels. Also there is a specialized real time processor to manage the array asynchronously.

Read More

Pictorial description of DAS NAS & SAN

DAS:

Direct-attached storage, or DAS, is the most basic level of storage which most people are familiar with, in which storage devices are part of the host computer, as with drives, or directly connected to a single server, as with RAID arrays or tape libraries. Network workstations must therefore access the server in order to connect to the storage device. DAS is ideal for localized file sharing in environments with a single server or a few servers - for example, small businesses or departments and work groups that do not need to share information over long distances or across an enterprise.  From an economical perspective, the initial investment in direct-attached storage is cheaper.

NAS:

Networked storage was developed to address the challenges inherent in a server- based infrastructure such as direct-attached storage. Network-attached storage, or NAS, is a special purpose device, comprised of both hard disks and management software, which is 100% dedicated to serving files over a network. 

SAN:

A storage area network (SAN) is an architecture to attach remote computer storage devices (such as disk arrays, tape libraries, and optical jukeboxes) to servers in such a way that the devices appear as locally attached to the operating system. Although the cost and complexity of SANs are dropping, they are uncommon outside larger enterprises. With their high degree of sophistication, management complexity and cost, SANs are traditionally implemented for mission-critical applications in the enterprise space. In a SAN infrastructure, storage devices such as NAS, DAS, RAID arrays or tape libraries are connected to servers using Fibre Channel. Fibre Channel is a highly reliable, gigabit interconnect technology that enables simultaneous communication among workstations, mainframes, servers, data storage systems and other peripherals. Without the distance and bandwidth limitations of SCSI, Fibre Channel is ideal for moving large volumes of data across long distances quickly and reliably.

Read More

Move data quickly between AIX LPARs using Logical Volume Manager

Move data quickly between AIX LPARs using Logical Volume Manager

Volume group export and import is fast, simple, and safe

Summary:  Migrating data from one LPAR to another can be challenging. You need to set aside some disk space on the new LPAR and create file systems. And whether you choose to copy the data across the network or do a backup and restore, a large amount of data can take many hours to move. Wouldn't it be good if you could gift wrap your data, send it to a new LPAR, and then unwrap it ready to use? The IBM AIX operating system Logical Volume Manager (LVM) lets you do just that by exporting a volume group from one LPAR and importing it to another one.

Migrate an entire volume group

Frequently used acronyms

• FTP: File Transfer Protocol
• LPAR: Logical Partition
• LVM: Logical Volume Manager
• ODM: Object Data Manager
• SAN: Storage Area Network

From the earliest days of AIX, the LVM has made management of disks, logical volumes, and file systems easy. The LVM lets you group disks together into volume groups. You can slice off an entire volume group from an AIX system with the exportvg command

When you export the volume group, you effectively package all its logical volumes and file systems along with the directories and files in them. You can then remove the disks from the source system and assign them to the target.

On the target LPAR, you do the reverse by importing the volume group with the importvgcommand. This also makes the volume group active and updates the new LPAR with details of file systems associated with the logical volumes. You can then do an integrity check of those file systems before you mount them ready for use.

In short, there are three steps to moving a volume group:

1. Remove the volume group and its disks from the source LPAR.
2. Reassign the disks to the target LPAR.
3. Configure the disks, volume group, and file systems on the target LPAR.

This article focuses on steps 1 and 3, which address the AIX-specific procedures at the LVM level. The reassignment of the disks (step 2) varies, depending on your storage configuration.

Before getting into the detail of the commands to run, there are some hints and warnings that are worth keeping in mind.

What are disks?

The disks that belong to a volume group are what AIX knows as physical volumes. But they may not be dedicated physical disks. The physical volumes may be SAN logical unit numbers (LUNs) or file-backed devices presented via the Virtual I/O Server. Whatever sort of disks they are, you need to make them available to the target LPAR before you can do the import of the volume group.

The whole volume group gets moved

When you export a volume group, the whole volume group becomes unavailable until it is imported again. This means the volume group to be exported can only contain data you don't need to keep on the original LPAR.

Migrating between sites

The procedure outlined here works between two LPARs on the same physical server. It works just as well if the disks are being assigned to a different server. You could export a volume group, use SAN replication to copy the data to a server on a remote site, and then import the volume group. You can even use the AIX LVM to migrate to a completely new SAN storage subsystem (see Resources).

Helpful commands

The AIX commands used are shown below in Table 1. Check the AIX command documentation for the syntax for these commands (see Resources).

Table 1. Commands for relocating volume groups

Command	Description
cfgmgr	Discover and configure new devices on the target LPAR.
df	Display file systems.
exportvg	Remove the volume group definition from the system.
fsck	Check the file system consistency.
importvg	Import a volume group definition from the system and activate the volume group.
lspv	List physical volume.
lsvgfs	List the file systems belonging to a volume group.
mount	Mount a file system.
rmdev	Remove the physical volume definitions from the source LPAR.
umount	Unmount a file system.
varyoffvg	Deactivate a volume group.
varyonvg	Activate a volume group.

Move or copy?

One benefit of using the volume group export and import procedure is that it allows you to move data rather than copy it. That saves you having to allocate additional storage to hold a copy of your data. It also saves you time. If you have a large amount of data or many thousands of files to move, the time saving can be significant.

But maybe you do want to create a spare volume group on the source LPAR and create a file system and copy data to it. You could then export your spare volume group so you can make it available on the target LPAR.

If you choose to do that, consider turning off JFS logging on your new file system before you start copying to it.

Improve performance with JFS2 NOLOG

You can speed up writes to a file system by mounting it with the NOLOG option. This feature of the IBM enhanced journaled file system (JFS2) was introduced with AIX V6.1. The NOLOG option is only intended for data you can easily copy or restore again or for temporary file systems with data that is not important or is easy to reproduce.

To mount a JFS2 file system without logging, use:

# mount -o log=NULL /mountpoint

Writes to this file system should be faster because journaling does incur a small performance penalty.

When the file system contains all the data it requires, you can export the volume group from the source LPAR and import it on the target. You should then mount the file system on the target LPAR using the normal JFS logging (without using the NOLOG option).

For more information about turning off JFS2 logging, see Resources.

Remove a volume group from the source LPAR

When you have your data ready to move, remove the volume group and its physical volumes from the source LPAR. To do that, the logical volumes belonging to that volume group must be closed. That requires you to unmount the file systems that belong to the volume group you are going to export.

List file systems in the volume group

To identify the file systems belonging to a volume group, use the lsvgfscommand:

# lsvgfs sparevg
# /scratch

Unmount file systems

Unmount any file systems in the volume group using the umount command:

# umount /scratch

You also need to deactivate any paging spaces and de-configure any system dump devices that belong to the volume group (see Resources for details).

When the logical volumes are closed, you should be able to deactivate the volume group.

Deactivate the volume group

You can make the volume group inactive with the varyoffvg command:

# varyoffvg sparevg

You can check the volume group status by listing physical volumes with lspv This shows the volume group name. The volume group status is no longer listed as active.

# lspv
# hdisk0          00cb07a4cee0b389                    rootvg          active
# hdisk1          00cb07a42fb87e1d                    sparevg

Export the volume group

You should now be able to export the volume group using exportvg:

# exportvg sparevg

After you run exportvg lspv displays the volume group as None

# lspv
# hdisk0          00cb07a4cee0b389                    rootvg          active
# hdisk1          00cb07a42fb87e1d                    None

You could import the volume group again on the source LPAR using importvg . However, because you're planning to move it to another LPAR, you need to remove the references to the disks from the operating system on the source LPAR.

Remove the physical volumes from the ODM

The ODM still keeps a record of the physical volumes at the operating system level. It is now safe to delete the physical volumes belonging to the volume group using rmdev.This removes references to the disks on the source LPAR, but it doesn't affect the data on the disks.

Volume group descriptor area and volume group status area

Each of the disks in the volume group has management data, including a volume group descriptor area and a volume group status area. This management data is stored on each physical volume of the volume group. When you remove the disks from the ODM, the disk management data remains intact.

# rmdev -d -l hdisk1
# hdisk1 deleted

Assign storage to target LPAR

The next step is to assign the storage to the target LPAR. How you do this depends on your storage configuration. Whoever is responsible for your storage subsystem should be able to present the disks to the new LPAR.

Import data on target LPAR

After you assign the disks to the target LPAR, you need to identify them at the operating system level and then make the volume group and file systems available.

Identify new disks on target LPAR

The new disks need to be discovered on the operating system of the target LPAR. A reboot of the LPAR would do this, but that's not necessary. You can run cfgmgr to configure the new devices:

# cfgmgr

The lspv command should show these disks with their Physical Volume Identifier (PVID). The volume group is shown as None, and the volume group status doesn't display as active.

# lspv
# hdisk0          00cb07a4b24eb714                    rootvg          active
# hdisk1          00cb07a42fb87e1d                    None

The disk name on the target LPAR (hdiskN) may not be the same name as the source LPAR. However, the PVID should be the same on the source and the target because it is a unique identifier for each physical volume.

Import the volume group on the target LPAR

Use the importvg command and a PhysicalVolume parameter to import the volume group. You only need to specify one physical volume, even if there are several physical volumes in the volume group. The importvg command finds any remaining physical volumes belonging to the same volume group and includes them in the import.

# importvg -y sparevg hdisk1

The system automatically runs the varyonvgcommand to activate the volume group after it is imported. If the volume group is concurrent capable, you will be prompted by the importvgcommand to activate the imported volume group manually usingvaryonvg

Updating file systems

If the volume group has file systems in it, the /etc/filesystems file is updated with information for the imported volume group's logical volumes and mount points. This saves you rebuilding the logical volumes and file systems yourself, as you would need to do before running a traditional restore or copy operation. The logical volume and file system characteristics are also inherited.

Before mounting the file systems, you need to verify their integrity.

Run fsck on all file systems

You should check the file system consistency for each file system. You can do this with the fsckcommand:

# fsck /scratch

Mount the file systems on target LPAR

You are now ready to mount the file systems, check permissions, and verify the file system contents. Once again, you can list all the file systems belonging to the volume group with lsvgfs.The mount command makes the file system available, along with all of its contents.

# mount /scratch

You can check the mounted file systems using df.Be sure to check the permissions on the mount points, directories, and files.

Summary

The AIX LVM provides a simple way of moving large amounts of data from one LPAR to another. It can reduce or even eliminate system downtime when you want to do data migration.

You can use the volume group export and import to refresh a database or to perform a disaster recovery test. It has plenty of uses when you need to move data quickly without requiring additional disk or making unnecessary demands on your backup infrastructure.

Source

www.unixmantra.com

Read More

SAN Switch cheat sheet

M-Series
M-Series (Mcdata) switches uses both web and CLI, the table below displays some but not all the CLI commands.

commadelim	Toggle comma-demlimited display mode
config	configure settings
login	login into CLI with different access rights
maint	maintenance settings
perf	Performance statistics
reserved	reserved for future development
show	display attributes
features	configure feature settings
ip	configure IP settings
logout	logout of the CLI
port	configure port data
security	configure security settings
snmp	configure snmp
switch	configure switch data
system	configure system data
zoning	confgiure zoning settings

There are several commands that allow you to navigate through the switch "..", "ctrl-U" and "root" .
M-Series (Mcdata) zoning commands are detailed in the below table

showactive	show actively running zoneSet
clearzone	clear WWN's in a zone
deletezone	remove zone from running config
activezoneset	activation of changes
addzone	add a new zone to the working area
addwwnmem	add a WWN to a zone
showpending	show pending zones
renamezone	rename a zone
deletewwn	delete a WWN from a zone
renamezoneset	rename a zone set

Zoning limits

• 64 zone sets (max)
• 2000 zones (max)
• 1024 zones per zone set (max)
• deafult should all ways be disabled (causes all port to see each other creating ghosts in a FA's login table)

B-Series
B-Series (Brocade) switches uses both web and CLI, the table below displays some but not all the CLI commands.

help	prints available commands
switchdisabled	disable the switch
switchenable	enable the switch
licensehelp	license commands
diaghelp	diagnostic commands
configure	change switch parameters (BB credits, etc)
diagshow	POST results since last boot
routehelp	routing commands
switchshow	display switch show (normally first command to run to obtain switch configuration)
supportshow	full detailed switch info
portshow #	display port info
nsshow	namesever contents
nsallshow	NS for full fabric
fabricshow	Fabric information
version	firmware code revision
reboot	full reboot with POST
fastboot	reboot without POST

B-Series (Brocade) zoning commands are detailed in the below table

zonecreate (zone)	create a zone
zoneshow	shows defined and effective zones and configurations
zoneadd	adds a member to a zone
zoneremove	removes a member from a zone
zonedelete	delete a zone
cfgcreate (zoneset)	create a zoneset configuration
cfgadd	adds a zone to a zone configuration
cfgshow	display the zoning information
cfgenable	enable a zone set
cfgsave	saves defined config to all switches in fabric across reboots
cfgremove	removes a zone from a zone configuration
cfgdelete	deletes a zone from a zone configuration
cfgclear	clears all zoning information (must disable the effective config first)
cfgdisable	disables the effective zone set

B-series creating a zone commands

Creating zone by WWN	zonecreate "zone1", "20:00:00:e0:69:40:07:08 ; 50:06:04:82:b8:90:c1:8d"
Create a zone configuration	cfgcreate "test_cfg", "zone1 ; zone2"
saving the zone configuration	cfgsave (this will save across reboots)
enable the zone configuration	cfgenable "test_cfg"
saving the zone configuration	cfgsave
view zoning information	zoneshow or cfgshow

MDS-Series
B-Series (Cisco) switches uses both web and CLI, the table below displays some but not all the CLI commands, the ones highlighted will be the ones used most often.
EXEC mode commands:

attach	connect to a specific linecard
bios	BIOS related commands
callhome	callhome commands
clear	reset functions
clock	manage the system clock
config	enter configuration mode
debug	debug functions
discover	discover information
exit	exit ffrom the submode
fcping	Ping a N_Port
fctrace	trace a route for a N_Port
load	load system image
no	disable debugging functions
reload	reboot the switch
setup	run the basic setup command facility
show	display running system information
system	system management commands
test	test command
write	write the current configuration
zone	zoning server commands

CONFIG mode commands:

aaa	config aaa
boot	configure boot variables
callhome	enter the callhome configuration mode
clock	configure time and date
fcalias	fcalias configuration commands
fcanalyzer	configure fabric analyzer
fcc	configure FCC congestion control
fcdomain	enter the fcdomain configuration mode
fcdroplatency	configure switch or network latency
fcflow	configure fcflow
fcinterop	interop commands
fcns	Name server configuration
fcroute	configure FC routes
fcs	Configure fabric config server
fctimer	configure fibre channel timers
fspf	configure fspf
interface	select an interface to configure
ip	configure IP features
no	Negate a command or set its default
ntp	NTP configuration
power	configure power supply
poweroff	power off module in switch
qos	Configure priority of FC control frames
radius-server	configure radius related parameters
role	configure roles
snmp-server	configure snmp
span	enter SPAN configuration mode
ssh	configure ssh parameters
switchname	configure systems network name
system	system config command
trunk	configure switch wide trunk protocol
username	configure username information
vsan	enter the vsan configuration mode
zone	zoning configuration commands
zoneset	zoneset configuration commands

There are several commands that allow you to navigate through the switch "cd", "copy", "dir", "find", "mkdir", "move", "rmdir" and "tail".
MDS-series creating a zone

create a zone and place it in the vsan	# zone name testzone1 vsan 4
add a member (port wwn example)	# member pwwn 10:00:00:00:c9:21:40:b8 (port on the host)
add a member (fabric port wwn example)	# member fwwn 10:01:10:01:10:ab:cd:ef (port on the switch)
add a member (FCID example - hard zoning)	# member fcid 0x7f0000 (24 bit address)
add a member (FC alias example)	# member fcalias payroll

MDS-series common commands

display logged in WWN, display vsan information	# show flogi database
display the specific vsan logins	# show fcns database vsan 4
activatng a zoneset	# zoneset activate name zone1 vsan 4
delete a zoneset	# clear zone dataabse vsan 4
configure the default zone	# zone default-zone permit vsan 4
display zones and zonsets	# show zoneset active # show zoneset # show zoneset active vsan 4
copy active zone set to the full zone set	# zone copy active-zoneset full-zoneset vsan 4
distribute the full zoning database to others switches	# zoneset distribute full vsan 4

Read More

How to find HBA Card WWPN Address on AIX

We need “WWPN Address” information of HBA card to create zoning on fiber channel switch for  assigning and unassign storage to a AIX host. Once you ask for storage from Storage Team the very first qestion will be what is the “WWPN Address” of HBA cards on your AIX. Now let’s see how to to find “WWPN Address” on AIX server.

Task 1 : Check how many fiber change adapters are there on the Server with below command

lsdev -Cc adapter

Example

um_lpar:/>lsdev -Cc adapter
ent0 Available 04-08 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent1 Available 04-09 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent2 Available 05-08 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
ent3 Available 05-09 2-Port 10/100/1000 Base-TX PCI-X Adapter (14108902)
fcs0 Available 06-08 FC Adapter
fcs1 Available 07-08 FC Adapter
ide0 Available 03-08 ATA/IDE Controller Device
sisscsia0 Available 02-08 PCI-X Dual Channel Ultra320 SCSI Adapter
vsa0 Available LPAR Virtual Serial Adapter

Alternatively you can run below command to grep  fiber channel adapter fcs on the system .

um_lpar:/>lsdev -Cc adapter |grep fcs
fcs0 Available 06-08 FC Adapter
fcs1 Available 07-08 FC Adapter

From above we can conclude that we have two fiber channel adapters or HBA cards fcs0 and fcs1

Note : There might be more than two Fiber Channel Adapters on the system  in that case we need to confirm what are the adapters are configured for multipath .  Then you might need to run  commands  specific to Storage vendor like for EMC run ”powermt display dev=all” command. For IBM storage we can use lspath , pcmpath query adapter , datapath query adapter etc.

Task 2 : Locate “Network Address” and it’s correspong numeric value means “WWPN Address” on the configuration details of each fcs adapters as below in yellow color.

um_lpar:/>lscfg -vl fcs0
fcs0 U787B.001.WMW8950-P1-C1-T1 FC AdapterPart Number……………..03N6439
EC Level………………..A
Serial Number……………1B550051EA
Manufacturer…………….001B
Customer Card ID Number…..197E
FRU Number……………… 03N6439
Brand…………………..H0
Device Specific.(ZM)……..3
Network Address………….10000000C94EFB3F
ROS Level and ID…………02881955
Device Specific.(Z0)……..1001206D
Device Specific.(Z1)……..00000000
Device Specific.(Z2)……..00000000
Device Specific.(Z3)……..03000909
Device Specific.(Z4)……..FF801413
Device Specific.(Z5)……..02881955
Device Specific.(Z6)……..06831955
Device Specific.(Z7)……..07831955
Device Specific.(Z8)……..20000000C94EFB3F
Device Specific.(Z9)……..TS1.91A5
Device Specific.(ZA)……..T1D1.91A5
Device Specific.(ZB)……..T2D1.91A5
Device Specific.(ZC)……..00000000
Hardware Location Code……U787B.001.WMW8950-P1-C1-T1

Now let see the configuration details of fcs1

um_lpar:/>lscfg -vl fcs1
fcs1 U787B.001.WMW8950-P1-C2-T1 FC AdapterPart Number……………..80P4543
EC Level………………..A
Serial Number……………1F4360AB91
Manufacturer…………….001F
Customer Card ID Number…..280B
FRU Number……………… 80P4544
Device Specific.(ZM)……..3
Network Address………….10000000C9407EEA
ROS Level and ID…………02881955
Device Specific.(Z0)……..1001206D
Device Specific.(Z1)……..00000000
Device Specific.(Z2)……..00000000
Device Specific.(Z3)……..03000909
Device Specific.(Z4)……..FF801413
Device Specific.(Z5)……..02881955
Device Specific.(Z6)……..06831955
Device Specific.(Z7)……..07831955
Device Specific.(Z8)……..20000000C9407EEA
Device Specific.(Z9)……..TS1.91A5
Device Specific.(ZA)……..T1D1.91A5
Device Specific.(ZB)……..T2D1.91A5
Device Specific.(ZC)……..00000000
Hardware Location Code……U787B.001.WMW8950-P1-C2-T1

In the above two output

Alternatively you can run the below command to grep for “Network Address” on each fiber channel adapter fcs to get “WWPN Address”

um_lpar:/> lscfg -vl fcs0|grep “Network Address”
Network Address………….10000000C94EFB3F
um_lpar:/> lscfg -vl fcs1|grep “Network Address”
Network Address………….10000000C9407EEA

Task 3 : We have got two “WWPN Address” for fcs0 and fcs1 respectively as below

10000000C94EFB3F
10000000C9407EEA

Read More

How to Find HBA Card WWPN Address on Linux

We need “WWPN address” information of HBA card to create zoning on fiber channel switch to assign and unassign storage to a Linux host. There are two differnet methods to find “WWPN Address” on Linux depending upon RHEL Linux versions :

1. On Redhat Linux 5 and above . 
2. On Redhat 4 and below.

Now let’s see the two different methods to find “WWPN Address” on Linux depending upon RHEL Linux version

Method 1: On Redhat 5 and above (including OEL and CentOS),the/sys/class/scsi_host/host{n}/device/fc_host:host{n}/port_name file contains the HBA WWPN information.

Where n refers the number of HBA card available on the system . Please substitude n with either 0, 1, or 2 according to your system.

For example:

So from below output we can conclude there are two HBA Card on the system.

$ cd /sys/class/scsi_host/
$ ls
host0 host1

RHEL version is 5.3 as below output

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.3 (Tikanga)

From below output  we got  the WWPN address of HBA host0

$ cat /sys/class/scsi_host/host1/device/fc_host:host0/port_name
0x10000000c9f2da9c

From below output  we got  the WWPN address of HBA host1

$ cat /sys/class/scsi_host/host1/device/fc_host:host1/port_name
0x10000000c9f2da9d

So we have got two “WWPN address” as for the system as below

0x10000000c9f2da9c
0x10000000c9f2da9d

Method 2: On Redhat 4 and below (including OEL and CentOS) , the /proc/scsi/[adapter-type]/[n] file contains the HBA WWPN information.

Where

adapter-type :It can be either qlaxxxx for QLogic adapters (or) lpfc for Emulex adapters.
n refers the number of HBA card available on the system . Please substitude n with either 0, 1, or 2 according to your system.

Read More

EMC PowerPath Powermt Command Examples

If your server has two HBA cards connected to EMC SAN storage device, you can implement load-balancing, and fail-over on the HBA cards using the EMC PowerPath software.

Powermt management utility helps you manage I/O paths from the server to the EMC storage device. In this article, let us discuss how to use powermt command with practical examples.

1. powermt display – Display High Level HBA I/O Paths
2. powermt display dev=all – Display All Attached LUNs
3. powermt display dev=emcpowera – Display specific LUN
4. powermt check_registration – Display PowerPath Registration Key / Status
5. powermt display options – Display EMC PowerPath Options
6. powermt display hba_mode – Display PowerPath HBA Mode
7. powermt display path – Display available I/O Paths
8. powermt displays port_mode – Display Port Status
9. powermt version – Display EMC PowerPath Version
10. powermt check – Check the I/O Paths
11. powermt set mode hba – Forcefully set the mode for a specific HBA
12. powermt remove – Delete an I/O Path
13. powermt config – Configure PowerPath
14. powermt restore – Make Dead I/O Path Alive
15. Powermt Save and Load – Save and Restore PowerPath Configurations

1. powermt display – Display High Level HBA I/O Paths

powermt display command (without any arguments), displays the available HBAs and their current status. The following examples, shows that there are 2 HBAs connected to the server, both the HBAs are in active condition with 2 I/O path’s each. qla2xxx indicates that it is using the Q-Logic HBA card.

This also indicates that the connected EMC device is CLARiiON, as it displays a value for “CLARiiON logical device count” as 1.

# powermt display
Symmetrix logical device count=0
CLARiiON logical device count=1
Hitachi logical device count=0
Invista logical device count=0
HP xp logical device count=0
Ess logical device count=0
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters ---------  ------ I/O Paths -----  ------ Stats ------
###  HW Path                       Summary   Total   Dead  IO/Sec Q-IOs Errors
==============================================================================
   1 qla2xxx                       optimal       2      0       -     0      0
   2 qla2xxx                       optimal       2      0       -     0      0

2. powermt display dev=all – Display All Attached LUNs

This is the most frequently used powermt command, which will display all the attached logical devices to the server.

# powermt display dev=all
Pseudo name=emcpowera
CLARiiON ID=AAA00000000000 [dev-server]
Logical device ID=1111111111111111111 [LUN 1]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 qla2xxx                   sdd       SP A4     active  alive      0      0
   1 qla2xxx                   sde       SP B4     active  alive      0      0
   2 qla2xxx                   sdf       SP A5     active  alive      0      0
   2 qla2xxx                   sdg       SP B5     active  alive      0      0
Pseudo name=emcpowerb
CLARiiON ID=AAA00000000000 [dev-server]
Logical device ID=2111111111111111111 [LUN 2]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP A, current=SP A       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 qla2xxx                   sdh       SP A4     active  alive      0      0
   1 qla2xxx                   sdi       SP B4     active  alive      0      0
   2 qla2xxx                   sdj       SP A5     active  alive      0      0
   2 qla2xxx                   sdk       SP B5     active  alive      0      0

The above indicates that there are two LUNs attached to the dev-server with 4 I/O paths from the server to the SAN storage device.

The above command output indicates the following:

• Pseudo name=emcpowera – The device name that can be used by the server. For example, /dev/emcpowera.
• CLARiiON ID=AAA00000000000 [dev-server] - EMC CLARiiON CX3 serial number and the server name.
• Logical device ID=11111111 [LUN 1] – LUN number. For example, LUN 1.
• state=alive; policy=CLAROpt; – This displays that this particular LUN is valid and using the CLAROpt policy.
• Owner: default=SP B, current=SP B – This indicates that the default (and current) owner for this LUN is storage processor SP B.

3. powermt display dev=emcpowera – Display specific LUN

When there are multiple LUNs connected to a server, you might want to view information about a specific LUN by providing the logical name of the LUN as shown below.

# powermt display dev=emcpowera
Pseudo name=emcpowera
CLARiiON ID=AAA00000000000 [dev-server]
Logical device ID=1111111111111111111 [LUN 1]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 qla2xxx                   sdd       SP A4     active  alive      0      0
   1 qla2xxx                   sde       SP B4     active  alive      0      0
   2 qla2xxx                   sdf       SP A5     active  alive      0      0
   2 qla2xxx                   sdg       SP B5     active  alive      0      0

If the device is not available, you’ll get a “Bad dev value” as shown below.

# powermt display dev=emcpowerc
Bad dev value emcpowerc, or not under Powerpath control.

4. powermt check_registration – Display PowerPath Registration Key / Status

If you’ve lost the PowerPath registration key that you’ve used during the EMC PowerPath installation, you can retrieve it using the following command.

# powermt check_registration
Key AAAA-BBBB-CCCC-DDDD-EEEE-FFFF
  Product: PowerPath
  Capabilities: All

5. powermt display options – Display EMC PowerPath Options

Displays the high level EMC SAN array options as shown below.

# powermt display options
        Default storage system class: all
        Show CLARiiON LUN names:      true
        Path Latency Monitor: Off
        Path Latency Threshold: 0 Sec.
        Storage
        System Class  Attributes
        ------------  ----------
        Symmetrix     periodic autorestore = on
                      status = managed
        CLARiiON      periodic autorestore = on
                      status = managed
        Hitachi       periodic autorestore = on
                      status = managed
        Invista       periodic autorestore = on
                      status = managed
        HP xp         periodic autorestore = on
                      status = managed
        Ess           periodic autorestore = on
                      status = managed
        HP HSx        periodic autorestore = on
                      status = managed

6. powermt display hba_mode – Display PowerPath HBA Mode

This is similar to #1, but displays whether hba is enabled or not, as shown in the last column of the output.

# powermt display hba_mode
Symmetrix logical device count=0
CLARiiON logical device count=1
Hitachi logical device count=0
Invista logical device count=0
HP xp logical device count=0
Ess logical device count=0
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters ---------  ------ I/O Paths -----  Stats
###  HW Path                       Summary   Total   Dead  Q-IOs Mode
==============================================================================
   1 qla2xxx                       optimal       2      0     0 Enabled
   2 qla2xxx                       optimal       2      0     0 Enabled

7. powermt display path – Display available I/O Paths

This displays all available path for your SAN device.

# powermt display paths
Symmetrix logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ Storage System -----    - I/O Paths -
###  HW Path                           ID          Interface     Total    Dead
==============================================================================
CLARiiON logical device count=1
==============================================================================
----- Host Bus Adapters --------- ------ Storage System -----    - I/O Paths -
###  HW Path                           ID          Interface     Total    Dead
==============================================================================
   1 qla2xxx                      AAA00000000000      SP A4          1       0
   1 qla2xxx                      AAA00000000000      SP B4          1       0
   2 qla2xxx                      AAA00000000000      SP A5          1       0
   2 qla2xxx                      AAA00000000000      SP B5          1       0
Hitachi logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ Storage System -----    - I/O Paths -
###  HW Path                           ID          Interface     Total    Dead
==============================================================================
Invista logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ Storage System -----    - I/O Paths -
###  HW Path                           ID          Interface     Total    Dead
==============================================================================
HP xp logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ Storage System -----    - I/O Paths -
###  HW Path                           ID          Interface     Total    Dead
==============================================================================
Ess logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ Storage System -----    - I/O Paths -
###  HW Path                           ID          Interface     Total    Dead
==============================================================================
HP HSx logical device count=0
==============================================================================
----- Host Bus Adapters --------- ------ Storage System -----    - I/O Paths -
###  HW Path                           ID          Interface     Total    Dead
==============================================================================

8. powermt displays port_mode – Display Port Status

Displays the status of the individual ports on the HBA. i.e Whether the port is enabled or not.

# powermt display port_mode
Storage class = Symmetrix
==============================================================================
----------- Storage System ---------------     -- I/O Paths --   --- Stats ---
    ID              Interface         Wt_Q      Total    Dead    Q-IOs  Mode
==============================================================================
Storage class = CLARiiON
==============================================================================
----------- Storage System ---------------     -- I/O Paths --   --- Stats ---
    ID              Interface         Wt_Q      Total    Dead    Q-IOs  Mode
==============================================================================
    AAA00000000000     SP A4           256          1       0        0  Enabled
    AAA00000000000     SP B4           256          1       0        0  Enabled
    AAA00000000000     SP A5           256          1       0        0  Enabled
    AAA00000000000     SP B5           256          1       0        0  Enabled

9. powermt version – Display EMC PowerPath Version

How to identify the version number of EMC PowerPath software?

# powermt version
EMC powermt for PowerPath (c) Version 5.3 (build 185)

10. powermt check – Check the I/O Paths

If you have made changes to the HBA’s, or I/O paths, just execute powermt check, to take appropriate action. For example, if you have manually removed an I/O path, check command will detect a dead path and remove it from the EMC path list.

# powermt check
 Warning: storage_system I/O path path_name is dead.
       Do you want to remove it (y/n/a/q)?

Note: If you want powermt to automatically remove all dead paths, without any confirmation, execute “powermt check force”.

11. powermt set mode hba – Forcefully set the mode for a specific HBA

You can change the mode of a specific HBA to either standby or active using this command. Following example, changes HBA #1′s mode from active to standby.

# powermt set mode=standby hba=1

After the above command, you can see the mode for HBA#1 changed to standby, as shown below.

# powermt display dev=all
Pseudo name=emcpowera
CLARiiON ID=AAA00000000000 [dev-server]
Logical device ID=1111111111111111111 [LUN 1]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 qla2xxx                   sdd       SP A4     standby alive      0      0
   1 qla2xxx                   sde       SP B4     standby alive      0      0
   2 qla2xxx                   sdf       SP A5     active  alive      0      0
   2 qla2xxx                   sdg       SP B5     active  alive      0      0

Note: Path mode can also be “unlic” indicating that you’ve not registered the PowerPath with proper License key.

Note: Some powermt commands (for example, powermt set port_disable), is restricted to certain platform and storage type — On AIX and Solaris, only Fibre channel is supported. On HP-UX, only iSCSI and Fibre are supported. On Linux, Only iSCSI (HBAs) and Fibre are supported.

12. powermt remove – Delete an I/O Path

Use this command to remove any specific I/O path (or) a whole device.

The following example has 4 I/O Paths.

# powermt display dev=all
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 qla2xxx                   sdd       SP A4     standby alive      0      0
   1 qla2xxx                   sde       SP B4     standby alive      0      0
   2 qla2xxx                   sdf       SP A5     active  alive      0      0
   2 qla2xxx                   sdg       SP B5     active  alive      0      0

To remove I/O Path sdd, execute the following powermt remove command.

# powermt remove dev=sdd

# powermt display dev=all
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   1 qla2xxx                   sde       SP B4     standby alive      0      0
   2 qla2xxx                   sdf       SP A5     active  alive      0      0
   2 qla2xxx                   sdg       SP B5     active  alive      0      0

Following will remove all I/O Path for a particular device. i.e To remove /dev/emcpowera, do the following. If the /dev/emcpowera is mounted and used by some program, following command will not work.

# powermt remove dev=emcpowera

13. powermt config – Configure PowerPath

This command checks for available EMC SAN logical devices and add those to PowerPath configuration list. Powermt config command, sets some of the options to it’s default values. For example, write throttling = off, HBA mode = active, CLARiiON policy = CLAROpt, etc.

Possible EMC SAN LUN policy values are: Adaptive, BasicFailover, CLAROpt, LeastBlocks, LeastIos, NoRedirect, Request, RoundRobin, StreamIO, or SymmOpt.

After you execute the powermt config, if you don’t like any of the default values, you should change it accordingly.

# powermt config

14. powermt restore – Make Dead I/O Path Alive

If you have dead I/O paths, and if you’ve done something to fix the issue, you can request PowerPath to re-check the paths and mark it as active using powermt restore command.

When you execute powermt restore, it does an I/O path check. If a previously dead path is alive, it will be marked as alive, and if a previously alive path is dead, it will be marked as dead.

For some reason, if you see the default owner and the current owner of a particular LUN is not the same storage processor, then execute the following command, which will make the current owner of the LUN same as the default owner.

# powermt restore dev=all

Instead of dev, you can also specify class in the powermt restore command. Class can be one of the following depending on your system.

• symm – Symmetrix
• clariion -  CLARiiON
• invista – Invista
• ess – IBM ESS
• hitachi – Hitachi Lightning TagmaStore
• hpxp -  HP StorageWorks XP, or EVA series
• hphsx – HP StorageWorks EMA, or MA
• all – All systems

15. Powermt Save and Load – Save and Restore PowerPath Configurations

powermt save – Save the current Powerpath Configuration

If you are changing the PowerPath configurations for testing purpose, you can save the current configuration using powermt save command.

Use this method to backup the current PowerPath Configurations.

# powermt save

This will save the current powermt configuration to /etc/powermt.custom file. Following is the partial content of this file.

# cat /etc/powermt.custom
global:version:5.3.0.0.0:4.1.0
path_c:sdd:sdd:qla2xxx:0x111
path_c:sde:sde:qla2xxx:0x111
adapter:  1:  1:qla2xxx:0x111:1:0:111:0:11:0
adapter:  2:  2:qla2xxx:0x111:1:0:111:0:111:0
arrPort_c:04000000:04000000:111:111
mpext_cfg:unused:Mp:0:symm:0:0
...
...

If you want to store the current PowePath configuration to a different file, do the following.

# powermt save file=/etc/powermt.21-Apr-2013

powermt load – Load a previously saved EMC PowerPath Configuration

When you are testing EMC PowerPath configuration, if you are not satisfied with the new configuration, you can go back to the previous configuration.

For example, to restore the configuration to a specific file that you created in the above powermt save example, do the following.

# powermt load file=/etc/powermt.21-Apr-2013

Read More