Friday, 29 November 2013

Hitachi Storage Management - AIX

Hitachi Storage Management - AIX

Always its tough to find the commands when we are dealing with the Hitachi storage attached luns in AIX. Hitachi storage uses HDLM (Hitachi Dynamic Link Manager) to manage disks on AIX operating system.

Here i am going to describe about the commands,terms and tips related to Hitachi Storage.

Lets get started.

Location of  Commands & Logs:

The location of the scripts:        /usr/DynamicLinkManager/bin/
DLMManager logs:                    /var/DynamicLinkManager/log
Operation logs (command history):   /var/opt/hitachi/HNTRLib2/spool/hntr2[1-16].log
The location of config file:        /usr/D*/drv/dlmfdrv.conf


1) To seek Help about commands:

# dlnkmgr help 
dlnkmgr { clear | help | offline | online | set | view }

2) View system and drive information:

#dlnkmgr view -help
#dlnkmgr view -sys [ -sfunc | -msrv | -adrv | -pdrv ] [-t] 
#dlnkmgr view -path [ -c | -hdev HostDeviceName ] [-t] 
#dlnkmgr view -path -item [pn] [dn] [lu] [cp] [type] [ic] [ie] [dnu] [hd] 

dlnkmgr view -hba                  shows info about the hbas
dlnkmgr view -lu                   shows the status of the path (online is OK)
dlnkmgr view -path                 shows that which device uses which path
dlnkmgr view -path -srt lu         useful command as: dlnkmgr view -lu -item
dlnkmgr view -lu -item all         gives a full list
dlnkmgr view -cha                  shows storage channel port, serial number of storage box
dlnkmgr view -sys -lic             shows license information
dlnkmgr view -drv                  shows storsge box serial, and iLU numbers

#dlnkmgr view -sys                shows sytem settings (HDLM version,load balance, health checking...)
    root@aix41: / # dlnkmgr view -sys
    HDLM Version                 : 05-94
    Service Pack Version         :
    Load Balance                 : on(rr)
    Support Cluster              :
    Elog Level                   : 3
    Elog File Size (KB)          : 9900
    Number Of Elog Files         : 2
    Trace Level                  : 0
    Trace File Size (KB)         : 1000
    Number Of Trace Files        : 4
    Path Health Checking         : on(30)
    Auto Failback                : off
    Intermittent Error Monitor   : off
#dlnkmgr view -hba
HbaID Port.Bus IO-Count IO-Errors Paths OnlinePaths
00000 08.04     45031              0         4    4
00001 08.03     45999              0         4    4
# dlnkmgr view -path
Paths:000002 OnlinePaths:000002
PathStatus IO-Count IO-Errors
Online 49960 0 

PathID PathName DskName iLU ChaPort Status Type IO-Count IO-Errors DNum HDevName
000000 08.14.0000000000075000.0000 HITACHI .OPEN-V .43194 0085 1G Online Own 25152 0 0 dlmfdrv2
000001 08.3D.0000000000085000.0000 HITACHI .OPEN-V .43194 0085 2G Online Own 24808 0 0 dlmfdrv3
KAPL01001-I The HDLM command completed normally. Operation name = view

3) Clear Error Counters:

# dlnkmgr clear -help
   dlnkmgr clear -pdst [-s]  
 ==> dlnkmgr clear -pdst                  this clears the statistics (I/O counts and I/O errors)

4) Activate/De-Activate dlmfdrv:

The online option allows you to activate a dlmfdrv

# dlnkmgr online -help
dlnkmgr online [-path] [-pathid AutoPATH_ID] [-s] 

Valid value
AutoPATH_ID {000000 - 999999}(Decimal)  

==> dlnkmgr online -pathid 000023      puts a path to online state

The offline option allows you to disactivate a dlmfdrv 

# dlnkmgr offline -help
dlnkmgr offline [-path] -pathid AutoPATH_ID [-s] 

Valid value
AutoPATH_ID {000000 - 999999}(Decimal) 

==> dlnkmgr offline -hba <adapter>     puts an hba to offline state (dlnkmgr offline -hba 08.08)

5)Set various dlm values:

# dlnkmgr set -help 
dlnkmgr set { -lb { on | off } 
| -ellv LogLevel 
| -elfs LogSize 
| -systflv TraceLevel 
| -pchk { on [ -intvl Interval-Time ] | off } 
| -afb { on [ -intvl Interval-Time ] | off } 
| -rsv on ReserveLevel } 

Valid value
LogLevel { 0 | 1 | 2 | 3 } (Default Value 3) 
LogSize { 100 - 9900 }(KB) (Default Value 1000) 
TraceLevel { 0 | 1 | 2 | 3 | 4 } (Default Value 0) 
Interval-Time { 1 - 1440 }(Minute) (Default Value 30) 
Interval-Time { 1 - 1440 }(Minute) (Default Value 1) 
ReserveLevel on { 0 | 2 } (Default Value "on 0") 

dlnkmgr set -help
dlnkmgr set -lb on -lbtype rr        sets load balancing on (rr:round robin, rr is recommended not exrr: extended rr)
dlnkmgr set -pchk on -intvl 10       sets path health checking on (invl: checking interval (in minutes))
dlnkmgr set -afb on -intvl 10        sets automatic failback on
dlnkmgr set -iem on -intvl 20 -iemnum 2 sets intermittent monitoring on (interval 20 min, the number of times the error is to oocur 2)
dlnkmgr set -ellv 3                  sets the error log collection level (3 is recommended, 0:no log, 1:errors only, 2:1+warnings, 3:2+infos)
dlnkmgr set -systflv 0               sets trace level (0 is recommended, after an error could be set to higher (1) to collect logs)
dlnkmgr set -elfs 9900               sets the error log file size in KB (9900 is recommended)
dlnkmgr set -elfn 2                  sets the number of error log files    (2 is recommended)
dlnkmgr set -systfs 1000             sets the trace file size (1000 is recommended)
dlnkmgr set -systfn 4                sets the number of trace files (4 is recommended)
dlnkmgr set -rsv on 2                sets the reservation level (2 is recommended it means persistent res. level, 0:ignore reservation)
                                     (it's important for reserving disks, if a LUN is not supporting it, it won't reserve the disk)

6) Configuring DLM Disks:


7) Adding/Removing Disks from HDLM

Add  disks to HDLM 
If HDLM active – you can remove disks from HDLM without reboot
#/opt/DynamicLinkManager/bin/dlmcfgmgr –a

Remove disks from HDLM with reboot
Comment string with disk description in file /opt/DynamicLinkManager/config/dlmfdrv.unconf
#vi /opt/DynamicLinkManager/config/dlmfdrv.unconf
#/opt/DynamicLinkManager/bin/dlmsetconf -u
#reboot -- -r

8) Miscellaneous Commands:

dlmrmdev                             removes dlm devices (/usr/D*/bin/dlmrmdev) (dlmrmdev -A makes umount, varyoff, rmdev)
dlmpr                                it clears persistent reservation in clustered environment
dlmpr -k hdiskX                      shows the reservation key

dlmpr -c hdiskX                      clears the reservation key

lssrc -a | grep -i dlm      shows if HDLM Manager running (startsrc/stopsrc -s DLMManager)

Hitachi Storage Details:

HBA: The cable port on the host is the host bus adapter (HBA)
CHA: The cable port on the storage subsystem is a port (P) on a channel adapter (CHA)
Product       : 9500V
SerialNumber  : 6123
LUs           : 12

iLU  SLPR HDevName VG OSPathID PathID PathName                        ChaPort CLPR Status     Type IO-Count   IO-Errors  DNum IEP
0062    - hdisk11  -  00000    000000 08.06.0000000000210300.0000     1A         - Online     Own           0          0    0 -
                      00001    000006 09.06.0000000000211300.0000     1B         - Online     Own           0          0    0 -
                      00002    000007 08.07.00000000000E0600.0000     0C         - Online     Non           0          0    0 -
                      00003    000009 09.07.00000000000E1600.0000     0D         - Online     Non           0          0    0 -

Product       : USP
SerialNumber  : 0022137
LUs           : 9

iLU  SLPR HDevName VG OSPathID PathID PathName                        ChaPort CLPR Status     Type IO-Count   IO-Errors  DNum IEP
016F    0 hdisk46  -  00000    000024 08.06.00000000000B8800.0000     3A         0 Online     Own     3201989          0    0 -
                      00001    000025 09.06.00000000000BA600.0000     3E         0 Online     Own     3201945          0    0 -
                      00002    000092 08.07.0000000000240A00.0000     4A         0 Online     Own     3198945          0    0 -
                      00003    000106 09.07.0000000000242A00.0000     4E         0 Online     Own    78042543          0    0 -

PathID:           HDLM manages a path by assigning an ID to it.
OSPathID:         The ID that AIX assigns to a path (lspath shows it as well)
Product:9500V -   HDLM performs load balancing between owner paths or between non-owner paths.
Owner paths are coming from the same CHA. Non-owner paths are coming from the other CHA.
Load is balanced first between the owner paths, if all the owner paths are lost, then between the non owner paths.

Product:USP -     All the paths are owner paths.
                  Load is balanced among all the paths.
    Online        I/O can be issued normally
    Online (E)    An error occured on the path, and none of the other paths are in Online staus (it still can be used)
    Offline (C)   Path was placed offline by using the command (manually)
    Offline (E)   I/O cannot be performed, because an error occurred in the path


Unix System Process Management

What is Process?

When you run a program(command) on your UNIX system , the system creates a special environment for it. This environment contains everything needed for the system to run as if no other programs running on the system the program.

Every time you run a command the system will initiate a process.In other words a process is a basic unit of execution in UNIX (and many other operating systems).

If, for example, three people are running the same program simultaneously, there are three processes there, not just one. In fact, we might have more than one process running even with only person executing the program, because (you will see later) the program can “split into two,” making two processes out of one.

Keep in mind that all Unix commands, e.g. cc and mail, are programs, and thus contribute processes to the system when they are running. If 10 users are running mail right now, that will be 10 processes. At any given time, a typical Unix system will have many active processes, some of which were set up when the machine was first powered up


The operating system process followed by an identification number known as the five-digit identifier or pid . Each process in the system has a unique PID.

PID can be repeated , because all possible numbers have been exhausted and the next pid rolls or back . At any given time , there are no two processes with the same PID in the system.

Finally each process  has its own parent process (PPID) from where its running.There will be there case where multiple processes will be initiated for a UNIX programme.

A process under unix consists of an address space and a set of data structures in the kernel to keep track of that process. The address space is a section of memory that contains the code to execute as well as the process stack.

The kernel must keep track of the following data for each process on the system:
  • the address space map,
  • the current status of the process,
  • the execution priority of the process,
  • the resource usage of the process,
  • the current signal mask,
  • the owner of the process.
  • How to start a Process:
When ever you run a command the system will start a process.There are two ways to  start a process depending upon  the visibility of the process.
  • Foreground Process
  • Background Process

Foreground processes:

By default, all processes will start running in the background . It takes its input from the keyboard , and transmits its output to the screen.

Eg: if you run  ls command  on directory /home/um  , the system will list the contents of the directory "home/um" ,

$ ls /home/um
 check.txt  pink.doc umdir1 umdir2 umdir3

Even if a program is running in the foreground and take some time , its means you can't run the commands on the same terminal as it is waiting for the process to complete which  gives prompt.

Background processes :

Suppose we want to execute a command but do not want to wait for its completion, i.e. we want to be able to issue other commands in the mean time. We can do this by specifying that the command be executed in
the background.

In order to run a command process as  background , we can do by appending an ampersand (‘&’) to the end of the command.

For example, suppose we have a very large program, which will take a long time to compile. We could give the command

which will execute the compilation while allowing me to submit other commands for execution while the compile is running.

If you press the Enter key now, you see the following:
[1]   +   Done                 sh &
This will be tells us the end of the background process.

The advantage of running a background process is that it can  run other commands, you do not have to wait until the end to start another one!

Listing of  processes :

Its is easy to list the process  by using the command ps.

user097@sshell ~ $ ps                                                           
  PID TTY          TIME CMD
31314 pts/4    00:00:00 bash
31401 pts/4    00:00:00 ps

if you want  elaborate information about processes use option "-f"  with 'ps'.

user097@sshell ~ $ ps -ef
user097  30929     1  0 08:36 ?        00:00:00 /usr/local/bin/anytermd -p 2397
user097  31314 30929  0 08:37 pts/4    00:00:00 /bin/bash
user097  31511 31314  0 08:37 pts/4    00:00:00 ps -ef
user097@sshell ~ $  

Here is the description of all the fields displayed by ps -f command:
UIDUser ID that this process belongs to (the person running it).
PIDProcess ID.
PPIDParent process ID (the ID of the process that started it).
CCPU utilization of process.
STIMEProcess start time.
TTYTerminal type associated with the process
TIMECPU time taken by the process.
CMDThe command that started this process.
There are other options which can be used along with ps command:
-aShows information about all users
-xShows information about processes without terminals.
-uShows additional information like -f option.
-eDisplay extended information.

Ending/Stopping a processes:

Ending a process can be done in several different ways. Often, from a console-based command, sending a CTRL + C keystroke (the default interrupt character) will exit the command. This works when process is running in foreground mode.

If a process is running in background mode then first you would need to get its Job ID using ps command and after that you can use kill command to kill the process as follows:

user097@sshell ~ $ ps -ef
user097  30929     1  0 08:36 ?        00:00:00 /usr/local/bin/anytermd -p 2397
user097  31314 30929  0 08:37 pts/4    00:00:00 /bin/bash
user097  31511 31314  0 08:37 pts/4    00:00:00 ps -ef

$kill 30929
Here kill command would terminate first_one process. If a process ignores a regular kill command, you can use kill -9 followed by the process ID as follows:

$kill -9 30929

Parent and Child Processes:

Each unix process has two ID numbers assigned to it: Process ID (pid) and Parent process ID (ppid). Each user process in the system has a parent process.

Most of the commands that you run have the shell as their parent. Check ps -f example where this command listed both process ID and parent process ID.

Zombie and Orphan Processes:

Normally, when a child process is killed, the parent process is told via a SIGCHLD signal. Then the parent can do some other task or restart a new child as needed. However, sometimes the parent process is killed before its child is killed. In this case, the "parent of all processes," init process, becomes the new PPID (parent process ID). Sometime these processes are called orphan process.

When a process is killed, a ps listing may still show the process with a Z state. This is a zombie, or defunct, process. The process is dead and not being used. These processes are different from orphan processes.They are the processes that has completed execution but still has an entry in the process table.

Daemon Processes:

Daemons are system-related background processes that often run with the permissions of root and services requests from other processes.

A daemon process has no controlling terminal. It cannot open /dev/tty. If you do a "ps -ef" and look at the tty field, all daemons will have a ? for the tty.

More clearly, a daemon is just a process that runs in the background, usually waiting for something to happen that it is capable of working with, like a printer daemon is waiting for print commands.

Friday, 1 November 2013

Dealing With Managed System Passwords

Dealing With Managed System Passwords:

There are two sets of the passwords when you are dealing with the power machines.These are very important when you are adding  managed systems to the HMC ( Hardware Management Console).

And those passwords are 
  •  Server (HMC Access) Password
  •  Advanced System Management (ASM) Password

1) Advanced System Management (ASM) Password:

Advanced System Management Interface (ASMI) is a graphical interface that is part of the service processor firmware. The ASMI manages and communicates with the service processor. The ASMI is required to set up the service processor and to perform service tasks, such as reading service processor error logs, reading vital product data, and controlling the system power. There will be seperate password for  ASM to access it.

The ASMI might also be referred to as the service processor menus

2) HMC Access password:

The HMC Access password is a managed system password used to authenticate the HMC to the server. It is one of three managed system passwords set when the system is first installed. The three user IDs whose passwords must be set are admin, general, and HMC (also referred to as HMC Access). 

There is no default password for the HMC ID. The HMC access password is initially clear and the user is prompted to set a password the first time a HMC connects. After setting or entering the HMC access password once, the HMC caches the password locally. It is important to record the password for future use because the password will need to be re-entered if a new or scratch-installed HMC is used.

If the HMC password has not been set, the HMC connection will go to the state Pending Authentication - Password Update Required. The user is prompted to set an initial password for each of the three accounts in the Update Password Pending Authentication window.

If the window does not appear, ensure the user is logged into the HMC as hscroot: Pre-version 7 HMC, right click on the server entry on the console. The window for setting passwords opens. Set the passwords as directed. On Version 7 or later, select the server. Under tasks, select the Update password task then enter the passwords.

Setting Passwords for ASM & HMC Access

When you are trying to add managed system,If you received the message Authentication Pending, the HMC prompts you to set the passwords for the managed system.

If you did not receive the message Authentication Pending, complete the following steps to set the passwords for the managed system.

Updating your server(HMC Access) password

To update your server password, do the following:
  1. In the navigation area, select the managed system.
  2. In the Tasks area, click Operations.
  3. Click Change Password. The Update Password window opens.
  4. Type the required information and click OK.

Updating your Advanced System Management (ASM) general password

Note: The default password for the general user ID is general, and the default password for the administrator ID is admin.
To update your ASM general password, do the following:
  1. In the navigation area of the HMC, select the managed system.
  2. In the Tasks area, click Operations.
  3. Click Advanced System Management (ASM). The Launch ASM Interface window opens.
  4. Select a Service Processor IP Address and click OK. The ASM interface opens.
  5. On the ASMI Welcome pane, specify your user ID and password, and click Log In.
  6. In the navigation area, expand Login Profile.
  7. Select Change Password.
  8. Specify the required information, and click Continue.

Resetting Passwords:

1) ASM Administrator Access Password Reset:

To reset the administrator password, contact an authorized service provider.

2) HMC Access Password Reset:

If the HMC Access password has already been set and an incorrect password is entered, the state is Failed Authentication - Incorrect password. The HMC will prompt the user to enter the HMC access password. After several invalid password attempts, the connection will go into the Failed Authentication - Firmware Password Locked state (0803-0001-00000203). The user must wait five minutes or reset the password in ASMI before any further attempts can be made.

To reset a lost HMC Access password, you should use one of the following methods:

oIf the admin password is known, use ASMI to reset the HMC password.  Refer this link .
Note: The default password for user admin is admin.
oIf the admin password is lost, contact an authorized service provider for information on how to reset both passwords. There are two methods available for the service provider:
- Provide a temporary login for user celogin
- Reset passwords and network settings using the FSP dip switches.

For further information, you should refer to Set Passwords for the Managed System , Reset the Advanced System Management (ASM) administrator password section  refer this link Database 'DCF Technotes (IBM i)', View 'Products', Document 'Changing an ASM Password'.

Resetting HMC Access Password  ( if ASMI  admin password known):

From the Advanced System Management (ASM) login screen, type the user ID and password, and click the Log in button or press the Enter key. 

Note: Ensure the desired system is selected.

Expand Login Profile, select Change Password, and then select the User ID you want to change.

Note: The Current password field indicates the user signed into ASM (not necessarily the User ID being changed).

Note:If the admin password is lost, contact your service provider for a temporary celoginprofile that can be used to reset admin.