Fetching data from Hitachi Command Suite with HiCommandCLI, PowerShell and Python

hcs_ps_hosts2ldevs

You may say it’s not worth writing a post on the subject, just use the Command Suite CLI, Luke and that’s all! But sometimes HCS CLI behavior is not so obvious. Telling the truth it may surprise you.

For example, when tryig to fetch information about particular host, we need to submit a “GetHost” command. Also if we want LDEVs information in that output, we have to specify additional “subtarget=LogicalUnit” parameter:

HiCommandCLI GetHost "http://hcs8.server.name:2001/service" "hostname=test" "subtarget=LogicalUnit" -u user -p password

And it will show a long output similar to the example from Hitachi Command Suite CLI Reference Guide (MK-90HC176-19, p. 4-366/4-367):

RESPONSE:
 An instance of Host
 objectID=HOST.39
 name=test
 hostID=39
 capacityInKB=8,097,280
 hostType=3
 managedBy=4
 osType=VMware
 statusOfDBUpdating=0
 virtualizationServerVersion=VMware ESX 4.0.0 build-171294
 virtualizationServerManagerName=manager01
 virtualizationServerManagerIpAddress=10.197.150.27
 List of 1 WWN elements:
   An instance of WWN
     WWN=10.00.00.00.C9.6F.EB.D6
 List of 6 Lu elements:
   An instance of LogicalUnit
     objectID=LU.R600.10037.1008
     devNum=1,008
     emulation=OPEN-V
     devCount=1
     devType=
     capacityInKB=2,000,000
     numberOfLBAs=4,000,000
     path=true
     commandDevice=false
     commandDeviceEx=0
     commandDeviceSecurity=false
     deviceGroupDefinition=false
     chassis=1
     arrayGroup=0
     raidType=RAID5(3D+1P)
     currentPortController=-1
     defaultPortController=-1
     isComposite=0
     trueCopyVolumeType=Simplex
     shadowImageVolumeType=Simplex
     quickShadowVolumeType=Simplex
     universalReplicatorVolumeType=Simplex
     globalActiveDeviceVolumeType=Simplex
     sysVolFlag=0
     externalVolume=0
     differentialManagement=false
     quickShadowPoolID=-1
     universalReplicatorPoolID=-1
     dpType=-1
     consumedCapacityInKB=2,000,000
     dpPoolID=-1
     threshold=-1
     tcaPoolID=-1
     dpPoolVolControlFlag=-1
     managementAreaPoolID=-1
     <... and so on ...>

If you think, it looks like a piece of crumpled XML with markup tags omitted, you will be absolutely right! HCS CLI communicates with HCS server using XML format.

So, the question is: “May we see plain XML output in HCS CLI”? And the anwser is: “Yes, sure!” Set -f option to specify format:

hcs_cli_opts

But not for “GetHost” command! When you try it HCS CLI returns the nice error:

KAIC90602-E The option "-f" and the command "GetHost" cannot be specified at the same time.

And it is very weird because of an XML nature of the HCS API protocol!

The same error is for creating a csv table! So we have to do all the thing manually to parse CLI output while preparing our simple storage consumption report.

For me, the easiest way is to write a script and connect with the HCS server directly (skipping an excessive Java HiCommandCLI interlayer) to send commands and receive responses in XML. And here is my example on how to do it in Pyton:

#!/usr/bin/python3

# -----------------------------------------------------------------------------
# "THE BEER-WARE LICENSE" (Revision 42):
# zmey20000@yahoo.com wrote this file. As long as you retain this notice you
# can do whatever you want with this stuff. If we meet some day, and you think
# this stuff is worth it, you can buy me a beer in return Mikhail Zakharov
# -----------------------------------------------------------------------------

# Hitachi Command Suite XML API usage example in Pyton.
 
import base64
import http.client
import xml.etree.ElementTree as ET
 
# ----------------------------------------------------------------------------
hcs_server = 'hcs8.server.name'
login = 'system'
password = 'manager'
command = 'StorageArray'
option = 'all'
 
# ----------------------------------------------------------------------------
b64lp = base64.b64encode((login + ':' + password).encode('ascii'))
 
body = """<?xml version="1.0" encoding="UTF-8"?>
<HiCommandServerMessage>
    <APIInfo version="7.6" />
    <Request>
       <StorageManager> 
            <Get target=""" + '"' + command + '" ' + """option=""" + '"' + option + '" ' + """>
                <StorageArray />
            </Get>
        </StorageManager>
    </Request>
</HiCommandServerMessage>"""
 
headers = {'Content-Type': 'text/xml',
           'User-Agent': 'Deck Eight:One Step:0',
           'Authorization': 'Basic ' + b64lp.decode('utf-8')}
 
httpc = http.client.HTTPConnection(hcs_server, 2001, timeout=30)
httpc.request('POST', '/service/ServerAdmin', body, headers)
resp = httpc.getresponse()
#print(resp.status, resp.reason)
 
data = resp.read()
data = data.decode('utf-8')
 
root = ET.fromstring(data)
for array in root.iter('StorageArray'):
    name = array.get('name')
    hardwareRevision = array.get('hardwareRevision')
    cacheInMB = array.get('cacheInMB')
    totalFreeSpaceInKB = array.get('totalFreeSpaceInKB')
    allocatedCapacityInKB = array.get('allocatedCapacityInKB')
    capacityInKB = array.get('capacityInKB')
    numberOfControllers = array.get('numberOfControllers')
    controllerVersion = array.get('controllerVersion')
    productName = array.get('productName')
    arrayType = array.get('arrayType')
    serialNumber = array.get('serialNumber')
 
    print(name, serialNumber, arrayType, productName, totalFreeSpaceInKB, allocatedCapacityInKB, capacityInKB, controllerVersion, numberOfControllers, cacheInMB, hardwareRevision)
 
httpc.close()

But sometimes we are bound to Windows and its native components. I have spend a day, lerning PowerShell, so here is my script which gets information about all Hosts and their LDEVs registered within the given HCS server:

# -----------------------------------------------------------------------------
# "THE BEER-WARE LICENSE" (Revision 42):
# zmey20000@yahoo.com wrote this file. As long as you retain this notice you
# can do whatever you want with this stuff. If we meet some day, and you think
# this stuff is worth it, you can buy me a beer in return Mikhail Zakharov
# -----------------------------------------------------------------------------

#
# Dump and correlate Hitachi Command Suite Hosts and LDEVs information
#

#
# v1.0	2016.11.30	Initial release
# v1.1	2016.12.16	Storage serial and model/type added

# Usage:
# hcs_hosts2ldevs -hcs_host hcs8.server.name -hcs_user user -hcs_pass password -hcs_cli X:\HCS\CLI\HiCommandCLI.bat -out_csv X:\path\to\your.csv


# Defaul values ---------------------------------------------------------------
param (
    [string]$hcs_host = "hcs8.server.name",
    [string]$hcs_user = "system",
    [string]$hcs_pass = "manager",
    [string]$hcs_cli = "C:\HCS_CLI\HiCommandCLI.bat",
    [string]$out_csv = "HCS-Hosts2LDEVs.csv"
)

# -----------------------------------------------------------------------------
$hcs_url="http://" + $hcs_host + ":2001/service"

Write-Host "Querying HCS data. You have time for a cup of coffee"

$flist = & $hcs_cli $hcs_url GetHost "subtarget=LogicalUnit" -u $hcs_user -p $hcs_pass | 
    where {$_ -cmatch "name=|capacityInKB=|osType=|instance|WWN=|displayName=|emulation=|consumedCapacityInKB|commandDevice|arrayGroupName|raidType|externalVolume|dpType|dpPoolID|objectID"}

Write-Host "Processing HCS data. Keep calm and enjoy your drink"
$i = 0
Add-Content $out_csv "Hostname,Host capacity (MB),OS,WWN,LDEV,LDEV capacity (MB),Storage System SN,Storage System Type,LDEV Used (MB),Emulation,Array Group,RAID level,Command Device,External,DP Type,DP Pool ID"
foreach ($ln in $flist) {

    # Show some progress indication
    $i += 1
    if ($i % 10000 -eq 0) {
        Write-Host Row $i : $flist.Length processed
    }    	
    
    # Split every line to fetch 'variable' and 'value' parts
    $hash=$ln.Trim().Split('=')

    # Fetch Host/WWN/LDEV data and combine everything together to prepare normal table format
    switch ($hash[0]) {
        "An instance of Host" {
            if ($LUN -ne "") {
                $LUNs += $LUN + $SS + $ST + $LUsed + $Emul + $RG + $RGLvl + $CMDDev + $Ext + $DPType + $DPPool
            }
            
            foreach ($wc in $WWNs) {
                foreach ($lc in $LUNs) {
                    if ("$Hst$OS$wc$lc" -ne "") {
                        Add-Content $out_csv $Hst$OS$wc$lc
                    }
                }
            }

            # Clean variables for the next host
            $Hst = ""
            $LUN = ""
            $OS = ","
            $Emul = ",";
            
            $WWNs = @()
            $LUNs = @()
            
            # We are at Host Level: 0
            $l = 0
            break
        }
        
        "name" {
            $Hst += $hash[1]
            break
        }

        "capacityInKB" {
            # Capacity can be found on Level 0 and Level 2
            switch ($l) {
                0 {$Hst += "," + $hash[1].Replace(".", "")/1024; break}
                2 {$LUN += "," + $hash[1].Replace(".", "")/1024; break}
            }               
        }

        "osType" {
            $OS = "," + $hash[1]
            break
        }
        
        "An instance of WWN" {
            # Go down to WWN Level: 1
            $l = 1
            break
        }

        "WWN" {
            $WWNs += "," + $hash[1].Replace(".", ":").ToLower()
            break
        }
        
        "An instance of LogicalUnit" {
            if ($LUN -ne "") {
                $LUNs += $LUN + $SS + $ST + $LUsed + $Emul + $RG + $RGLvl + $CMDDev + $Ext + $DPType + $DPPool
                $LUN = ""; $SS = ""; $ST = ""; $Emul = ","; $LUsed = ","; $CMDDev = ","; $RG = ","
                $RGLvl = ","; $Ext = ","; $DPType = ","; $DPPool = ","
            }
            
            # We are finally at LDEV Level 2 deep
            $l = 2
            break
        }
        
        "displayName" {
            $LUN += "," + $hash[1]
            break
        }
		"objectID" {
			if ($l -eq 2) {
			# Storage serial
				$SS = "," + $hash[1].Split('.')[2]
			# Storage type/model
				$ST = "," + $hash[1].Split('.')[1]
			}
			break
		}
        
        "emulation" {
            $Emul = "," + $hash[1]
            break
        }
        
        "commandDevice" {
            switch ($hash[1]) {
                "false" {$CMDDev = "," + "0"}
                "true" {$CMDDev = "," + "1"}
            }
            #$CMDDev = "," + $hash[1]
            break
        }
        
        "arrayGroupName" {
            $RG = "," + $hash[1]
            break
        }
        
        "raidType" {
            $RGLvl = "," + $hash[1]
            break
        }
        
        "consumedCapacityInKB" {
            $LUsed = "," + $hash[1].Replace(".", "")/1024 
            break 
        }
        
        "externalVolume" {
            $Ext = "," + $hash[1]
            break;
        }
        
        "dpType" {
            $DPType = "," + $hash[1]
            break
        }
        
        "dpPoolID" {
            $DPPool = "," + $hash[1]
            break
        }
    }  
}

# Must process final line as it was not created by the main loop

if ($LUN -ne "") {
    $LUNs += $LUN + $SS + $ST + $LUsed + $Emul + $RG + $RGLvl + $CMDDev + $Ext + $DPType + $DPPool
}

foreach ($wc in $WWNs) {
    foreach ($lc in $LUNs) {
        Add-Content $out_csv $Hst$OS$wc$lc
    }
}

Write-Host "Done."

As a result it creates an csv file.

Just run the command below:

hcs_hosts2ldevs -hcs_host hcs8.server.name -hcs_user user -hcs_pass password -hcs_cli X:\HCS\CLI\HiCommandCLI.bat -out_csv X:\path\to\your.csv

to get the table shown in the picture at the beginning of the post. Hope everything works for you, because if you didn’t add Hosts into Hitachi Command Suite you have to scan for host-groups on every port of all the storage systems registered in the HCS server. But this is a completely different story. Good luck!

Advertisements

About mezzantrop

10 years of experience in large SAN and storage environments: mainly Hitachi, HP and Brocade. Now I am a proud SAN/storage IBMer. Empty – expect-like tool author. FreeBSD enthusiast.
This entry was posted in My projects, Storage and tagged , , , , , , , , , . Bookmark the permalink.

One Response to Fetching data from Hitachi Command Suite with HiCommandCLI, PowerShell and Python

  1. Pingback: Online command-line reference for scripting | #define me human

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s