Improving the VMAX ReportPack Part 2: Electric Boogaloo

The title of this post of course comes from the timeless 1984 classic Breakin' 2: Electric Boogaloo. The fact that every sequel to every movie has not had the subtitle Electric Boogaloo since then is one of history's greatest tragedies. Who among us would not have loved to watch Arnold Schwarzenegger struggle to out-dance the T1000 to stop it from bulldozing a local recreation center in Terminator 2: Electric Boogaloo?

Today we'll tweak the stock VMAX ReportPack to help with another common workflow. You've noticed that a particular FA is getting very hot and causing host latency, but there are many hosts attached to it. How do we track down the chattiest hosts so we can move them to less busy FAs?

Let's start by exporting the report definition so we can import it into My Reports. Why export instead of just copy/pasting? Because exporting the definition will dereference all the links, which will be useful for making our changes while guaranteeing we don't screw anything up.

Once it's imported into your Sandbox, we'll add a node for our first addition, a Top N report of the busiest storage groups. We'll need to connect to the associated Masking Views, which are parttype Access. If you take a look at an Access part in Management of Database Metrics, you'll see that there is a property called director which is a comma separated list of all the directors in the Masking View. That means we're going to need to use a type of Complex Expansion called an 'in-list'.

Remember, the complex expansion wizard is there to help. The first step lets you choose a template. In the second step you choose the property in your current dataset. In the third, we choose the matching property in the target dataset. The third set will also let us do fancy tricks, like split up the source into a lot of pieces first, or match a substring or regular expression on the target.

Since we're trying to match the part from our Controller with a member of the list director from an Access, the expansion configuration will go like this:

Of course the Filter will be parttype=='Access' & vstatus=='active' applied to Selection Only. Since this filter/expansion combo will return many parts of type Access, we'll want to build a child node with a Simple Expansion on part.

This gets us connected to a list of Masking Views, but all of the good statistics we want are associated with parts of parttype Storage Group, so Masking View will just be a stepping stone. We'll create a second-level child node with Filter parttype=='Storage Group' & vstatus=='active' applied to Selection Only. This time we just want to connect the sgname on our Access to the part.

Both of these complex expansions have a bug as configured so far: in a multi-VMAX environment you'll end up connecting an FA on one VMAX to a Masking View on a different one. To deal with that we'll add a second complex expansion to each one which joins device to device. This ensures we stay on the same array.

Finally, we'll want to sum the ReadRequests and WriteRequests on the Storage Group into a result called IOPS and Nop it up to the top node.

The overall configuration will look something like this:

Which gives us an item in the FA drilldown that looks something like this:

It sure would be useful to get a list of hosts associated with that Storage Group, wouldn't it? Let's add a node that will split the devices property (which is a comma-separated list of LUNs) on our Access and connect the results to parttypes of LUN.

Once that's done we can cheat to get a list of hosts by using the Host Linker located in <<Templates>>.

Copy/paste this node undernearth the LUN splitting node and most of the work is done for us. We'll just do a few extra things:

Add property part from the connected LUNs into a Nop formula for display
Nop the DeviceName property from the Host Linker up a level for display
Copy/paste the IOPS formula from the Storage Group node onto the LUN node. You'll want to change the result name to something like LUNIOPS.
Change the child of the TopN report into a table with columns for LUN name, Hosts, and IOPS. We can sort by IOPS to get the busy ones at the top.
Change the Host Linker's report configuration so it's Always Visible

The result is that we can now drill into the busiest Masking Views to find the busiest LUNs within them, and then drill to details on the attached hosts.

Pretty slick, huh?

Actually it's still not everything I want. For example, it would be nice to drill from a LUN down to a Top-N list of the Virtual Machines using that LUN by IO. That, however, will require a custom Host Linker, which I think will make a good post for another day.

Download Front-End Directors Summary Definition v2