Search

Session Metrics

Overview

As of Login Enterprise 5.5, Session Metrics can be displayed in charts for Load Tests and Continuous Tests. Administrators can collect additional performance metrics from Windows Performance Monitor (PerfMon) and Windows Management Instrumentation (WMI) to display with EUX Score and Application metrics. When enabled, Session Metrics are captured every 5 seconds by default.

You can link a Session Metric Group to specify performance counters for Load Tests and Continuous Tests. A Session Metric Group is a new entity that organizes performance counters as you define them. The Group can contain metrics sourced from PerfMon and WMI if desired, allowing you to gather a unique dataset that meets your needs.

The system captures and displays the percentage of CPU and Memory Utilization by default. As of release 5.5, to create custom session metrics, you need to use the Public API. Before 5.4, only application response timers and the EUX Score were captured for Continuous and Load Tests.

In Additional resources, you’ll find a link with examples of WMI and PerfMon Session Metric definitions. Note that these are provided as examples only. Login VSI has limited ability to recommend particular metrics for data collection. The customer is responsible for understanding and defining metrics for their monitoring needs.

Why use Session Metrics?

Administrators can now enable Session Metric collection from target Windows VDI or PCs during Load Testing. Reviewing test results to correlate spikes in degraded EUX Scores or VSImax density with relevant performance counters, such as CPU, Memory, Disk Utilization, and other metrics is now easier. These Session Metrics may provide root cause analysis clues as to where, or why, the expected density or performance is not achieved. Specifically, you can use Session Metrics to pinpoint what agents, apps, or processes are consuming large amounts of resources, impacting the overall performance and scalability of the system. With a fuller view of the workspace under stress, teams now have the data they need to make decisions, fine-tune optimization settings, and measure the impact of changes to the workspace in detail. When comparing two results, Session Metrics can identify granular changes to the consumption of key performance indicators.

For Continuous Testing, administrators can now capture Session Metrics in real-time, 24/7, without requiring an extra agent. It is now easier to correlate spikes in performance counters, such as CPU and Memory Utilization, with degradation in the EUX Score, Logon Times, or application response timers. Specifically, Session Metrics for targeted leading indicators can provide an early warning system for infrastructure degradation that can be supplemented with EUX Scores and Application Performance for further evidence of an issue.

With the addition of Session Metric capabilities, application-specific resource utilization can be measured across version updates and upgrades. WMI offers unique visibility into GPU usage for effective performance testing of graphics-intensive workloads and fine-tuning configuration settings. There are also platform-specific performance metrics, such as ICA/HDX and Blast protocol information that you can correlate with Zoom and Teams workloads to measure video quality during sizing, e.g. Frames/sec.

Tech tips

When you enable Session Metric collection using Custom Metrics Groups, note that adding additional metrics to a Group increases resource consumption. This may impact the expected EUX Score and VSImax results. For this reason, consider capturing a benchmark of your EUX and VSImax before enabling Session Metrics. This will help assess the impact of collecting the session metrics you’re interested in.

Before you create a WMI query, define the performance counters you want to collect. This will minimize query overhead. For better performance, choose PerfMon over WMI when a specific session metric is available in both sources.

Creating custom Session Metrics

In evolving environments, the demand for customized session metrics becomes increasingly apparent. Customizing session metrics in Login Enterprise enables you to gather more detailed and personalized data, facilitating the generation of reports that are specific to your environment. For instance, in Continuous Testing scenarios, this capability provides real-time insights finely tuned to your unique setup. Leveraging these metrics lets you optimize testing processes and enhance the overall customer experience.

You can create custom Session Metrics in one of the following ways:

The first step you need to take is to define the performance counters you want to collect.

Windows Performance Monitor (PerfMon)

Performance counter details for a PerfMon Session Metric definition

Windows Performance Monitor (PerfMon) is a Windows tool designed for analyzing the performance of applications and the overall system. It offers a variety of performance counters covering CPU usage, memory usage, disk activity, and network performance. Each counter will have details such as its name, description, and instance information.

The steps below explain how to find details about specific performance counters in PerfMon on your Windows system:

  1. Launch PerfMon:
    • Open Performance Monitor on your Windows system. You can usually find it in the "Administrative Tools" or by searching for "perfmon" in the Start menu.
  2. Add the Desired Counter:
    • In PerfMon, click "+" to add a performance counter.
    • Choose the performance object (category), counter (specific metric), and, if applicable, instance (e.g., for a specific process).

For example, suppose you want to monitor the "% Idle Time" performance counter for logical disks. This metric reports the percentage of time during the sample interval that the selected disk instance, or aggregate of all disk instances, was idle. In the following example, the red outline indicates the 'counterCategory,' the yellow outlines the 'counterName,' and the purple highlights the 'counterInstance'. Make sure you have copied the counterCategory, counterName, and counterInstance of your performance counter, for example, in Notepad as you will need it in further steps.

1.png

You can adapt the formula for other performance counters depending on your monitoring scenario.

Once you’ve identified the PerfMon performance counters you’d like to collect with Login Enterprise, you need to create and import your Session Metric definition.

Structuring a PerfMon Session Metric definition for an API request

To structure a PerfMon Session Metric definition, use a JSON schema. Start by defining a JSON schema that outlines the structure of a PerfMon Session Metric. The schema will include properties such as the type of metric, counter category, counter name, counter instance, display name, and unit.

Make sure to select the relevant schema format for your version of Login Enterprise, as there was a schema change between versions 5.4 and 5.5.

Here's a template of a JSON schema for a LogicalDisk performance counter:

// 5.5 schema
{
"type": "PerformanceCounter",
"measurement": {
"counterCategory": "LogicalDisk", 
"counterName": "% Idle Time", 
"counterInstance": "_Total", 
"displayName": "Disk Idle time",
"unit": "%"
},
"tag": ""
}

// 5.4 schema
{
"type": "PerformanceCounterDefinition",
"counterCategory": "LogicalDisk", 
"counterName": "% Idle Time", 
"counterInstance": "_Total", 
"displayName": "Disk Idle time",
"unit": "%"
"tag": ""
}

In this example:

"type": specifies the type of the metric, which is "PerformanceCounter" as per the schema.

"measurement": an object containing the properties specific to the performance counter.

"counterCategory": indicates that the performance counter is related to the "LogicalDisk" category.

"counterName": specifies the particular counter within the "LogicalDisk" category, in this case, "% Idle Time."

"counterInstance": (optional): represents the instance of the counter. In this example, it's "_Total."

"displayName": provides a user-friendly display name for the metric, such as "Disk Idle time".

"unit": specifies the unit of measurement for the counter, which is "%". This value will be shown as the y-axis label for charts showing the metric. Note that charts display simultaneously only metrics with the same unit.

"tag": (optional): allows you to sort, filter, and search for information in future Login Enterprise versions.

You can customize the values of these properties based on your specific use case or the actual performance metrics you are monitoring.

Now that you've structured your JSON object, you need to use it in an API request to create your Session Metric definition. This way, you will be able to add a Session Metrics definition to a Session Metric Group, and edit or delete it as necessary.

Windows Management Instrumentation (WMI)

Performance counter details for a WMI Session Metric definition

Windows Management Instrumentation (WMI) is a powerful management framework in Windows for monitoring, configuration management, event handling, and remote administration of your system. WMI provides more detailed information than PerfMon, making it preferred in certain scenarios.

The steps below explain how to find details about WMI classes and their specific performance counters.

To identify available WMI performance counters, start with collecting available WMI classes. To do this, open the PowerShell console on your Windows system.

  1. Using the Start Menu:
    • Click on the "Start" button.
    • Type "PowerShell" in the search bar.
    • Click on "Windows PowerShell" or "PowerShell" to open the PowerShell console.
  2. Using Run:
    • Press Win + R to open the Run dialog.
    • Type "powershell" and press Enter.

To retrieve a list of all available WMI classes that can be queried, use the following command:

Get-WmiObject Win32_PerfFormattedData | Select-Object -Unique __Class | Format-Table -AutoSize

To run the command, press Enter.

2.png

In this output, some examples of significant WMI classes related to performance monitoring include:

  • Win32_PerfFormattedData_PerfOS_Processor
    • Provides processor performance information, including CPU usage, interrupts, and related statistics.
  • Win32_PerfFormattedData_PerfOS_Memory
    • Offers performance data related to system memory usage, including available bytes, cache size, and memory-related rates.
  • Win32_PerfFormattedData_PerfDisk_LogicalDisk
    • Provides performance data related to logical disks, including metrics for disk transfer rates, queue lengths, and average response times.
  • Win32_PerfFormattedData_GPUPerformanceCounters_GPUEngine
    • Contains GPU engine performance data, offering metrics for GPU utilization, video decoding, and processing activity.
  • Win32_PerfFormattedData_PerfOS_System
    • Offers performance data related to the overall system, including metrics such as context switches, system calls, and other counters.
  • Win32_PerfFormattedData_PerfProc_Process
    • Provides performance data related to individual processes, including CPU and memory usage.
  • Win32_PerfFormattedData_Tcpip_NetworkAdapter
    • Includes performance data related to network adapters, offering metrics such as bytes sent and received per second, packet rates, and error counts.
  • Win32_PerfFormattedData_Counters_VMwareBlastImagingCounters (specific to VMware Horizon only)
    • Contains performance data related to VMware Blast Imaging, offering metrics for imaging operations and bandwidth usage in a VMware Horizon environment.

For example, suppose you want to want to query the PhysicalDisk class. This WMI class provides performance data specifically related to physical disk drives on a Windows system. It includes various performance counters that allow you to monitor the performance and activity of individual physical disk drives. You can collect such metrics as DiskReadsPerSec (the rate at which read operations are performed on the disk) and DiskWritesPerSec (the rate at which write operations are performed on the disk) among others.

To retrieve all counters for the WMI class (in this example, PhysicalDisk), run the following command:

Get-WmiObject -Query "SELECT * FROM Win32_PerfRawData_PerfDisk_PhysicalDisk"

The SELECT * part of the query specifies that you want to retrieve all properties or counters available for each instance of the PhysicalDisk class, so the output may be extensive.

3.png

If you are looking for specific counters or properties, you can modify the query to include only the ones you are interested in. For example, suppose you want to collect the DiskWritesPerSec and DiskReadsPerSec metrics.

To retrieve specific counters (DiskWritesPerSec, DiskReadsPerSec), modify the command in the following way:

Get-WmiObject -Query "SELECT DiskWritesPerSec, DiskReadsPerSec FROM Win32_PerfRawData_PerfDisk_PhysicalDisk"

4.png

You can replace DiskWritesPerSec and DiskReadsPerSec with the specific counters you want to retrieve. By specifying only the desired properties in the SELECT statement, you focus the data returned and limit the performance overhead when polling the target system.

Once you’ve defined the performance counters for a WMI Session Metric definition, you need to create a WMI Session Metric definition.

Structuring a WMI Session Metric definition for an API request

As you’ve learned, some WMI classes will have multiple instances. In the following JSON example, the "Win32_PerfRawData_PerfDisk_PhysicalDisk" WMI class has two instances, namely "C:" and "_Total". Since no particular instance is specified, the 'summarizeOperation' will aggregate all available instances of the class, calculating the average 'DiskWritesPerSec' and 'DiskReadsPerSec'.

If you’re looking for a more targeted scenario, you may want to define a specific instance over the aggregate. This allows you to tailor your performance dataset towards your specific scenario whether it is performance analysis, troubleshooting, or optimization.

{
"type": "WmiQuery",
"wmiQuery": "SELECT DiskWritesPerSec, DiskReadsPerSec FROM Win32_PerfRawData_PerfDisk_PhysicalDisk",
"namespace": "root\cimv2",
"instanceField": "Name",
"measurements": [
{
"displayName": "Avg Writes Per Sec",
"unit": "OPS",
"propertyName": "DiskWritesPerSec",
"summarizeOperation": "avg", 
}, 
{
"displayName": "Avg Reads Per Sec", 
"unit": "OPS",
"propertyName": "DiskReadsPerSec",
"summarizeOperation": "avg"
} 
],
"tag": "diskUtilization"
}

In this example:

"type": specifies the type of query that collects metrics from WMI, which is "WmiQuery" as per the schema.

"wmiQuery": contains the WMI query string to retrieve the metrics you’re interested in. The string has to be in the (in the SELECT, FROM, WHERE format).

"namespace": (optional): defines the WMI namespace to specify the database for the query. If left blank, the default namespace "root\cimv2" will be used.

"instanceField": specifies the field name describing the instance, not the actual instance name. For example, if you use '_Total' as an instance, you would reference the 'Name' column in the dataset listing the instances.

"measurements": defines an array of measurements to be extracted from the WMI query results.

"displayName": provides a user-friendly display name for the measurement of the WMI metric.

"unit": Specifies the unit of measurement for the WMI metric. This value will be shown as the y-axis label for charts showing the metric. Note that charts display simultaneously only metrics with the same unit.

"propertyName": specifies the property name corresponding to the WMI property.

"summarizeOperation": specifies the summarization operation for the measurement. This way, you can aggregate multiple instances to generate results using operations such as min, max, avg, and sum. If the summarizeOperation is set to 'none,' up to 16 instances will be returned in a random order.

"tag": (optional): allows you to sort, filter, and search for information in future Login Enterprise versions.

You can customize the values of these properties based on your specific use case or the actual performance metrics you are monitoring.

Now that you've structured your JSON object, you need to use it in an API request to create your Session Metric definition. This way, you will be able to add a Session Metric Definition to a Session Metric Group, and edit or delete it as necessary.

The procedure for creating a WMI Session Metric definition in the Swagger API interface is the same as the procedure for creating a PerfMon Session Metric definition. For more information, see Creating a Session Metric in the Login Enterprise Public API.

Creating a Session Metric in the Login Enterprise interface

Creating a Session Metric

This section will walk you through the process of creating custom Session Metrics using PerfMon and WMI.

To add a custom Session Metric:

  1. In the Sidebar menu, go to Configuration > Session metrics > Metrics.
  2. On the top-right, click Add new session metrics. Note that the CPU and Memory utilization metrics are built-in (default ones) and cannot be deleted.

    5.png

  3. In the Create new session metric window, provide the general information for your session metric:
  • Name: Provide a unique and descriptive name for your custom metric.
  • Description (Optional): Add a brief description of the custom metric, if applicable.
  • In Source for session metrics, select the source you’d like to collect session metrics from, PerfMon or WMI.

6.png


Creating a PerfMon Session Metric definition

In Settings, provide the following information for a PerfMon session metric definition:

  • Counter Category: The category (object) of the performance counter.
  • Counter Name: The name of the performance counter.
  • Counter Instance (optional): An instance of the performance counter, if applicable.
  • Display Name: A user-friendly name for charts and reports.
  • Unit: The unit of measurement for the performance counter. This label appears on the y-axis for charts showing the metric. Note that charts only display metrics with the same unit simultaneously.
  • Save your settings to apply the changes. If you have more metrics to add, continue following the steps for each.

Once session metric definitions are created, you can't edit the settings. This ensures the integrity and accuracy of your test results.

Once you’ve created a PerfMon Session Metric definition, you need to create a Group for Session Metric definitions.

7.png


Creating a WMI Session Metric definition

In Settings, provide the following information for your WMI session metric definition:

  • WMI Query: The WMI query statement for data collection. Use the SELECT, FROM, WHERE format.
  • Name Space: The WMI namespace to specify the database for the query. When left blank, the default namespace, "root\cimv2" will be used.
  • Instance Field: The name of the field that describes the instance. Note this is NOT the actual instance name. For example, if you use '_Total' as an instance, you would reference the 'Name' column in the dataset listing the instances.

8.png


Adding a WMI measurement

Adding measurements is key for monitoring system performance, troubleshooting, and establishing baselines. Measurements provide the data needed for performance analysis, capacity planning, and proactive issue identification. They enable alerts, trend analysis, and optimization, supporting effective system management and adherence to service level agreements (SLAs).

To create a new measurement, click Add Measurement and provide the following information:

  • Property name: The name of the property for data collection.
  • Summarize operation: The summarization operation for the measurement, e.g. Min, Max, Avg, etc. If set to None, up to 16 instances will be returned in a random order.
  • Display name: A user-friendly name for charts and data storage.
  • Unit: The unit of measurement for the WMI metric. This label appears on the y-axis for charts showing the metric. Note that charts only display metrics with the same unit simultaneously.
  • Save your settings to apply the changes. If you have more metrics to add, continue following the steps for each.

Once session metric definitions are created, you can't edit the settings. This ensures the integrity and accuracy of your test results.

Once you’ve created a WMI Session Metric definition, you need to create a Group for Session Metric definitions.

9.png

Creating a Group for Session Metric definitions

A Group is a collection of previously specified session metrics. Note that at the test level, you can connect only one Group. The Group can have as many session metric definitions as you would like, but the test can only contain one group.

To create a new Group for Session Metric definitions:

  1. In the Sidebar menu, go to Configuration > Session metrics > Groups.
  2. On the top-right, click Add new group.

    10.png

  3. In the Add new group window, provide the general information for the Group:
  • Name: Provide a name for the Group to easily identify and categorize related metrics.
  • Description (Optional): Add a brief description of the Group, if applicable.
  • Save your settings to apply the changes. If you have more Groups to add, continue following the steps for each.

Once you’ve created a Group, you need to add Session Metric definitions to a Group.

11.png

Adding a Session Metric definition to a Group

Once you’ve created the Group, you’ll see an option to add session metrics definitions as the next step.

To add a session metric definition to a Group:

  1. Click Add session metric definitions.

    12.png

  2. In Add session metric definitions, select checkboxes next to the session metric definitions you want to add to a Group.
  3. Click Save.

13.png

Once you’ve added Session Metric definitions to a Group, you need to link the Group to a Test. This way the Test will capture the metrics specified in the Group.

Linking a Group to a Test

To learn how to link a Group with Session Metric definitions to a Test, see Linking a Group to a Test.

Creating a Session Metric in the Login Enterprise Public API

Accessing the Public API in the Login Enterprise

To access the Public API in the Login Enterprise, follow these steps:

  1. Log in to the Login Enterprise with your credentials
  2. In the Sidebar menu, scroll down and find External notifications.

    14.png

  3. In External notifications > tab bar menu, select Public API.

    15.png

  4. In Public API, click Public API documentation (this will redirect you to the API Reference).

    16.png

  5. Under Download OpenAPI specification > v7.0 preview, click API Console Swagger (the Swagger API interface with the v7-preview Public API will open).

17.png

Creating a Session Metric definition

  1. In the OpenAPI Specification, scroll down and find UserSessionMetricDefinition
  2. In UserSessionMetricDefinition, click POST /v7-preview/-user-session-metric-definition

    18.png

  3. In the top-right corner, click Try it out (The Request body field will open).

    19.png

  4. Paste the JSON object you’ve defined in Structuring a PerfMon Session Metric definition for an API request into the Request body field.
  5. Click Execute.

20.png

If successful, the API response will return a key (ID) of a new Session Metric definition. Copy the ID to the clipboard as you will need this ID when adding your Session Metric to a Session Metric Group.

{
"id": "755ef1b3-0a2d-4718-9746-3eabbcc69229"
}

Creating a Group for Session Metric definitions

  1. In the OpenAPI Specification, scroll down and find SessionMetricDefinitionGroup.
  2. In SessionMetricDefinitionGroup, click POST /v7-preview/-session-metric-definition-groups.
  3. In the top-right corner, click Try it out (The Request body field will open).
  4. In the "name" parameter, specify the name of your Group, e.g. Custom Metrics.
  5. The "description" field is optional. You can fill it out to provide additional information about the parameter, explaining that the custom metric group is a logical collection of related metrics that facilitates management and analysis.
  6. In "definitionKeys", provide the ID of the Session Metric definition returned from the API request in Creating a Session Metric definition.
  7. Click Execute.

21.png

If successful, the API response will return a groupId of a new Session Metric Group. Copy the ID to the clipboard as you will need this ID when adding members to a Session Metric Group.

{
"id": "a2f7d6d2-1dd8-4f42-a455-b741535527e1"
}

Adding a Session Metric definition to a Group

To add Session Metric definitions, for example, CPU and Memory to a Group, take the following steps:

  1. In the OpenAPI Specification, scroll up and find UserSessionMetricDefinition.
  2. In UserSessionMetricDefinition, click GET /v7-preview/user-session-metric-definitions.
  3. In the top-right, click Try it out, and then Execute.

22.png

The API response will return a list of existing Session Metrics. Note that as of Login Enterprise 5.5, default Session Metrics are shown as "BuiltIn":

{
"type": "BuiltIn",
"key": "12a1cbf2-cf51-ee11-92e0-000d3a1d9eab",
"metricId": 1,
"sessionMetricsType": "cpuUsage",
"displayName": "CPU",
"unit": "%"
},...

Adding Members to a Group

To add existing Session Metrics (Members) to a Group, take the following steps:

  1. In the OpenAPI Specification, scroll down and find SessionMetricDefinitionGroupMembers.
  2. In SessionMetricDefinitionGroupMembers, click POST /v7-preview-session-metric-definition-groups/{group-id}/members.
  3. In the top-right, click Try it out (The Request body field will open).
  4. In groupId, specify the ID of the Group returned from the API request in Creating a Group for Session Metric definitions.
  5. In Request body, provide the "key" (ID) of each Session Metric you want to add to a Session Metric Group. To add, for example, only the "CPU" Session Metric, specify the "key" returned from the API request in Adding a Session Metric definition to a Group.
  6. Click Execute.

23.png

Session Metric Groups can contain one or more desired performance counters.

Note, however, that each Session Metric in the Group will be polled every 5 seconds. As you add more Metrics to a Group, the CPU overhead will increase correspondingly.

Linking a Group to a Test

Once you’ve created a Group for Session Metric definitions, you need to link the Group to a Test. This way, the Test will capture the Session Metric definitions specified in the Group. To do this:

  1. Navigate to the Login Enterprise Sidebar menu > Configuration > Manage tests.
  2. In Manage tests, select Continuous tests or Load tests from the dropdown. Note that Session Metric Groups are supported only for Continuous tests and Load tests.
  3. Click on the test, then scroll down and find Test settings.
  4. In Session metrics > Metrics group, select the Session Metric Group from the dropdown.

24.png

As of Login Enterprise 5.6, you can control the frequency of Session Metric polling to manage performance overhead effectively. This feature allows you to specify whether metrics are collected from each session or only from 1 out of X sessions. It’s a useful tool you can use to balance the need for metrics gathering, particularly in medium to large-size Session Metric Groups.

Visualizing Session Metrics in charts

In Continuous Testing Results and Load Testing Results, Session Metrics appear as radio buttons. Note that charts only display metrics with the same unit (%, Ops/Sec, Bytes, etc.) simultaneously. This unit is set when you structure a Session Metric definition for an API request. For more details, see Structuring a PerfMon Session Metric definition for an API request or Structuring a WMI Session Metric definition for an API request.

Below is a chart showing performance counters from a Group linked to the Test in the previous step.

25.png

26.png

Additional resources