Search

Load Testing: Results

Introduction

You can view the results of individual Load Tests and compare up to five Load Test instances. The Load Test Results page provides a comprehensive overview of measurements, including charts, graphs, and detailed events for each Test. You can also download both single and multiple Test results as PDF reports for easy sharing and analysis. See the instructions below for more details.

Single Load Test

When a Load Test is completed, you can start looking at the results. To do this, in the Login Enterprise sidebar menu > Results, navigate to Load testing.

Frame 70.png

The overview page will display a list of previously executed Load Test(s). The columns display various information, such as:

  • Time - Displays the date and time of the Test.
  • Name - Displays the name and sequence number of the Test.
  • EUX score - End User Experience score, i.e. a specific set of tools that produce measurements meant to closely approximate how a human would feel about using this machine. In other words, a score from 0 to 10. For more details, see EUX score.
  • VSImax - Maximum number of concurrent active VDI sessions your hardware will support without compromising responsiveness. For more details, see VSImax.
  • Logins - Percentage of sessions that succeeded in logging in on the target environment.
  • Engine - Percentage of Engines that succeeded in running throughout the entire Load Test.
  • Apps - Percentage of Application scripts that executed successfully.
  • Comment - Displays the configured comment for the Test.

Frame 791.png

Results page

Top table

Details the Test summary with the information about the Test described above.

1762f476-f1ac-4989-a204-02813bc97a71.png

To learn how to read the information in the EUX score / VSImax, Logon time, and Application response tables, see the Results page section in this article.

Charts and graphs

You can choose from four data sources for your charts:

  • EUX Metrics: the built-in EUX measurements.
  • Logon Time: login phase times.
  • Application Response Time: timers from your Application scripts.
  • Session Metrics: custom configured session metric data.

Frame 792.png

To learn how to read the information in the Charts table, see the Charts and Graphs section in this document.

For the EUX Metrics, Login Time, and Application Response Time charts, you can switch between two methods of presenting data from the chart menu:

  • One graph per category: Displays each measurement in a separate graph.
  • One chart per measurement: Combines multiple measurements into a single graph.

For Session Metrics, the system only shows one measurement per chart.

2.png

Auto refresh

The page will automatically refresh every minute while a Load Test is running. This feature is available only in the single Load Test view and not in comparison views.

3.png

Stopping a Load Test

You can stop a Load Test directly from the results page, provided the Test is still running. This option is available only for single Load Tests. To stop the Load Test, click Stop Running Test. The screenshot below highlights the button to help you locate it easily.

4.png

Exporting data

You can export data from Load Tests in CSV format. This feature is particularly useful for integrating data from different sources. For more information, see Exporting raw data.

Generating a Load Test PDF report

You can download a PDF report with the results of your Load Test. The Load Test report covers such subject areas as Test Details, Test Results, Login and Application specifics, EUX score, and metrics to characterize system performance. It serves as a valuable resource in understanding the performance of your system under different workloads and implementing necessary improvements.

You can also download a Load Test report via the Public API. For more information, see Downloading a Load Test report via the Public API.

To generate a Load Test report:

1. In the Load Testing results, click “>“ (right arrow) next to the Test you’re interested in.

5.png

2. Click Generate PDF report (A browser pop-up will open).

6.png

3. In the browser pop-up, click Save to save the PDF report.

Alternatively, select the checkbox next to the specific Test, and in the top menu toolbar, click Generate PDF report.

7.png

You can customize a Load Test report depending on your needs. For details and examples, see Configuring a Load Test PDF report.

Multiple Load Tests

If you run multiple Load Tests, you can compare the results once the Tests are completed. Here’s why you might want to do this:

  • Multiple Test comparison: Compare up to five Load Tests simultaneously, allowing for a more detailed and thorough performance analysis.
  • Baseline flexibility: Easily designate and change the Baseline Test for comparison, enabling more meaningful insights by selecting the most relevant Test as the point of reference.
  • Detailed results display: View comprehensive results for each Test, even if direct comparison isn’t possible. This ensures no data is left out and provides a complete overview. The Baseline Test will be the leading Test, serving as the primary reference point for evaluating all subsequent Test results.
  • Expandable charts and graphs: Access a dedicated charts tab with expandable sections for different metrics, making it easier to visualize performance data.
  • Interactive elements: Utilize interactive legends and sticky headers to keep controls accessible and manage chart visibility and metrics effortlessly.
  • Intuitive layout: Enjoy an organized and easy-to-navigate interface, with charts and tables that are clearly structured and sorted by date for quick reference.
  • Sticky header: Benefit from a sticky header that keeps essential controls visible as you scroll.
  • PDF report generation: Generate comprehensive PDF reports for up to five Load Tests. The inclusion of multiple Tests in a single report saves time and effort, and the familiar layout ensures consistency and ease of use.

Overview page

You can select 2 to 5 Load Tests for comparison, with a "Compare" button available when multiple Tests are chosen.

Frame 360.png

Results page

Top table

  • Displays all selected Tests with an option to designate one Test as the Baseline.
  • The selected Tests are compared against the baseline Test.
  • By default, the oldest Test is set as the baseline.
  • Tests are sorted by date, with the oldest at the bottom and the newest at the top.
  • In the comparison tables, columns are sorted by date from left (oldest) to right (newest).
  • Baseline Test results are highlighted in green in the header.
  • If you change the Baseline Test, the highlighting and the percentage differences will update accordingly, but the sorting will remain unchanged.

Frame 361.png

EUX / VSImax table

  • Shows results of all Tests, even if a comparison isn’t possible, e.g. missing data in the Baseline Test.
  • Displays values without percentage differences when comparison isn't possible.

Frame 362.png

Logon times table

The logon times table represents various stages and metrics associated with a user's login process on a Windows-based system. Here’s a breakdown of what each metric measures:

  • Display protocol connection - Measures the time from when the launcher starts the session to when profile loading begins. This metric is found in Windows Event Logs.
  • Profile load - Measured using User Profile events from the Windows Event Viewer.
  • GPO processing - Only measures synchronous group policies. Asynchronous ones have minimal impact on user experience and are therefore not measured.
  • Total login time - Measures how quickly a user account can log in, load the user profile, and become fully interactive.

Frame 363.png

Application response time table

  • The first row always shows the "Apps Performance" value based on the average response times of all baseline applications.
  • The "Apps Performance" value is shown only if the Test includes all applications and measurements present in the baseline.
  • The average response time is calculated solely for the baseline applications and measurements, excluding any additional ones.
  • Tests that do not include all the baseline applications and measurements will not have a value in the "Apps Performance" field.
  • Results per application are displayed below the total.
  • Only applications present in the Baseline Test are shown.
  • If you switch the Baseline Test, the applications shown will be updated accordingly.

Frame 364.png

Hiding, showing, and sorting columns

You can customize your Load Test results table by adding, hiding, or sorting columns. Here's how to manage your columns effectively:

Configuring your columns

1. Configuration icon: In the top right corner of the Load Test results table, you'll find a configuration icon. Click it to open a popup for column management.

Frame 696.png

2. Adding columns: You can add additional columns to your table. The following new columns are available:

  • Connector (Type)
  • Login Window
  • # of Users
  • Test Duration
  • EUX Steady State
  • EUX Baseline

To learn more about the Baseline and Steady State EUX scores, see the Baseline and Steady State EUX scores.

3. Hiding columns: If there are columns you don’t want to see, you can hide them using the configuration options.

4. Sorting columns: To sort columns, simply drag and drop them into your preferred order.

Frame 697.png

Notes and considerations

  • The Time and Test Name columns are fixed and cannot be hidden or moved.
  • Your configuration settings are saved in your local storage. This means your column preferences will persist when you navigate away from the page or log out and back in.
  • The configuration may reset if you use a different browser or clear your browser history.

Including or excluding a Test from the comparison

You can include or exclude a Test from the comparison without needing to go back a step. Here’s how it works:

1. Select up to 5 Tests:

Frame 577.png

2. To exclude a Test, simply unselect it on the Results page. The Test will be removed from the comparison and will not appear in the PDF report if generated.

Please note:

  • You cannot deselect the baseline Test.
  • You cannot set a deselected Test as the baseline.

Charts and graphs

Charts tab

Here, you can expand 4 sections/categories:

  • EUX / VSImax metrics
  • Logins
  • Applications
  • Session Metrics

Frame 365.png

Chart display options

  • You can choose to display charts either one per row or two side-by-side.

Frame 398.png

Frame 399.png

  • A chart control element allows you to navigate and manage chart visibility and metrics.

Frame 400.png

Category interaction

  • Clicking a category in the chart control will expand the section and show related charts.
  • You can select visibility for specific metrics, e.g. active sessions, login window finished.

Frame 401.png

  • Categories without data are displayed but greyed out and unclickable.

Sticky header

  • Ensures the chart control and display options are accessible regardless of your position on the page.

Interactive legend

  • If you click a legend item, it will highlight that item in the chart, with other items being greyed out.

Frame 402.png

  • If you click the item again, it will remove the highlight.

Frame 403.png

VSImax

  • Representation: In the charts, the VSImax is displayed as an orange dot.
  • Tooltip: When hovering over the chart, the orange VSImax dot will be displayed before the session count in the tooltip.
  • Legend: There is only one item for VSImax in the chart legend, and it applies to all Tests.
  • Condition: Be aware that not all Tests will have a VSImax. If the VSImax in the table is denoted as ">5" (greater than 5), it means that during the Test, the performance or system was acceptable throughout. This indicates that the VSImax is greater than the number of users in the Test, and there will not be an orange icon in the charts for such cases.

EUX and Logons

  • These metrics behave the same way in the charts as they do in the tables. The charts are always displayed regardless of whether the Baseline Test has data.

Applications and Session Metrics

  • These metrics depend on the Baseline Test. Charts for these measurements are only shown if the Baseline Test includes the relevant data. For these categories, we will only display the applications and measurements that are present in the Baseline Test. If the other Tests have applications and/or measurements that are not in the Baseline Test, they will not be displayed.
  • Test name display
    • In the chart legend and tooltip, Test names are truncated in the middle if they are too long.
  • Switching between old and new pages
    • An option is available to switch between the old and new page designs. This option is only available when viewing one or two Tests. If there are three or more Tests, this link will not be available.

Frame 408.png

  • Information messages

    • Below the top table, info messages are displayed in the following scenarios:

      • When one or more Tests were run with different Login Enterprise versions.

      • When one or more Tests were run with different EUX versions.

Downloading charts as an image

You can download charts as images for sharing or use in custom reports and presentations. Downloading a chart as an image applies to all charts: Load Test, Continuous Test, and Appliance Health.

To download a chart, follow these steps:

  1. Locate the Download icon: The download icon is located at the top right corner of the chart.
  2. Select image format: When you hover over the icon, three download options will appear: PNG, SVG, and JPG.
  3. Download the image: The selected image will be saved to your Downloads folder, or the directory you've configured as your default Downloads location.

Generating a Load Test PDF report

You can generate and download a PDF report comparing the results of up to 5 Load Tests. You can do this in one of the following ways:

a. Using the toolbar on the Load Test results page:

Frame 409.png

b. Using the Generate PDF report button on the Compare page:

Frame 410.png

Configuring a Load Test PDF report

You can select what you would like your Load Test PDF report to include. So, once you click Generate PDF report, you can configure the following:

  • Introduction text: Lets you choose if you want to include the introduction text containing general information about Login Enterprise.
  • Results: Lets you choose which results you want to include or exclude. For example:
    • EUX/VSImax results
    • Login results
    • Applications results
    • Session Metrics results
  • Charts: Lets you choose if you want to include lines for active sessions and/or login window finished.
    • Show Login window finished in charts
    • Show Active sessions in charts
  • Category: Lets you choose if you want measurements of the same category to be presented in one chart or one chart per measurement/application. This option does not apply to Session Metrics.
    • One graph per category
      • This will combine the lines of the measurements of the same category into 1 graph. Examples of categories are EUX and logins. It works slightly differently for applications, and when this option is chosen, the categorization will be done on the application.
    • One graph per measurement
      • This option will show 1 graph for each timer. For example: EUX has 7 measurements and Logins have 4. When this option is chosen, they will all have their own graph.
    • Note: this option is only available for a report of 1 Load Test. When comparing Load Tests the option is not available and there will always be a graph per measurement.

Below is an example of configurations for a single Load Test PDF report.

Frame 412.png

The configurations for a PDF report comparing multiple Load Tests will look like this:

Frame 411.png

Downloading Load Test PDF report via the Public API

Starting with Login Enterprise 5.14, you can download the Load Test PDF report via the Public API using the following endpoint: /publicApi/v7-preview/reports/load-test-runs/pdf. For more information, see Accessing the Public API. 

Load Test PDF report breakdown

The PDF report summarizes the findings obtained from the Load Testing process. It includes the following:

  • General information, including Introduction, EUX score, and Metrics.
  • Test specifics, such as the product version, EUX version, VSImax version, Connector, Launcher group, number of users, Login window, Test duration, Date, and Workload.
  • Test results: all the results, such as Success rates (table), EUX score (table), and VSImax (table).
  • Charts, including the EUX score, EUX timers, and Logins specific charts with Display protocol connection, Profile load, GPO processing, and Total login time.
  • Application-specific charts
    • Depending on the scripts/workloads attached to the Test, the report will contain a chart for each timer.
  • Session Metrics charts
    • Depending on the Session Metrics linked to the Test, each Session Metric will have its own chart.

For more details, see the attached Load Test report samples under Additional resources.

Important information

Active sessions and login window finished:

  • In a single Load Test PDF report, all charts will include lines for active sessions and the login window finished line.
  • In a PDF report comparing two Load Tests, all charts will include lines for active sessions, but none will include the login window finished line.

Comparing logic:

  • The report will determine which Test is compared to which, mirroring the UI. To switch the Tests in the report, switch them in the UI first, then generate the PDF report.
  • The Test considered as the baseline will be labeled "Baseline" in tables.

EUX & VSImax-related messages

The EUX and VSImax information, warning, and error messages are shown in the comment column within their respective tables.

Additional resources

If you have questions or need additional information on specific Load Test configurations, feel free to get in touch with our support at support@loginvsi.com.