Overview
If you run multiple Load Tests, you can compare the results once the Tests are completed. Here’s why you might want to do this:
- Multiple Test comparison: Compare up to five Load Tests simultaneously, allowing for a more detailed and thorough performance analysis.
- Baseline flexibility: Easily designate and change the Baseline Test for comparison, enabling more meaningful insights by selecting the most relevant Test as the point of reference.
- Detailed results display: View comprehensive results for each Test, even if direct comparison isn’t possible. This ensures no data is left out and provides a complete overview. The Baseline Test will be the leading Test, serving as the primary reference point for evaluating all subsequent Test results.
- Expandable charts and graphs: Access a dedicated charts tab with expandable sections for different metrics, making it easier to visualize performance data.
- Interactive elements: Utilize interactive legends and sticky headers to keep controls accessible and manage chart visibility and metrics effortlessly.
- Intuitive layout: Enjoy an organized and easy-to-navigate interface, with charts and tables that are clearly structured and sorted by date for quick reference.
- Sticky header: Benefit from a sticky header that keeps essential controls visible as you scroll.
- PDF report generation: Generate comprehensive PDF reports for up to five Load Tests. The inclusion of multiple Tests in a single report saves time and effort, and the familiar layout ensures consistency and ease of use.
Comparing Load Tests
Overview page
You can select 2 to 5 Load Tests for comparison, with a "Compare" button available when multiple Tests are chosen.
Results page
Top table
- Displays all selected Tests with an option to designate one Test as the Baseline.
- The selected Tests are compared against the baseline Test.
- By default, the oldest Test is set as the baseline.
- Tests are sorted by date, with the oldest at the bottom and the newest at the top.
- In the comparison tables, columns are sorted by date from left (oldest) to right (newest).
- Baseline Test results are highlighted in green in the header.
- If you change the Baseline Test, the highlighting and the percentage differences will update accordingly, but the sorting will remain unchanged.
EUX / VSImax table
- Shows results of all Tests, even if a comparison isn’t possible, e.g. missing data in the Baseline Test.
- Displays values without percentage differences when comparison isn't possible.
Logon times table
The logon times table represents various stages and metrics associated with a user's login process on a Windows-based system. Here’s a breakdown of what each metric measures:
- Display protocol connection - Measures the time from when the launcher starts the session to when profile loading begins. This metric is found in Windows Event Logs.
- Profile load - Measured using User Profile events from the Windows Event Viewer.
- GPO processing - Only measures synchronous group policies. Asynchronous ones have minimal impact on user experience and are therefore not measured.
- Total login time - Measures how quickly a user account can log in, load the user profile, and become fully interactive.
Application response time table
- The first row always shows the "Apps Performance" value based on the average response times of all baseline applications.
- The "Apps Performance" value is shown only if the Test includes all applications and measurements present in the baseline.
- The average response time is calculated solely for the baseline applications and measurements, excluding any additional ones.
- Tests that do not include all the baseline applications and measurements will not have a value in the "Apps Performance" field.
- Results per application are displayed below the total.
- Only applications present in the Baseline Test are shown.
- If you switch the Baseline Test, the applications shown will be updated accordingly.
Hiding, showing, and sorting columns
You can customize your Load Test results table by adding, hiding, or sorting columns. Here's how to manage your columns effectively:
Configuring your columns
1. Configuration icon: In the top right corner of the Load Test results table, you'll find a configuration icon. Click it to open a popup for column management.
2. Adding columns: You can add additional columns to your table. The following new columns are available:
- Connector (Type)
- Login Window
- # of Users
- Test Duration
- EUX Steady State
- EUX Baseline
To learn more about the Baseline and Steady State EUX scores, see the Baseline and Steady State EUX scores.
3. Hiding columns: If there are columns you don’t want to see, you can hide them using the configuration options.
4. Sorting columns: To sort columns, simply drag and drop them into your preferred order.
Notes and considerations
- The Time and Test Name columns are fixed and cannot be hidden or moved.
- Your configuration settings are saved in your local storage. This means your column preferences will persist when you navigate away from the page or log out and back in.
- The configuration may reset if you use a different browser or clear your browser history.
Including or excluding a Test from the comparison
You can include or exclude a Test from the comparison without needing to go back a step. Here’s how it works:
1. Select up to 5 Tests:
2. To exclude a Test, simply unselect it on the Results page. The Test will be removed from the comparison and will not appear in the PDF report if generated.
Please note:
- You cannot deselect the baseline Test.
- You cannot set a deselected Test as the baseline.
Charts and graphs
Charts tab
Here, you can expand 4 sections/categories:
- EUX / VSImax metrics
- Logins
- Applications
- Session Metrics
- To learn about the EUX score and VSImax, see the Login Enterprise EUX Score and VSImax.
- For the Login performance, see Login performance.
- To customize action scripts for applications and advanced system utilities, see Applications.
- For the information on Session Metrics, see the Session Metrics.
Chart display options
- You can choose to display charts either one per row or two side-by-side.
- A chart control element allows you to navigate and manage chart visibility and metrics.
Category interaction
- Clicking a category in the chart control will expand the section and show related charts.
- You can select visibility for specific metrics, e.g. active sessions, login window finished.
- Categories without data are displayed but greyed out and unclickable.
Sticky header
- Ensures the chart control and display options are accessible regardless of your position on the page.
Interactive legend
- If you click a legend item, it will highlight that item in the chart, with other items being greyed out.
- If you click the item again, it will remove the highlight.
Downloading charts as an image
You can download charts as images for sharing or use in custom reports and presentations. Downloading a chart as an image applies to all charts: Load Test, Continuous Test, and Appliance Health.
To download a chart, follow these steps:
- Locate the Download icon: The download icon is located at the top right corner of the chart.
- Select image format: When you hover over the icon, three download options will appear: PNG, SVG, and JPG.
- Download the image: The selected image will be saved to your Downloads folder, or the directory you've configured as your default Downloads location.
Updates and new features in charts and graphs
Multiple Load Tests
VSImax
- Representation: In the charts, the VSImax is displayed as an orange dot.
- Tooltip: When hovering over the chart, the orange VSImax dot will be displayed before the session count in the tooltip.
- Legend: There is only one item for VSImax in the chart legend, and it applies to all Tests.
- Condition: Be aware that not all Tests will have a VSImax. If the VSImax in the table is denoted as ">5" (greater than 5), it means that during the Test, the performance or system was acceptable throughout. This indicates that the VSImax is greater than the number of users in the Test, and there will not be an orange icon in the charts for such cases.
EUX and Logons
- These metrics behave the same way in the charts as they do in the tables. The charts are always displayed regardless of whether the Baseline Test has data.
Applications and Session Metrics
- These metrics depend on the Baseline Test. Charts for these measurements are only shown if the Baseline Test includes the relevant data. For these categories, we will only display the applications and measurements that are present in the Baseline Test. If the other Tests have applications and/or measurements that are not in the Baseline Test, they will not be displayed.
Additional changes and notes
-
Test name display
- In the chart legend and tooltip, Test names are truncated in the middle if they are too long.
-
Switching between old and new pages
- An option is available to switch between the old and new page designs. This option is only available when viewing one or two Tests. If there are three or more Tests, this link will not be available.
-
Information messages
-
Below the top table, info messages are displayed in the following scenarios:
-
When one or more Tests were run with different Login Enterprise versions.
-
When one or more Tests were run with different EUX versions.
-
-
Single Load Tests
1. Single Load Test charts
You can switch between two methods of presenting data from the chart menu:
- One graph per category: Displays each measurement in a separate graph.
- One chart per measurement: Combines multiple measurements into a single graph.
The available categories and their options are:
a. EUX metrics
-
- One graph per category: Displays two charts—one for the overall EUX score and one for the six EUX timers.
- One chart per measurement: Displays seven individual charts, one for each EUX metric.
b. Logon time
-
- One graph per category: Combines all four logon time measurements into a single chart.
- One chart per measurement: Displays four separate charts, one for each logon time measurement.
c. Application response times
-
- One graph per category: Displays measurements per application, so you may see multiple lines within a single graph. The number of lines and graphs will vary based on the number of applications (scripts) you set up and the number of measurements within each application script.
- One chart per measurement: Available for applications. When selected, each measurement will have its own chart, with no more than one line per chart. The number of charts will depend on the number of measurements within each application script.
Session Metrics are not available in the chart options for combined views and are always shown with one measurement per chart.
2. Auto refresh
The page will automatically refresh every minute while a Load Test is running. This feature is available only in the single Load Test view and not in comparison views.
3. Stopping a Load Test
You can stop a Load Test directly from the results page, provided the Test is still running. This option is available only for single Load Tests. To stop the Load Test, click Stop Running Test. The screenshot below highlights the button to help you locate it easily.
Generating a Load Test Report
Single Load Test
You can download a PDF report with the results of your Load Test. The Load Test Report covers such subject areas as Test Details, Test Results, Login and Application specifics, EUX score, and metrics to characterize system performance. It serves as a valuable resource in understanding the performance of your system under different workloads and implementing necessary improvements.
To generate a Load Test Report:
1. In the Sidebar menu, navigate to Results > Load Testing.
2. In the Load Testing results, click “>“ (right arrow) next to the Test you’re interested in.
3. Click Generate PDF report (A browser pop-up will open).
4. In the browser pop-up, click Save to save the PDF report.
Alternatively, select the checkbox next to the specific Test, and in the top menu toolbar, click Generate PDF report.
Multiple (up to 5) Load Tests
You can generate and download a PDF report comparing the results of up to 5 Load Tests. You can do this in one of the following ways:
a. Using the toolbar on the Load Test results page:
b. Using the Generate PDF report button on the Compare page:
Configuring a Load Test PDF report
You can select what you would like your Load Test PDF report to include. So, once you click Generate PDF report, you can configure the following:
- Introduction text: Lets you choose if you want to include the introduction text containing general information about Login Enterprise.
-
Results: Lets you choose which results you want to include or exclude. For example:
- EUX/VSImax results
- Login results
- Applications results
- Session Metrics results
-
Charts: Lets you choose if you want to include lines for active sessions and/or login window finished.
- Show Login window finished in charts
- Show Active sessions in charts
-
Category: Lets you choose if you want measurements of the same category to be presented in one chart or one chart per measurement/application. This option does not apply to Session Metrics.
- One graph per category
- This will combine the lines of the measurements of the same category into 1 graph. Examples of categories are EUX and logins. It works slightly differently for applications, and when this option is chosen, the categorization will be done on the application.
- One graph per measurement
- This option will show 1 graph for each timer. For example: EUX has 7 measurements and Logins have 4. When this option is chosen, they will all have their own graph.
- Note: this option is only available for a report of 1 Load Test. When comparing Load Tests the option is not available and there will always be a graph per measurement.
- One graph per category
Below is an example of configurations for a single Load Test PDF report.
The configurations for a PDF report comparing multiple Load Tests will look like this:
Downloading Load Test PDF report via the Public API
Starting with Login Enterprise 5.14, you can download the Load Test PDF report via the Public API using the following endpoint: /publicApi/v7-preview/reports/load-test-runs/pdf. For more information, see Accessing the Public API.
Load Test PDF report breakdown
The PDF report summarizes the findings obtained from the Load Testing process. It includes the following:
- General information, including Introduction, EUX score, and Metrics.
- Test specifics, such as the product version, EUX version, VSImax version, Connector, Launcher group, number of users, Login window, Test duration, Date, and Workload.
- Test results: all the results, such as Success rates (table), EUX score (table), and VSImax (table).
- Charts, including the EUX score, EUX timers, and Logins specific charts with Display protocol connection, Profile load, GPO processing, and Total login time.
-
Application-specific charts
- Depending on the scripts/workloads attached to the Test, the report will contain a chart for each timer.
-
Session Metrics charts
- Depending on the Session Metrics linked to the Test, each Session Metric will have its own chart.
For more details, see the attached Load Test report samples under Additional resources.
Important information
Active sessions and login window finished:
- In a single Load Test PDF report, all charts will include lines for active sessions and the login window finished line.
- In a PDF report comparing two Load Tests, all charts will include lines for active sessions, but none will include the login window finished line.
Comparing logic:
- The report will determine which Test is compared to which, mirroring the UI. To switch the Tests in the report, switch them in the UI first, then generate the PDF report.
- The Test considered as the baseline will be labeled "Baseline" in tables.
EUX & VSImax-related messages
The EUX and VSImax information, warning, and error messages are shown in the comment column within their respective tables.
Additional resources
- For the accounts and account groups, see Accounts.
- For the Launchers overview and best practices, see the Launchers: Overview and best practices.
- To learn about the Login Enterprise Windows Launcher setup, configuration, maintenance, and more, see the Login Enterprise Windows Launcher.
If you have questions or need additional information on specific Load Test configurations, feel free to get in touch with our support at support@loginvsi.com.