Overview
Application Tests verify that your user software functions correctly and remains within established operational parameters. Similar to Load Tests, these are one-time, on-demand Tests that return data and events specific to the current Test. They are not scheduled to run periodically.
How it works, capabilities, methodology
An Application Test launches a single session. It only requires one session because it does not gather long-term tracking data, unlike Continuous Tests, and it does not measure aggregate performance, unlike Load Tests. Its sole purpose is to select a single virtual desktop infrastructure (VDI) and run a set of application scripts to verify that your software is functional. EUX testing cannot be enabled for these Tests.
Application scripts for Application Tests focus on functional areas rather than metrics. While a Continuous Test might benefit from timing every operation with measurable durations, an Application Test aims to verify that each part of the program functions correctly. The scripts should check various aspects of the program, aborting if any errors occur. If timers are used, they are intended to identify significant issues, such as tasks that never complete, rather than small changes in speed.
Unlike Load Tests, Application Tests can trigger alert e-mails for threshold violations.
Handling script failures and achieving granularity
Since script failure is an all-or-none option, using a single Test to assess multiple functions can be challenging. If any function fails, the entire script fails. However, if you need more granular results and want to continue checking for issues even after the first failure, you can use events and timers. One option is to use CreateEvent in your script to log an event in the Login Enterprise events for the Test run, indicating that a subtask failed, and then proceed to the next subtask. Another approach is to use timers to reflect your functional results. Instead of using StartTimer and StopTimer to measure an interval, use SetTimer to report a single value. Define success as a specific number, e.g. 1000, corresponding to 1 second, and failure as a higher number, like 2000. (Ensure the failure value is higher than the success value because alerts are triggered for numbers exceeding the threshold.) Then, set an alert for any timer that exceeds 1500. While you could use SetTimer to report 1 or 2 and alert on 2, it is generally better to use more noticeable values in your code.
Note that EUX and VSImax, designed as aggregate statistical measures, are unavailable in Application Tests. However, as Application Tests utilize only one session, the Desktop Connector remains accessible. For more information, see the Desktop connection.
Creating an Application Test
1. In the Sidebar menu, navigate to Configuration > Manage Tests, and select Application Tests.
2. In Application Tests, click “+“ on the top-right.
3. In the Info window, provide the following:
- Name: Specify the display name for the Application Test.
- Description: Optionally, provide an informative description of the Application Test.
- Connector: Choose the type of Connector to be utilized by the Launcher.
Each Connector has its own settings. Once you define your Connector, specify the necessary information in the Settings section. Here’s an example of the Citrix StoreFront Connector configurations.
4. Click Save to apply the changes.
- For more information about connectors and connection configurations, see Connectors and Connection Configurations.
- For more information on the accounts, see Accounts.
- For the Launchers overview and best practices, see the Launchers: Overview and best practices.
- To learn about the Login Enterprise Windows Launcher setup, configuration, maintenance, and more, see the Login Enterprise Windows Launcher.
Configuring an Application Test
Test settings
Once you’ve created an Application Test, you need to configure the Test settings, such as:
- Verbose logging - Display the command line number that triggered an error during application execution failures. This may affect Test results as the line number is returned during execution.
- Logon time tracking process - Track the complete duration from when the Connector initiates the login process for the user session until the final user-session executable (usually Windows Explorer) is up and running. You can also overwrite the default process of time tracking with a process of your choice. For more information, see Measuring total login time.
Report settings
You can choose to be notified when applications pass or fail. If you do not enable notifications, the report will only indicate failures, which is usually the most relevant information.
You can also configure the system to send an email once the Application Test is complete. If the email notification switch is enabled but no email address is specified, the default email address from the email configuration will be used. You can enter multiple email addresses in the Email Notification field, separated by commas. For more information on setting up email notifications, see Email notifications.
Additionally, you can personalize the report by adding your company logo.
The generated PDF file supports a maximum of 100 screenshots. This means if you include 100 applications and enable "Include successful applications," you will reach this limit. Additionally, any use of the "TakeScreenshot" functionality within application scripts will count towards the 100 screenshot limit. To manage this, if you have a large number of applications, we recommend keeping "Include successful applications" turned off.
Adding Applications
You can add the preferred applications for the Virtual User to test. You can also add Application groups and an option “to wait”. To do this:
In Actions, click the green “+“ to add a new action, and select what you’re going to add in the pop-up window.
- To learn about applications and how to create and add them, see Applications and Workload Customizations.
- For more information on application configurations, see Application Configuration.
- For the Application groups, see Application groups.
After you've defined your workload, you can choose to have the Virtual User repeat the defined tasks before logging off. You can specify how many times the workload should be repeated using the 'Repeat all steps above' setting. To enable workload repetition, simply toggle the switch to turn it on, and then specify the number of iterations you'd like to execute before logging off.
You also have the option to set applications to Leave application running. When you toggle the 'Leave application running' switch, the application will remain open even after the workflow script is completed. Keep in mind that closing the application later will require additional steps. This feature does not apply to web scripts.
For more information, see Configuring Applications: Run once and Leave running settings.
Defining thresholds
You can also define thresholds for all actions, similar to the Continuous Testing setup. These thresholds ensure that the application not only functions correctly but also maintains its performance across different configurations. If performance thresholds are not met, you will be notified in the Application Testing report.
Starting the Application Test
1. In the Sidebar menu, navigate to Configuration > Manage Tests, and select Application Tests.
2. In the Application Tests, click Start next to the Test you’d like to start.
Note that even if you configured a user group that contains multiple users, the Application Test scenario only uses one user.
After you click Start, a confirmation dialog window will appear, displaying the current application configuration. If the configuration is correct, click Confirm to begin the Test.
Once you start a Test, you will be redirected to the Login Enterprise Dashboard. Here, you can view the current status of your recently started Application Test. The dashboard displays how many applications have started and if any have failed. Performance measurements are also shown.
- For the Application testing results, in the Login Enterprise Sidebar menu, navigate to Results > Application Testing.
- For information on how to view and analyze the Application testing results, see Application Testing: Results.
If you have questions or need additional information on specific Application Test configurations, feel free to get in touch with our support at support@loginvsi.com.