Search

Manage Tests: Application Testing

Overview

Application Tests verify that your user software functions correctly and remains within established operational parameters. Similar to Load Tests, these are one-time, on-demand Tests that return data and events specific to the current Test. They are not scheduled to run periodically.

How it works, capabilities, methodology

An Application Test launches a single session. It only requires one session because it does not gather long-term tracking data, unlike Continuous Tests, and it does not measure aggregate performance, unlike Load Tests. Its sole purpose is to select a single virtual desktop infrastructure (VDI) and run a set of application scripts to verify that your software is functional. EUX testing cannot be enabled for these Tests.

Application scripts for Application Tests focus on functional areas rather than metrics. While a Continuous Test might benefit from timing every operation with measurable durations, an Application Test aims to verify that each part of the program functions correctly. The scripts should check various aspects of the program, aborting if any errors occur. If timers are used, they are intended to identify significant issues, such as tasks that never complete, rather than small changes in speed.

Unlike Load Tests, Application Tests can trigger alert e-mails for threshold violations.

Handling script failures and achieving granularity

Since script failure is an all-or-none option, using a single Test to assess multiple functions can be challenging. If any function fails, the entire script fails. However, if you need more granular results and want to continue checking for issues even after the first failure, you can use events and timers. One option is to use CreateEvent in your script to log an event in the Login Enterprise events for the Test run, indicating that a subtask failed, and then proceed to the next subtask. Another approach is to use timers to reflect your functional results. Instead of using StartTimer and StopTimer to measure an interval, use SetTimer to report a single value. Define success as a specific number, e.g. 1000, corresponding to 1 second, and failure as a higher number, like 2000. (Ensure the failure value is higher than the success value because alerts are triggered for numbers exceeding the threshold.) Then, set an alert for any timer that exceeds 1500. While you could use SetTimer to report 1 or 2 and alert on 2, it is generally better to use more noticeable values in your code.

Note that EUX and VSImax, designed as aggregate statistical measures, are unavailable in Application Tests. However, as Application Tests utilize only one session, the Desktop Connector remains accessible. For more information, see the Desktop connection.

Creating an Application Test

1. In the Sidebar menu, navigate to Configuration > Manage Tests, and select Application Tests.

Frame 264.png

2. In Application Tests, click “+“ on the top-right.

Frame 266.png

3. In the Info window, provide the following:

  • Name: Specify the display name for the Application Test.
  • Description: Optionally, provide an informative description of the Application Test.
  • Connector: Choose the type of Connector to be utilized by the Launcher.

Frame 265.png

Each Connector has its own settings. Once you define your Connector, specify the necessary information in the Settings section. Here’s an example of the Citrix StoreFront Connector configurations.

Frame 123.png

4. Click Save to apply the changes.

Configuring Application Test settings

Once you’ve created an Application Test, you need to configure the Test settings, such as:

  • Verbose logging - Display the command line number that triggered an error during Application execution failures. This may affect Test results as the line number is returned during execution.
  • Logon time tracking process - Track the complete duration from when the Connector initiates the login process for the user session until the final user-session executable (usually Windows Explorer) is up and running. You can also overwrite the default process of time tracking with a process of your choice. For more information, see Measuring total login time.

Frame 965.png

Report settings

  • Include successful applications: You can choose to capture a screenshot of successful Applications when the script completes without errors. Disabling this option skips the screenshot, but the results of the successful Applications (such as measurements) will still be included in the report.
  • Email notification: You can choose to be notified when Applications pass or fail. If you do not enable notifications, the report will only indicate failures, which is usually the most relevant information.

You can also configure the system to send an email once the Application Test is complete. If the email notification switch is enabled but no email address is specified, the default email address from the email configuration will be used. You can enter multiple email addresses in the Email Notification field, separated by commas. For more information on setting up email notifications, see Email notifications.

Additionally, you can personalize the report by adding your company logo.

Frame 966.png

There is no strict limit on the number of screenshots in a generated PDF report. However, note that including a large number of screenshots (for example, up to 1,000 screenshots) may cause browser downtime and increase the PDF file size.

For optimal performance and compatibility, we recommend keeping the number of screenshots below 250. At around 250 screenshots, the PDF size is typically under 20 MB, which aligns with email systems that limit attachments to 20 MB.

Adding Applications to an Application Test

You can add the preferred Applications for the Virtual User to test. You can also add Application groups and an option “to wait”. To do this:

In Actions, click the green “+“ to add a new action, and select what you’re going to add in the pop-up window.

Frame 967.png

After you've defined your workload, you can choose to have the Virtual User repeat the defined tasks before logging off. You can specify how many times the workload should be repeated using the 'Repeat all steps above' setting. To enable workload repetition, simply toggle the switch to turn it on, and then specify the number of iterations you'd like to execute before logging off.

You also have the option to set Applications to Leave Application running. When you toggle the 'Leave Application running' switch, the Application will remain open even after the workflow script is completed. Keep in mind that closing the Application later will require additional steps. This feature does not apply to web scripts. 

Frame 968.png

Defining thresholds

You can also define thresholds for all actions, similar to the Continuous Testing setup. These thresholds ensure that the Application not only functions correctly but also maintains its performance across different configurations. If performance thresholds are not met, you will be notified in the Application Testing report.

Frame 969.png

Access control settings

You can assign roles to Tests to control access to the Test configuration and results. By assigning roles, only users with the designated roles will be able to view or modify the Test configuration and results, depending on their permissions.

Frame 958.png

For detailed instructions on creating and managing roles, permissions, and dependencies, see the Role-Based Access Control.

Adjusting Engine start timeout

The setting for configuring the timeout for engine start is not available through the Login Enterprise web interface. To adjust this setting, you need to use the API.

You can do so by accessing the /publicApi/v7-preview/tests/{testId} endpoint and specifying the engineStartTimeout parameter.

The timeout for the Engine start setting controls how long the Virtual Appliance will wait for the Engine on the Target to come online after a session is launched, and before the connection is considered a failure (login failure).

Key considerations:

  • Default timeout: If no value is specified, the timeout defaults to 300 seconds (5 minutes).
  • Web interface limitation: This option is not available through the Appliance web interface. If you create a Test via the web interface, the timeout will be left empty and default to 300 seconds.
  • Update via API: To modify the timeout, first create a Test, then use the update endpoint to set the timeout. The timeout cannot be set during Test creation.

How to set the timeout:

  1. Create a Test (timeout will be empty by default).
  2. Use the Update Test endpoint to set the desired timeout (in seconds).

For details on accessing the Public API in Login Enterprise, see Accessing the Public API.

Starting the Application Test

1. In the Sidebar menu, navigate to Configuration > Manage Tests, and select Application Tests.
2. In the Application Tests, click Start next to the Test you’d like to start.

Frame 272.png

Note that even if you configured a user group that contains multiple users, the Application Test scenario only uses one user. 

After you click Start, a confirmation dialog window will appear, displaying the current Application configuration. If the configuration is correct, click Confirm to begin the Test.

Frame 273.png

Once you start a Test, you will be redirected to the Login Enterprise Dashboard. Here, you can view the current status of your recently started Application Test. The dashboard displays how many Applications have started and if any have failed. Performance measurements are also shown.

Frame 274.png

Application Test results

For information on the Application Test results, see the Application Testing: Results.

If you have questions or need additional information on specific Application Test configurations, feel free to get in touch with our support at support@loginvsi.com.