Manage Tests: Continuous Testing

Login Enterprise provides three test types, one of which is referred to as a Continuous Test. This article will provide an overview on creating, configuring, and enabling Continuous Tests.

 

Creating a Continuous Test

Navigate to the Manage Continuous Test page, where all Continuous Test configurations are displayed. 

To create a new Continuous Test, select the green "+" icon in the top-right section of the table. The following screen will appear, where the Continuous Test details can be specified.

The following information is required to create a Continuous Test:

• Name: the display name of the Continuous Test
• Description: (optional) a friendly description of the Continuous Test
• Environment: (optional) the Login Enterprise Environment that the Continuous Test should be associated with. For more information, see this article.
•  Connector: the type of Connector that should be used by the Launcher. For more information, see this article.

Depending on the chosen Connector, there will be additional connection parameters to specify. You will need to specify the Account Group, and the Launcher Group, that will be used for the test. For more information on Account Groups, see this article. For more information on Launcher Groups, see this article.

Configuring Test Settings

Once the basic test information has been entered and saved, further test configuration is possible. The first menu to configure is the Test Settings.

The menu provides the ability to enable or disable particular data gathering, troubleshooting, and scheduling information. 

• EUX Score: Enable or disable the EUX Score calculations during test runs
• Custom Screenshots: Enable or disable custom screenshots within Application Scripts. For more information see the TakeScreenshot section in this article. 
• Verbose Logging: Display the line number of the command that raised an error during Application Execution failures. Note, this will impact test results as the line number is returned during execution.

Configuring the Test Schedule

Continuous Tests, unlike other available test types, can be scheduled. There are several scheduling options:

• Login Every: Launch a session every X minute(s), using Launchers based on a round-robin pattern
• Run a session from each Launcher every: Launch a session every X minute(s), from all Launchers in the specified Launcher Group
• Run concurrent session(s): Launch X concurrent sessions, using Launchers based on a round-robin pattern

The difference between these options is that with the "Log in every X minutes" a virtual user will log in to the environment, run the test, and log of. It will then wait for the configured amount of time and do the same actions again. With the "run concurrent session(s)" option, the users will keep logging in, do the actions that are defined and log out. However, they will not wait for a set amount of time and will re-start immediately after logging out. When choosing for "Run a session from each launcher every X minutes" it will launch one session from every launcher configured for the environment within the specified time frame.

Daily and Weekly Scheduling

Login Enterprise Administrators can apply granular scheduling to their Continuous Tests, based on either a daily or weekly schedule. The two examples below setup a 6am-5pm Continuous Test schedule on a daily, and weekly basis. For example, this provides the ability to exclude schedule downtime from the testing cadence.

Drag your mouse along the desired time-frames within the scheduling interface. The bars are displayed in an hourly view, where each individual block is 15-minutes. Be sure to "SAVE" to confirm and apply your scheduling preferences.

Session Metrics

As of Login Enterprise 5.4, Session Metrics can now be captured from Windows Performance Monitor and WMI as part of Continuous and Load Tests. Use the toggle to enable or disable gathering Session Metrics, and if enabled, choose the desired Session Metrics Group from the drop-down menu to specify which performance counters should be collected. For more information on Session Metrics, see this article.

 

Actions

In the Actions table, Login Enterprise administrators can define the Virtual User workload. The "Add Action(s)" button can be used to select an individual application script to include, or an Application Group to add all of its member applications.

Once selected, choose whether you would like to add an individual Application, or an Application Group. If Applications, a list of all applications that are available in the Login Enterprise appliance is displayed, whose checkboxes can be used to add Applications to the test configuration. To learn more about applications and how to create them see this area of our Knowledge Base.

Alternatively, rather than adding an individual Application workflow, an Application Group can be selected. For more information on Application Groups, see this article.

Once the workload has been defined, the Virtual User optionally be configured to repeat their defined workload before logging off. The number of times the workload should be repeated is set by the "Repeat all steps above" setting. To enable workload repeat, first toggle the switch to enable repeating, and then specify how many iterations should be executed before logging off.  

As of Login Enterprise 4.6, Applications can be set to "leave applications running" or "run once". The "Leave application running" switch will prevent the application from being closed once the workflow script is completed. This requires additional steps to close the application later. For more information about this feature please see this page. This feature is not applicable to Browser-based Application workflows. 

If using the "Repeat all steps above" feature, you can enable the "Run Once" switch to prevent the same application from being run each iteration, for example if there is an Application workflow being used as a prepatory step, like Preparing Office.

Notifications and Thresholds

 

One of the key benefits of Continuous Testing is that Virtual Users are not reliant on any active real user sessions. This means they can heedlessly execute workloads, identify failure or degradation, and alert IT Operations teams in a truly proactive fashion.

In the Notifications tables, you can specify threshold times that when broken, trigger alerting mechanisms. Once enabled with the toggle, Login Times and Latency can be reported on. Additionally, any application failures are alerted by default.

Using the “Add new notification” button, Login Enterprise administrators can measure Application Launch and Response Times against thresholds as well. Once application(s) or Application Groups have been specified as part of the test, the Application Launch time, and any custom timers that were defined, will appear in the drop-down menu.

There are 4 session notifications that you can always enable. 

• Login Failure
• Login
• Latency
• Application Failure

 For more detailed information about the standard notifications and email configurations click here.

Executing Continuous Tests

Now that you've configured your Continuous Test, it is time to start it. All of the existing Continuous Tests are displayed within the Manage Continuous Tests page. From here you can drill-down into the configuration of an existing Test, or perform start/stop operations on existing Tests.

To start a Continuous Test, for example the "Availability Check" below, simply click the toggle switch to enable it. This will activate the test, which will launch in accordance with the schedule that has been set for it. 

Further, as of Login Enterprise 5.6, Continuous Tests can be enabled and disabled in bulk, to provide increased ease of management for larger Login Enterprise deployments. These operations are facilitated by the new banner on the Continuous Tests page. To perform bulk start/stop operations, first select the checkboxes of all Tests that should be considered during the bulk toggle. 

Based on the desired end state, the "ENABLE ALL" or "DISABLE ALL" buttons can be used.