EUX Score 

The EUX Score (End User Experience Score) represents the performance of any windows machine (virtual, physical, cloud or on premises) and ranks it between 0 and 10 as experienced by a user. The EUX score can be determined with any number of users (1 and up). In the case of VDI systems with shared hardware it will decrease as more users are added to the platform as performance will go down as systems get more crowded. 

The EUX score values were determined by tests we performed using human user feedback. We let real users work on different virtual machines. Each individual was asked to perform a fixed set of operations and give a score between 0 to 10, based on their experience. At the same time, our virtual user was gathering performance metrics. By varying the performance characteristics in between sessions, we were able to correlate our EUX score with a score that each human user assigned to their session. We tuned the EUX algorithm to yield a score of 5.5 at the point where our human test subjects indicated their experience was starting to degrade significantly. 

Determining the EUX score 

The EUX score is built up from several actions that reflect typical user operations that we found correlate well with a human users experience while working on the system. These actions allow us to measure the following types of operations: 

• Launch a complete application with UI until they are ready for user input 
• Keyboard input processing 
• CPU heavy actions 
• Actions that mix CPU and Disk IO 

We found that using the average response times of our performance metrics did not accurately reflect how our human users reported their experience. Their experience was better reflected by using a subset of the results, more specifically the subset of slower response times. This makes sense when you realize that when your system is unresponsive for part of the time, the experience is already degrading. 

In addition to these performance metrics, we keep track of application- and session failures and use those as input for the EUX score. Because as more errors occur, the user experience deteriorates.  

VSIMax 

The VSIMax defines the EUX tipping point of your system. If you go beyond the VSIMax your EUX will start to degrade significantly.  

As explained above, the EUX performance input metrics were chosen to reflect the user experience. In addition to this we carefully chose and tuned the timers to also reflect the load that a virtualization system experiences when increasing the number of sessions that run on that system. 

VSIMax is triggered when the user experience dips below a certain value. We determined two criteria for this threshold: 

• An absolute value, which indicates at which point the experience starts to degrade 
• A relative value, based on the performance of the system when it is running optimally 

The reason for the absolute value is obvious. The second criterion needs more explanation.  

We ran capacity planning experiments on several systems with different configurations. A recurring pattern was found in nearly all test results. While adding more sessions, at a certain point the performance starts to degrade very rapidly. Ideally you want VSIMax to be triggered before the point of that collapse, or just when it starts to happen. To determine this point, the second criterion proved to be more reliable. When correlating our results with the hypervisor’s internal metrics, we found that at around 85% of the initial EUX score, we are at the point where we are generating too much load. Even though the exact percentage did vary in between systems, we found that 85% was a good general number to base VSIMax on. 

Calculation of VSIMax 

The VSIMax score is calculated in two steps. The first step determines the threshold from which to compare the EUX to as it changes over time. The second step determines the number of sessions that were running when we crossed that threshold. 

Step 1: Determining the Threshold 

The baseline represents the EUX score when the system under test is running without any load. The closest we get to that state, the better. To that end, we use the EUX score of the median response times. 

To find the baseline, we scan the complete test run to determine the five consecutive minutes with the highest average. That average is the score we use as the baseline. We record where that took place as the starting point for the next step. 

Step 2: Determining the VSIMax 

In this step we are looking at which point the EUX score falls below the threshold found in the previous step for three consecutive minutes. The threshold will be set to 85% of the average that was found in the previous step or 5.5, whichever is highest. 

We use the EUX score as our input. We return the session count that was found at the start of the stretch of three minutes where we dipped below the threshold. 

If we encounter any missing data after we find our baseline, the result cannot be interpreted, and we will not get a VSIMax score.