Search

SilverBench with Platform Metrics

Overview

This workload automates the execution of SilverBench benchmarks and uploads the results as Platform Metrics to Login Enterprise. It provides an objective and standardized benchmarking process that is quick to configure, yielding valuable insights for baselining, performance comparisons, and configuration assessments. The script currently works with Edge and Chrome. See Configurable Variables for command lines for each application.

Key Features

  • Self-contained Execution: Runs specified SilverBench benchmarks and handles results automatically.
  • Data Integration: Uploads benchmark results as Platform Metrics to Login Enterprise.
  • Quick Configuration: Requires minimal setup by updating variables at the top of the script.
  • Objective Analysis: Facilitates easy comparison across different configurations.

Note on Terminology:

In this document, the terms script, workload, and Login Enterprise Application are used interchangeably, all referring to the SilverBench_Template.cs file that automates the benchmarking process.

Script Workflow

  1. Retrieve and assign metadata used for self-contained Platform Metrics script execution, attribution
    • Get computer metadata for Platform Metrics attribution (hostname)
    • Get %TEMP% path, %TEMP%\LoginPI, %TEMP%\LoginPI\Results for temporary results storage
  2. Uses the command line to specify className, titles to search for within the respective browser
    • If "msedge" is present within the command line, it specifies Edge-specific UI controls
    • If "chrome" is present within the command line, it specifies Chrome-specific UI controls
  3. Run SilverBench benchmark
    • Launch Browser to "https://silver.urih.com/"
    • Run the standard Benchmark
    • Wait 120s (configurable) for Benchmark to complete
  4. Write payload, PowerShell script
    • Once the benchmark completes, we parse the browser for the SilverBench score
    • An object is created to store the Benchmark score, timestamp, and other metadata

  5. Generate PowerShell Script

    • Creates a PowerShell script to upload the benchmark data to the Login Enterprise API.

  6. Upload Platform Metrics
    • Executes the PowerShell script to upload the data as Platform Metrics to Login Enterprise.

Configurable Variables

  • TARGET
    • The script is capable of running within Edge and Chrome environments, using the following command lines:
      1. chrome.exe -incognito --force-renderer-accessibility "https://silver.urih.com/"
      2. msedge.exe -inPrivate --force-renderer-accessibility "https://silver.urih.com/" 
  • string timeOffset = "0:00";
    • Time offset from UTC in hours:minutes. Adjusts timestamps to your local time zone.
    • Examples:
      • "-7:00" for Pacific Standard Time (PST).
      • "+7:00" or "7:00" for UTC+7.
  • string configurationAccessToken = "**********";
    • Your configuration access token for the Login Enterprise API. To obtain:
      1. Log into Login Enterprise.
      2. Navigate to External notifications > Public API.
      3. Click New system access token.
      4. Provide a name and select Configuration from the Access level dropdown.
      5. Save and copy the token provided. Store it securely.

  • string baseUrl = "https://myLoginEnterprise.myDomain.com/";

    • The base URL of your Login Enterprise instance, including the ending slash.

  • string environmentId = "**********";

    • Your environment key/ID in Login Enterprise. To obtain:

      1. Log into Login Enterprise.

      2. Navigate to Configuration > Environments.

      3. You have two options:

        • Option A: Use an existing Environment. Click on the desired environment.
          The environment ID is the unique identifier at the end of the browser's address bar URL (e.g., 3221ce29-06ba-46a2-8c8b-da99dea341c4).

        • Option B: Create a new Environment. Click Add Environment. Fill out the required information (only Name is necessary). Click Save. After saving, the unique environment ID will be at the end of the browser's address bar URL.

    • For more information on Environments, see the Environments.

Interpreting Results

Once the wait period following the test execution is complete, the script attempts to parse the screen to find the SilverBench score. If there is a valid score, e.g. P21,950, we parse the numerical score and send it to Login Enterprise using the Platform Metrics API. 

  • timestamp: the timestamp is retrieved before executing the test
  • metricId: the name of the score (i.e. "SilverBench Score")
  • instance: the COMPUTERNAME of the target device where benchmarks were run
  • componentType: the name of the browser used (currently supports "msedge" and "chrome")
  • group: "SilverBench"

The timeOffset variable above is crucial to ensure that datapoints are attributed to their accurate UTC times.