Machine learning capabilities in HCL OneTest™ Server

HCL OneTest™ Server uses Machine learning (ML) algorithms to analyze test results that match certain requirements and criteria, and then presents the findings as insights and recommendations after the tests are run. You can use the insights and recommendations to interpret the test results and identify the problem areas in the system under test.

HCL OneTest™ Server has incorporated ML capabilities to provide an automated means to identify problems in the system under test.

Tests analyzed through ML

The following types of tests are supported for analysis by using ML capabilities in HCL OneTest™ Server:

Performance tests
Schedules

Parameters analyzed through ML

The following parameters in Performance tests or Schedules are supported for analysis by using ML capabilities in HCL OneTest™ Server:

The Response Time Lock-Step Pattern parameter.
The Response Time Standard Deviation Pattern parameter.
The Throughput Drop Pattern parameter.

HCL OneTest™ Server uses different analyzers for each of the parameters. For example, HCL OneTest™ Server uses the Response Time Lock-Step Pattern analyzer to analyze the tests for the Response Time Lock-Step Pattern parameter.

Criteria for analysis of the parameters

The following table lists the criteria used by the analyzers for analysis of the parameters in HCL OneTest™ Server:


Parameter analyzed	Criteria for analysis
Response Time Lock-Step Pattern	The analyzer identifies the Response Time Lock-Step Pattern parameter in the overall page response time observed against the user count based on the following criteria: Tests have a minimum of 20 users. Tests run for longer durations for a trend to be observed. Tests contain a minimum of three different user groups and each group has an activity of more than two minutes.
Response Time Standard Deviation Pattern	The analyzer attempts to detect the response time of pages that are more than thrice the value of the standard deviation calculated for the page response time as the Response Time Standard Deviation Pattern parameter.
Throughput Drop Pattern	The analyzer attempts to detect sudden drops in network throughput as the Throughput Drop Pattern parameter that is based on the following criteria: Note: Sudden drops in throughput might be related to the performance tool itself, issues with network connectivity, or issues with signal-scalability of the system under test. Tests have a minimum of 20 users. Tests have stages with a reasonably high number of samples within a stage. The length and intensity of the throughput drops that are less than 20% of the median observed within a stage are considered by the analyzer. When no throughput is received within 60 seconds, such drops are considered in the analysis. Short shark-tooth patterns in drops are not considered by the analyzer. Parts of the stage with increasing users are not considered for analysis. For example, when the test adds one user every minute and the user range is between one user to 500 users, the Throughput Drop Pattern parameter is not analyzed.

Analysis at the project level

When you as a Project Owner or Tester configure a run of a Performance Test or a Schedule from your project in HCL OneTest™ Server, you must ensure that the criteria required for ML analysis are met. During the test run, HCL OneTest™ Server uses its ML capabilities to analyze the supported parameters and presents its findings as insights on the Insights page.

Any member of your project can view the insights by clicking Analyze > Insights from the navigation in the project or from the Insights section on the Overview page in the project.

Project members after viewing the insights can react by either agreeing or disagreeing with the insights suggested by HCL OneTest™ Server.

Analysis at the team space level

HCL OneTest™ Server analyzes different results of the tests that are run across projects in a team space for the supported parameters that match the required criteria, and presents its findings as a recommendation about the analyzers on the Analyzers page in the team space.

Any member of your team space can view the recommendations by clicking Infrastructure > Analyzers from the navigation in the team space.

After viewing the recommendation for a specific parameter, members of the project can alter the default threshold value that specifies the level of confidence required in the analysis of the parameter and run the tests again to improve the accuracy of the recommendations and insights.

You can also improve the accuracy of the recommendations by running multiple tests with differing loads repeatedly.

Using Machine Learning capabilities

You can find information in the following documentation about using ML capabilities in HCL OneTest™ Server:

To understand the requirements of Performance tests or Schedules that are analyzed by using ML capabilities in HCL OneTest™ Server, see Test run considerations for using Machine Learning capabilities.
To configure a run of a Performance Test, see Configuring a run of a Performance test.
To configure a run of a Schedule, see Configuring a run of a Rate Schedule or VU Schedule.
To view the ML insights into the Performance tests or Schedules in your project, which were analyzed after a run, see Viewing Machine Learning insights. You can also select your reaction to agree or disagree with the insights provided when you view the insight.
To understand how to interpret the insights, see Interpretation of insights.
To view the ML analysis of the supported parameters when Performance tests or Schedules that are in different projects in your team space are run, see Viewing Machine Learning analyzers.
To understand how to interpret the recommendations of the ML analysis of the supported parameters, see Interpretation of parameter analysis.
To view the reactions of project members to the ML insights, see Viewing reactions marked by members to Machine Learning (ML) insights.
To modify the settings of the ML analyzer for a specific parameter, see Modifying Machine Learning analyzer settings.
To delete the ML insights provided about the Performance tests or Schedules that were analyzed, see Deleting Machine Learning insights.
To rerun the ML analysis without rerunning the test asset, see Rerunning Machine Learning (ML) analysis.