Test run considerations for using Machine Learning capabilities

Before you configure a Performance Test or Schedule run to use Machine Learning (ML) capabilities, you must first read the considerations that you need to take into account.

When you want HCL OneTest™ Server to analyze Performance tests or Schedules by using ML capabilities, you must ensure that Performance tests or Schedules conform to the following criteria:
  • Must contain HTML tests.
  • Must contain stress or performance user profiles.
  • Must contain the following minimum stress test requirements:
    • 20 users or more.
    • A minimum of three different user groups and each group has an activity of more than two minutes.
    • Stages that have a reasonably high number of samples within each stage.
    • Stages do not have an increasing number of users.
    • Tests run for longer durations so that a trend can be observed.
You can refer to the following table to view the criteria for analysis of the parameters:
Parameter analyzed Criteria for analysis
Response Time Lock-Step Pattern The analyzer identifies the Response Time Lock-Step Pattern parameter in the overall page response time observed against the user count based on the following criteria:
  • Tests have a minimum of 20 users.
  • Tests run for longer durations for a trend to be observed.
  • Tests contain a minimum of three different user groups and each group has an activity of more than two minutes.
Response Time Standard Deviation Pattern The analyzer attempts to detect the response time of pages that are more than thrice the value of the standard deviation calculated for the page response time as the Response Time Standard Deviation Pattern parameter.
Throughput Drop Pattern The analyzer attempts to detect sudden drops in network throughput as the Throughput Drop Pattern parameter that is based on the following criteria:
Note: Sudden drops in throughput might be related to the performance tool itself, issues with network connectivity, or issues with signal-scalability of the system under test.
  • Tests have a minimum of 20 users.
  • Tests have stages with a reasonably high number of samples within a stage.
  • The length and intensity of the throughput drops that are less than 20% of the median observed within a stage are considered by the analyzer. When no throughput is received within 60 seconds, such drops are considered in the analysis. Short shark-tooth patterns in drops are not considered by the analyzer.
  • Parts of the stage with increasing users are not considered for analysis.

    For example, when the test adds one user every minute and the user range is between one user to 500 users, the Throughput Drop Pattern parameter is not analyzed.