Creating search rule experiments

Create a search rule experiment to find out how effective certain search rules are at driving sales. For example, you can use two or more different search actions in a single search rule, each on a different path, and run both paths on your storefront for a time. You can then view a report that shows which path is generating more clicks, orders, and revenue from the search results page.

Before you begin

Review the search rule examples and guidelines in Search rule experiments.

About this task

Search rule experiments are configured so that only the first page of search results is considered when HCL Commerce collects experiment statistics. For example, if a customer clicks a link on the second page of the search results, the click is not counted in the statistics. This configuration is intentional because search rules are designed to influence shoppers by promoting a relatively small set of products that would likely fit on one page. If you want more pages of search results to be considered in the statistics, ask a site administrator to change the default configuration before the experiment starts. Site administrators can change the value of the SearchStatisticsResultPagesTrackingThreshold property.

Procedure

  1. Open the Marketing tool.
  2. Do one of the following things:
    • Open an existing search rule that you want to experiment with, and deactivate it.
    • Start a new search rule that contains one set of elements that you want to test. For example, a trigger followed by an action (a target is optional). Complete the properties for the search rule, and for each element on the path, as you normally would.
  3. From the Branching section in the palette, drag the Experiment element Experiment element onto the path to the left of the element that you want to test.
    For example, if you want to test the Specify Top Search Results action, drag the Experiment element to the left of that action.
    The search rule is split into two paths.
  4. Optional: You can add more paths to the experiment by right-clicking the Experiment element, and then clicking Add Path. You can delete any paths by right-clicking the green path arrow, and then clicking Delete.
  5. Click the Experiment element.
  6. Enter the experiment properties:
    PropertyDescription
    Name Enter a name to identify the experiment.
    Branch type
    Random path
    This branch type creates weighted random paths. You assign each path a percentage so that, for example, 50% of customers follow Path A and 50% follow Path B.
    First path for which the customer qualifies
    Use this branch type only when each of your paths starts with a target. The customer can follow only one path, which is the first path that starts with a target for which the customer qualifies. The paths are evaluated from top to bottom as represented in the Search Rule Builder.
    Frequency of element changes for each customer Select one of the following options:
    No change - Customer always sees the same content
    This option specifies that customers who perform the search multiple times always follows the same path during the life of the experiment.
    Session - Customer sees the same content within a session
    This option specifies that customers who perform the search multiple times always follows the same path during the current session. If they qualify for a different path, they might follow a different path during the next session.
    Request - Customer might see different content on each view
    This option specifies that customers who perform the search multiple times are not restricted to the same path for any length of time. If they qualify for a different path, they might follow a different path the next time they search.
    Maximum number of customers This option specifies the number of customers who are to participate in the experiment. If you do not specify the maximum number of customers, you must specify an end date.

    When the maximum number of customers is reached, new customers are assigned the control path (top path in the experiment) but statistics are no longer collected because the experiment is ended.
    Start Date Define a start date and time for the experiment. You can either enter a date in the field, or select a date by using the calendar tool. Similarly, you can enter a time directly into the field, or you can select one using the clock tool. If you do not specify a start date, the experiment starts when the activity is scheduled to start.
    End Date Define an end date and time for the experiment. You can either enter a date in the field, or select a date by using the calendar tool. Similarly, you can enter a time directly into the field, or you can select one using the clock tool. If you do not specify an end date, the experiment runs when the activity is scheduled to run, unless you specify a value in the Maximum number of customers field.
    Session length

    You can specify the amount of time in seconds from when a customer performed the search to the time of their order for the order data to be included in the revenue statistics. The default is 3600 seconds (one hour). If one hour is sufficient for your experiment, you can leave this field empty. If the Frequency of element changes for each customer is set to Session, the session length is also the amount of time during which the customer follows the same path in the experiment. After the session time expires, the customer might be assigned a different path in the experiment.
    Status Select the status of the experiment. You can select either Running or Suspended.
  7. Click the Paths tab, and enter each path's properties in the Paths table:
    PropertyDescription
    Name Enter a meaningful name in the field for each path. This name identifies the path in both this tab and the Statistics tab.
    Percentage The Percentage column is visible only if you selected the Random path branch type.

    Enter the percentage weighting for each path. For example, if you have two paths, A and B, and each is set at 50%, then the server assigns Path A to 50% of customers and Path B to the other 50%. You can assign any percentage to the paths in the experiment, but the total percentage for all paths must be 100%. Typically, you would assign proportionate percentages to multiple paths (for example, 50/50 for two paths 25/25/25/25 for four paths), but this proportionate percentage is not required.
    Sequence This column displays the path's position relative to the other paths in the experiment. This column is hidden by default; to display it, see Using table views.
    Unique ID This column displays the internal database ID for this object; this ID is used for troubleshooting purposes. This column is hidden by default; to display it, see Using table views.
  8. Set up all the paths in the experiment by adding, removing, or changing targets and actions and their properties.

    Tip: You can leave one of the paths empty. Customers assigned the empty path see search results that are not affected by a search rule action. By comparing statistics from the empty path to statistics on the other paths, you can determine whether your search rules are having a positive effect.

  9. Save the search rule experiment.
  10. Activate the search rule.

Example

The following is an example of a search rule experiment with two equally weighted paths that change the search results in different ways:
Example of a search rule experiment