Search and Marketing statistics in WebSphere Commerce

In a typical deployment model, business users typically do not have access to the production system. However, for business users to review live production data, such as search and marketing statistics, it is required that the data be replicated into a staging or authoring environment to be viewed in the Management Center. The production system is not impacted by the data replication operation because it is the staging or authoring system that pulls the production data back into the staging database, without involving any production CPU cycles.

The DownloadStatistics scheduler command performs statistics replication from production into the staging or authoring environment. DownloadStatistics reads the search section in the wc-component.xml file from the Infrastructure component for the tables to replicate:

There are two types of statistics tables. The first type is a log-style statistics table where incremental data is appended to the end of the table over time. This table only requires incremental replication. The other type of statistics table is the running-counter style where all rows are active with a running counter. In this case, a full table replication is required.
The production system database connection is the datasource connection to the production system for Quick Publish. This data pipeline is used for replicating statistics data from the production system back to the staging area.
The DownloadStatistics scheduler command must be configured to run on a recurring basis from a staging or authoring environment, where business users review the statistic in the Management Center.

The following code is an example of the DownloadStatistics scheduler job configuration in the infrastructure component wc-component.xml file:

<_config:extendedconfiguration>
  <_config:configgrouping name="DownloadStatistics-Configurations">
    <_config:property name="SearchStatistics" value="DownloadStatistics-SRCHSTAT"/>
    <_config:property name="ElementStatistics" value="DownloadStatistics-DMELESTATS"/>
    <_config:property name="EMarketingSpotStatistics" value="DownloadStatistics-DMEMSPOTSTATS"/>
    <_config:property name="ExperimenStatistics" value="DownloadStatistics-DMEXPSTATS"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-SRCHSTAT">
    <_config:property name="TableName" value="SRCHSTAT"/>
    <_config:property name="Mode" value="move"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-DMELESTATS">
    <_config:property name="TableName" value="DMELESTATS"/>
    <_config:property name="Mode" value="copy"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-DMEMSPOTSTATS">
    <_config:property name="TableName" value="DMEMSPOTSTATS"/>
    <_config:property name="Mode" value="copy"/>
  </_config:configgrouping>
  <_config:configgrouping name="DownloadStatistics-DMEXPSTATS">
    <_config:property name="TableName" value="DMEXPSTATS"/>
    <_config:property name="Mode" value="copy"/>
  </_config:configgrouping>
</_config:extendedconfiguration>

Once search statistics are enabled in production, it is required to set up the replication scheduler job in the staging area so that business users can view the search statistics without the need for a production connection. The following search and marketing statistics tables are configured by default:

SRCHSTAT
DMELESTATS
DMEMSPOTSTATS
DMEXPSTATS

These tables contains data that represents statistics data from the Marketing component and WebSphere Commerce search.

Note: On the system to which data is replicated, that system cannot collect statistics from its local environment. For example, if search statistics is configured to be replicated from production to staging, then the staging server can only be used to review production search statistics and cannot have search statistics that are enabled in the staging environment. This is to avoid key collision in the staging database.

You must ensure that the SRCHSTAT table in the staging and production system is regularly cleaned (dbclean) to maintain overall performance when search statistics are enabled.

Tuning search statistics

You can configure search statistics properties in the catalog component configuration file (wc-component.xml) file. For more information, see Search properties in the component configuration file (wc-component.xml) (WebSphere Commerce EAR).