Creating and indexing shards

You can set up and index a specified number of sharding cores by defining them in an input properties file and running the parallel-process utility in the Utility server Docker container.

Before you begin

  1. Decide on the sharding configuration you want to use. For example, you can select a language-specific core that needs to be sharded.
  2. Decide on the number of shards to create.

About this task

During this task, you perform the following high-level steps:
  1. Create and populate the input properties file that is used to process indexing shards.
  2. Run the di-parallel-process utility to index the shards.

Procedure

Prepare the input properties file
  1. Create or update the input properties file that will be used for indexing, based on your environment and requirements.
    The sharding input properties file is used by the di-parallel-process utility to process indexing shards. It contains the following sections of properties:
    • System properties such as the password property file path.
    • Database properties, which are used to establish database connections with the database server.
    • Global preprocessing and indexing properties, which are used for preprocessing and indexing by all shards.
    • Master search server properties, which specify the master index cores where all shard data is merged.
    • Horizontal Shard properties, which specify the horizontal shard properties.
    The Utility Docker image contains two sample properties files. The example di-parallel-process.properties file uses the en_US locale and contains commented-out definitions for shards A, B, C and V (V is for vertical sharding). You can find the following files in the directory /opt/WebSphere/CommerceServer90/properties/parallelprocess/.
    di-parallel-process.properties
    The sample sharding input properties file for a Linux operating system that uses DB2. Refer to the latest di-parallel-process.properties configuration file in the HCL Commerce Utility docker container if you are using the Oracle database.
    password.properties
    The sample password properties file, referenced by the sample sharding input properties file. It contains passwords encrypted by the wcs_encrypt utility.

    For more information about the properties file and expected values, see Sharding input properties file.

Run the shard indexing utility
  1. Go to the following directory:
    • HCL Commerce DeveloperWCDE_installdir\bin
    • HCL Commerce runtime environment/opt/WebSphere/CommerceServer90/bin in the Utility server Docker container.
  2. Run the utility:
    • HCL Commerce Developer di-parallel-process.bat input_properties_file
    • HCL Commerce runtime environmentdi-parallel-process.sh input_properties_file [-dbuser dbuser] [-dbuserpwd dbuserpwd] [-searchuser searchuser] [-searchuserpwd searchuserpwd]

    Where:

    input_properties_file
    The path for the utility's input properties file can be either relative or absolute. For example, ../../di-parallel-process-linux-oracle.properties. The default path in the Utility server Docker container is /opt/WebSphere/CommerceServer90/properties/parallelprocess/di-parallel-process.properties.
    dbuser
    Optional: The name of the user who is connecting to the database.
    dbuserpwd
    Optional: The password for the user who is connecting to the database.
    searchuser
    Optional: The search application administrative user name.
    searchuserpwd
    Optional: The search application administrative user password.
  3. Ensure that the utility runs successfully. Either check the exit code, or the wc-dataimport-parallel-processor.log file for more information. The path for wc-dataimport-parallel-processor.log is:
    • HCL Commerce DeveloperWCDE_installdir\logs.
    • HCL Commerce runtime environment/opt/WebSphere/CommerceServer90/logs/.
    If you want the log to provide greater detail, set the log level to FINEST in the /opt/WebSphere/AppServer/profiles/default/installedApps/localhost/ts.ear/xml/config/dataimport/parallel-processor-logging.properties properties file.
    Tip: Permitted logging levels differ between development and runtime environments. The logging levels are:
  4. If you are using indexed contract prices, complete the following steps:
    1. Open the following file for all shards: WC_installdir/instances/instance_name/search/solr/home/MC_masterCatalogId/Shards/locale/CatalogEntry_#Shard-Tag#/conf/wc-data-config.xml.
    2. Find all instances of the following table name TI_CNTRPRICE _0_#Shard-Tag#.
    3. Replace them with the following table name: TI_CNTRPRICE _0. This update enables the utilities to look for contract prices in the TI_CNTRPRICE_0 table, instead of the shard-specific tables.
    4. Save your changes.
    5. For specific usage, see Indexing contract prices in HCL Commerce Search

What to do next

Replicate the merged index into the repeater, then to the other nodes.