Setting up the HCL Commerce Developer Search environment

HCL Commerce Version 9.1 provides support for Elasticsearch by default. Use the HCL Commerce Developer Search environment to make customizations to the search solution.

Note: If you are using the Solr-based search solution, use the HCL Commerce Developer Legacy environment to develop your Search customizations. For more information, see:

Before you begin

The HCL Commerce Developer Search environment requires the following before you can use it to develop your Elasticsearch-based search customizations:

  • Important: The Elasticsearch-based Search solution is resource intensive. It is recommended to dedicate standalone hardware or hosted virtual machine resources and not run it on a local development machine.
  • Before installing HCL Commerce Developer software, ensure that your HCL Commerce development environment meets the hardware and software requirements.
  • The requires a database.

    The supports and HCL Commerce Version 9.1.2.0 or later. For more information about installing and configuring a development environment database, see Install and configure your development environment database.

  • HCL Cache caches classes that can be modified in newer versions of HCL Commerce. To avoid errors in de-serializing an old version of the class, it is strongly recommended to clear Redis keys after upgrading HCL Commerce. Redis keys can be cleared with the Redis flushdb or flushall commands.
WindowsNote: When using Microsoft Windows:
  • Ensure ample resources are allocated to Docker.

    To allocate resources to Docker, right click on your Docker executable, or shortcut to it, and select Advanced. Allocate at minimum: 8 CPU cores, 32GB of memory, and 80GB of storage to ensure adequate performance.

  • Ensure the following network setting is enabled within the Docker settings > General settings: 
    Expose daemon on tcp://localhost:2375 port

Procedure

  1. Obtain the HCL Commerce Docker images.
    To start the deployment process, download the required software.
    1. Review the list of the latest available download packages to ensure that you are obtaining the most up-to-date version of HCL Commerce software.
    2. Go to the .
    3. Search for Package ID HCL_Commerce_Devops_Version_9.1, or File ID HCL_Commerce_DevOps_9.1.x.x.bundle.
    4. Download the file.
    5. Download the Data Docker images.
      • HCL_Commerce_Enterprise_9.1.x.x_Data_Query_Server_x86-64.tgz
      • HCL_Commerce_Enterprise_9.1.x.x_Data_NiFi_Server_x86-64.tgz
      • HCL_Commerce_Enterprise_9.1.x.x_Data_NiFi_Registry_Server_x86-64.tgz
      • HCL_Commerce_Enterprise_9.1.x.x_Data_Ingest_Server_x86-64.tgz
      • Optional:HCL_Commerce_Enterprise_9.1.x.x_Approval_Server_x86-64.tgz
      • Optional:HCL_Commerce_Enterprise_9.1.x.x_GraphQL_Server_x86-64.tgz
      Load each Docker image. For each downloaded image file, run the following command.
      docker load -i imagename

      For example:

      docker load -i HCL_Commerce_Enterprise_9.1.0.0_Data_Query_Server_x86-64.tgz
    6. Clone the project in Git from the downloaded git bundle, and check the code into your source control management system. Run the following command.
      git clone bundleName projectName
      Where:
      bundleName
      The filename of the bundle you are cloning.
      projectName
      The name of the git project that you are creating.
      For example: 
      git clone HCL_Commerce_DevOps_9.1.0.0.bundle HCL_Commerce_DevOps
  2. Deploy the Elasticsearch-based Search solution.
    • HCL Commerce Version 9.1.6.0 or laterDeployment option #1: Deploy the Elasticsearch-based Search solution via the deployment script.

      This option is limited to deployment within a Linux environment, due to the reliance on Bash scripts (.sh files).

      1. Update the env.sh bash script with the details of your environment.
        1. Open env.sh for editing.
        2. Set the following environment details depending on your version of .
          • HCL Commerce Version 9.1.12.0 or laterFor 9.1.12.0 and greater, the following values must be set:
            SEARCH_ENGINE=elasticsearch
DEPLOY_TYPE=toolkit
LICENSE_ACCEPTED=accept

TOOLKIT_HOST=<toolkitHost>
TOOLKIT_HOST_IP=<toolkitHostIP>
SEARCH_HOST=<searchHost>
SEARCH_HOST_IP=<searchHostIP>

DB_TYPE=<db2|oracle>
DB_HOST=<databaseHost>
DB_NAME=<databaseName>
DB_USER=<databaseUser>
DB_PASS=<databaseUserPassword>

SEARCH_QUERY_IMAGE=<searchQueryImage>
SEARCH_NIFI_IMAGE=<searchNifiImage>
SEARCH_REGISTRY_IMAGE=<searchRegistryImage>
SEARCH_INGEST_IMAGE=<searchIngestImage>

SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord>
ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword>
SPIUSER_PWD_BASE64=<base46SpiuserPassword>
          • HCL Commerce Version 9.1.12.0 or laterFor 9.1.12.0 and greater, optionally set the following configurations:
            APPROVAL_ENABLED=<true|false>
APPROVAL_APP_IMAGE=<approvalImage>
SESSION_KEY_ENCRYPT=<encryptedSessionKey>
GRAPHQL_ENABLED=<true|false>
GRAPHQL_APP_IMAGE=<graphqlImage>
ALLOW_TELEMETRY=<yes|no>
CACHE_ENABLED=<true|false>
CACHE_APP_IMAGE=<cacheManagerImage>
          • For 9.1.0.0 to 9.1.11.0:
            SEARCH_ENGINE=elasticsearch
DEPLOY_TYPE=data
DATA_FOR_TOOLKIT=true
LICENSE_ACCEPTED=accept

TOOLKIT_HOST=<toolkitHost>

DB_TYPE=<db2|oracle>
DB_HOST=<databaseHost>
DB_NAME=<databaseName>
DB_USER=<databaseUser>
DB_PASS=<databaseUserPassword>

SEARCH_QUERY_IMAGE=<searchQueryImage>
SEARCH_NIFI_IMAGE=<searchNifiImage>
SEARCH_REGISTRY_IMAGE=<searchRegistryImage>
SEARCH_INGEST_IMAGE=<searchIngestImage>

SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord>
ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword>
SPIUSER_PWD_BASE64=<base46SpiuserPassword>
          Note:
        3. Save and close the file.
        4. Back up the env.sh script containing your environment details for future deployment.
      2. If you are using an database, ensure that there is a copy of the required JDBC driver (ojdbc8.jar) located within the volumes/jdbc/ directory.
      3. Run the deployment bash script.
        ./deploy.sh
        This script produces an Elasticsearch-based Search Docker Compose file that is based on the environment configuration in env.sh.
        • HCL Commerce Version 9.1.12.0 or laterFor 9.1.12.0 and greater, this file is docker-compose-toolkit.yml.
        • For 9.1.0.0 to 9.1.11.0, this file is docker-compose-data.yml.

        The deployment will set the project to commerce, so the created Docker services will have commerce_ prefix. For example, commerce_nifi_1.

        The deployment script will automatically start the deployment.

    • HCL Commerce Version 9.1.6.0 or laterDeployment option #2: Deploy the platform manually.

      This option is the only deployment that is compatible with a Windows host operating system, due to its non-reliance on Bash scripts (.sh files).

      1. Create a Docker Compose deployment file for your environment.
        1. Select a deployment Docker Compose template file based on the database that you will use for the deployment.

          The template files are follow the naming convention: docker-compose-elasticsearch-data-<DB_TYPE>-toolkit-template.yml.

          Where <DB_TYPE> can be db2 or oracle.

        2. Copy the template into your Docker Compose deployment file (docker-compose-toolkit.yml).
        3. Add optional services to your Docker Compose deployment file (docker-compose-toolkit.yml).
          • GraphQL (docker-compose-elasticsearch-svc-graphql-toolkit-template.yml)
          • Cache Manager (docker-compose-elasticsearch-data-toolkit-cache-app-svc-template.yaml)
        4. HCL Commerce Version 9.1.12.0 or later ( docker-compose-svc-approval-toolkit-template.yml)
          Note: In addition to including the content from this Docker Compose template, to enable the you must overwrite volumes/approval-app/boot-config/app-secure.properties with the contents of volumes/approval-app/boot-config/app-secure.properties.nonssl.
      2. Update the env.sh bash script with the details of your environment.
        1. Open env.sh for editing.
        2. Set the following environment details depending on your version of .
          • HCL Commerce Version 9.1.12.0 or laterFor 9.1.12.0 and greater, the following values must be set:
            SEARCH_ENGINE=elasticsearch
DEPLOY_TYPE=toolkit
LICENSE_ACCEPTED=accept

TOOLKIT_HOST=<toolkitHost>
TOOLKIT_HOST_IP=<toolkitHostIP>
SEARCH_HOST=<searchHost>
SEARCH_HOST_IP=<searchHostIP>

DB_TYPE=<db2|oracle>
DB_HOST=<databaseHost>
DB_NAME=<databaseName>
DB_USER=<databaseUser>
DB_PASS=<databaseUserPassword>

SEARCH_QUERY_IMAGE=<searchQueryImage>
SEARCH_NIFI_IMAGE=<searchNifiImage>
SEARCH_REGISTRY_IMAGE=<searchRegistryImage>
SEARCH_INGEST_IMAGE=<searchIngestImage>

SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord>
ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword>
SPIUSER_PWD_BASE64=<base46SpiuserPassword>
          • HCL Commerce Version 9.1.12.0 or laterFor 9.1.12.0 and greater, optionally set the following configurations:
            APPROVAL_ENABLED=<true|false>
APPROVAL_APP_IMAGE=<approvalImage>
SESSION_KEY_ENCRYPT=<encryptedSessionKey>
GRAPHQL_ENABLED=<true|false>
GRAPHQL_APP_IMAGE=<graphqlImage>
ALLOW_TELEMETRY=<yes|no>
CACHE_ENABLED=<true|false>
CACHE_APP_IMAGE=<cacheManagerImage>
          • For 9.1.0.0 to 9.1.11.0:
            SEARCH_ENGINE=elasticsearch
DEPLOY_TYPE=data
DATA_FOR_TOOLKIT=true
LICENSE_ACCEPTED=accept

TOOLKIT_HOST=<toolkitHost>

DB_TYPE=<db2|oracle>
DB_HOST=<databaseHost>
DB_NAME=<databaseName>
DB_USER=<databaseUser>
DB_PASS=<databaseUserPassword>

SEARCH_QUERY_IMAGE=<searchQueryImage>
SEARCH_NIFI_IMAGE=<searchNifiImage>
SEARCH_REGISTRY_IMAGE=<searchRegistryImage>
SEARCH_INGEST_IMAGE=<searchIngestImage>

SPIUSER_PWD_ENCRYPTED=<encryptedSpiUserPassord>
ADMIN_SPIUSER_PWD=<plainTextSpiuserPassword>
SPIUSER_PWD_BASE64=<base46SpiuserPassword>
          Note:
        3. Save and close the file.
        4. Back up the env.sh script containing your environment details for future deployment.
      3. Copy your configured env.sh values into the Docker .env file on the same level as your Docker Compose file (docker-compose-toolkit.yml). This allows for docker-compose to use these environment variables in the deployment.
      4. Set file permission for your persistent volumes.

        Some volumes are mounted to your running containers and require write permissions.

        Ensure that you recursively set write access to /volumes/, to ensure that it and all of the contained sub-directories are writable.

      5. Run the following command to start the Elasticsearch Search solution.
        docker-compose up -d
    • Legacy deployment option: Deploy a legacy version of the platform manually.
      In the event that you need to support an older version of 9.1, this option is the only method available for deploying its corresponding Elasticsearch-based solution.
      • This deployment option is no longer supported with 9.1.6.0 and greater.
      • This is the only deployment option for versions 9.1.0.0 through 9.1.5.0.
      1. From the docker-compose folder, copy the file DB2docker-compose-elasticSearch-data-db2-toolkit-template.yml or Oracledocker-compose-elasticSearch-data-oracle-toolkit-template.yml to docker-compose.yml.
      2. Update the .yml image tag for all HCL Commerce Docker containers.
        1. Determine the image repository and tag by running the Docker images.
        2. Update each image tag in the .yml file with the its respective repository:tag. For example, image: commerce/search-nifi-app:9.1.0.0
      3. Accept the LICENSE parameter for all docker containers, for example: - LICENSE=accept
      4. Update the following configuration for Nifi to point to the database.
          AUTH_JDBC_URL: "jdbc:DBType://DBHost:50000/mall"
  AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/DBDriver"
  AUTH_JDBC_USER_NAME: "DBUser"
  AUTH_JDBC_USER_PASSWORD: "DBUserPassword"
  NLP_ENABLE_LANGUAGE_CODE: "nlpEnableLanguageCode"

        For example:

        DB2
        AUTH_JDBC_URL: "jdbc:db2://tk.hclcommerce.com:50000/mall"
  AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/db2jcc4.jar"
  AUTH_JDBC_USER_NAME: "wcs"
  AUTH_JDBC_USER_PASSWORD: "wcs1"
  NLP_ENABLE_LANGUAGE_CODE: "EN"
        Oracle
        AUTH_JDBC_URL: "jdbc:oracle:thin:@//tk.hclcommerce.com:1521/orclpdb1"
AUTH_JDBC_DRIVER_CLASSNAME: "oracle.jdbc.driver.OracleDriver"
AUTH_JDBC_DRIVER_LOCATION: "/opt/nifi/nifi-current/lib/ojdbc8.jar"
AUTH_JDBC_USER_NAME: "wcs"
AUTH_JDBC_USER_PASSWORD: "wcs1"
NLP_ENABLE_LANGUAGE_CODE: "EN"
      5. Update the configuration for data-query.
          TX_HOST: "TXHost"

        For example:

        TX_HOST: "tk.hclcommerce.com"
      6. Create the following sub-directories within in the same directory where docker-compose.yml is located, and ensure that their permissions are set to 777.
        Important: If you are upgrading from 9.1.0.0 or 9.1.1.0 to a newer version of , or if you have any issues related to the search index or NiFi service connector, you must delete the contents of these directories.
        • nifiLogs
        • HCL Commerce Version 9.1.1.0 or laterelasticsearch/data
        • HCL Commerce Version 9.1.1.0 or laterelasticsearch/config
        • HCL Commerce Version 9.1.1.0 or laterzookeeper
        • HCL Commerce Version 9.1.1.0 or laternifi/data
        • HCL Commerce Version 9.1.2.0 or laterredis/data
      7. HCL Commerce Version 9.1.2.0 or laterCopy the hcl-cache-search folder to the location where the Docker Compose file (docker-compose.yml) is located.
      8. If you are using an or database, ensure that there is a copy of the required JDBC driver (ojdbc8.jar) in the same directory where the Docker compose file (docker-compose.yml) is located.
      9. Run the following command to start the Elasticsearch Search solution.
        docker-compose up -d
  3. Verify that all the containers are up and healthy.
  4. Set your HCL Commerce development environment Elasticsearch environment variables.
    Note: If you want to change back to the Solr search engine at a later data, make a backup of jndi.xml under Liberty_Dir\usr\servers\crsServer\configDropins\overrides\, and namebindings.xml under WCDE_InstallDir\wasprofile\config\cells\localhost\nodes\localhost\servers\server1\. Then, you can revert to Solr with your Elasticsearch settings backed up.
    1. Navigate to the WCDE_InstallDir/bin directory.
    2. Run the following command script:
      configureElasticSearch.bat elasticSearchDataQueryHost elasticSearchDataQueryPort elasticSearchIngestHost elasticSearchIngestPort elasticSearchHost elasticSearchPort elasticSearchScheme HCL Commerce Version 9.1.3.0 or laterredisHost HCL Commerce Version 9.1.3.0 or laterredisPort toolingSDKInstallDir reactStoreSDKInstallDir

      Where:

      elasticSearchDataQueryHost
      The Elasticsearch query hostname or IP address.
      elasticSearchDataQueryPort
      The Elasticsearch query port number.
      elasticSearchIngestHost
      The Elasticsearch ingest hostname or IP address.
      elasticSearchIngestPort
      The Elasticsearch ingest port number.
      elasticSearchHost
      The Elasticsearch hostname or IP address.
      elasticSearchPort
      The Elasticsearch port number.
      elasticSearchScheme
      The Elasticsearch URL scheme. Values: http or https.
      HCL Commerce Version 9.1.3.0 or laterRedisHost
      The Redis hostname or IP address.
      HCL Commerce Version 9.1.3.0 or laterRedisPort
      The Redis port number.
      toolingSDKInstallDir
      The local installation directory of the HCL Commerce Developer Tooling environment Software Development Kit.
      Note:
      • If the toolingSDKInstallDir path contains any spaces, ensure that it is fully encapsulated within double quotation marks ("").
      reactStoreSDKInstallDir
      Optional: The local installation directory of the HCL Commerce Developer React Store environment Software Development Kit.
      Note:
      • The reactStoreSDKInstallDir parameter is optional.
      • If the reactStoreSDKInstallDir path contains any spaces, ensure that it is fully encapsulated within double quotation marks ("").
      For example: 
      configureElasticSearch.bat elasticsearch.commerce.com 30921 elasticsearch.commerce.com 30801 elasticsearch.commerce.com 30200 http elasticsearch.commerce.com 6379 C:\git-src\github01\toolingsdk C:\git-src\github01\store-web
  5. Optional: Enable the HCL Commerce Cache Manager application.
    HCL Commerce Version 9.1.12.0 or laterNote: For releases of HCL Commerce 9.1.12.0 and greater, the Cache Manager application can be deployed more easily by updating the env.sh file with CACHE_ENABLED=true and then specifying the CACHE_APP_IMAGE.
  6. HCL Commerce Version 9.1.12.0 or later If you are upgrading from HCL Commerce 9.1.11.0 to 9.1.12.0 or greater, you must update the database to launch the Emerald or Sapphire marketplace.
    In a database command prompt, run the following SQL.
    insert into storeconf values(32,'hcl.marketplace.self-registration','true',1);
insert into storeconf values(31,'hcl.marketplace.self-registration','true',1);

Results

Your HCL Commerce development environment is now configured to develop Elasticsearch-based search customizations.

What to do next

Build your Elasticsearch search index. To build your search index, see Building the Elasticsearch index.