Building the Elasticsearch index
Once you have installed the HCL Commerce core applications and set up an initial storefront environment, start the default Search Docker containers. Then, follow these instructions to connect the Version 9.1 Search microservices into a working system.
Procedure
-
Build the Search index.
The preferred approach is to make the index-building call directly to the Ingest server. For every index except the price index, you can get better details and more complete logs this way. First, determine the name of the connector you are building. You would usually build against
auth.reindex
, which is analogous to performing a complete index build with Solr. You can build against other connectors if you want to index specific subsets.-
Build the index using the following REST call. This example assumes you
are indexing the
auth.reindex
connector.POST https://ingestServerHostname:port/connectors/auth.reindex/run?storeId=11
This call will return a runID. For example:{ "runId": "i-26144b4d-cd1c-4679-b2c2-870fc27e6095" }
-
You can check on the status of the build while it is ongoing. To do
this, take the runID returned in Step 1.a and make the following REST call,
inserting the runID after/runs/ in the string.
Using the above runID, for example, the call would
be:
GET http://ingestServerHostname:port/connectors/auth.reindex/runs/i-26144b4d-cd1c-4679-b2c2-870fc27e6095/status
The status information that is returned tells you approximately how much progress has been made in building the index.{ "date": "2022-04-06T17:51:44.462", "runId": "i-26144b4d-cd1c-4679-b2c2-870fc27e6095", "fromType": "Ingest", "message": "Indexing running, current progression of indexing is at process group: auth.reindex - WaitLink - Category Stage 1a", "status": -1, "progress": "30% (30 out of 101 pipes processed) " }
CAUTION:When indexing is complete, the call produces a report summarizing the index, with any errors and special conditions noted. For example,- The percentage value provided in the progress field does not refer to time. In the above example, 30% does not necessarily mean that you are 30% of the indexing time has elapsed. Rather, this percentage is a measure of the number of pipes processed so far.
- With subsequent calls you may see the progress number rise or fall. For example, your first call may return a progress value of 60%, while a second one done a short time later may return 30%. This occurs because the indexing process is broken up into different catalogs, languages, and so on and , and does not process them all at once.
By removing the /status at the end of the call, you can retrieve more detailed logs to review any errors. The price index is an exception; it can only be built against the Transaction server. This is because the price calculations are done on the Transaction server, so you still need to invoke the POST call against the Transaction server."message": "Indexing run finished according to Nifi queue being empty for given connector. {\"start\":\"2022-04-06T17:50:17.821Z\",\"end\":\"2022-04-06T17:53:02.346Z\",\"run\":\"i-26144b4d-cd1c-4679-b2c2-870fc27e6095\",\"severities\": {\"I\":15,\"W\":2} ,\"codes\":{\"DI1002I\":14,\"DI1050W\":2},\"locations\":{\"warning\": {\"Others\":2} ,\"info\":{\"Others\":14,\"Price Stage 2, Copy Contract Prices\":1}},\"elapsed\":{\"absolute\":{\"connector\":164525,\"summary\":{\"Attribute Pipeline\":\"4548\", \"Catalog Pipeline\":\"5155\",\"Category Pipeline\":\"13956\",\"Price Pipeline\":\"2911\",\"Product Pipeline\":\"21389\",\"Store Pipeline\":\"25336\",\"URL
Alternatively, you can use the following method to build the index.-
Issue the following command from within a REST client.
Leave the Body empty, and use basic authentication with the username spiuser and the password spiuserPassword.POST: https://ingestServerHostname:ingestServerPort/connectors/auth.reindex/run?storeId=11
Note:- The default password for the spiuser user is
passw0rd
for HCL Commerce 9.1.0.0 through 9.1.8.0, andQxV7uCk6RRiwvPVaa4wdD78jaHi2za8ssjneNMdu3vgqi
for HCL Commerce 9.1.9.0 and greater. - It is essential to set your own spiuser password to secure your deployment. For more information, see Setting the spiuser password in your Docker images.
In the example, the storeId is given as 11; replace this with the storeId for your own stores.
The default values are:- AuroraESite:
storeId=1
- AuroraB2BSite:
storeId=2
- Emerald
storeId=11
- Sapphire
storeId=12
OR
From a command prompt, issue the following command:
curl --user spiuser:spi_plain_text_password --insecure -X POST "https://ingestServerHostname:ingestServerPort/connectors/auth.reindex/run?storeId=11"
This call will return a runID value. For example:{ "runId": "i-26144b4d-cd1c-4679-b2c2-870fc27e6095" }
- The default password for the spiuser user is
-
Wait for the index build to complete, or observe its status using its
runID value.
Issue the following GET REST command to obtain the build status.Use the returned value for runID from the previous step.
GET http://ingestServerHostname:ingestServerPort/connectors/auth.reindex/runs/runId/status
OR
From a command prompt, issue the following command:curl --user spiuser:spi_plain_text_password --insecure -X GET "http://ingestServerHostname:ingestServerPort/connectors/auth.reindex/runs/runId/status"
- If your build is in progress, a message similar to the following
will provide details of its current
status.
{ "date": "2022-04-06T17:51:44.462", "runId": "i-26144b4d-cd1c-4679-b2c2-870fc27e6095", "fromType": "Ingest", "message": "Indexing running, current progression of indexing is at process group: auth.reindex - WaitLink - Category Stage 1a", "status": -1, "progress": "30% (30 out of 101 pipes processed) " }
Note: Progress percentage is a simple metric based on the number of pipes processed. It does not accurately reflect the progress in terms of time. - Once complete, a message similar to the following will detail
the result of the completed
build.
You can remove the /status at the end of the REST GET to obtain more detailed logs to review any encountered errors."message": "Indexing run finished according to Nifi queue being empty for given connector. {\"start\":\"2022-04-06T17:50:17.821Z\",\"end\":\"2022-04-06T17:53:02.346Z\",\"run\":\"i-26144b4d-cd1c-4679-b2c2-870fc27e6095\",\"severities\": {\"I\":15,\"W\":2} ,\"codes\":{\"DI1002I\":14,\"DI1050W\":2},\"locations\":{\"warning\": {\"Others\":2} ,\"info\":{\"Others\":14,\"Price Stage 2, Copy Contract Prices\":1}},\"elapsed\":{\"absolute\":{\"connector\":164525,\"summary\":{\"Attribute Pipeline\":\"4548\",\"Catalog Pipeline\":\"5155\",\"Category Pipeline\":\"13956\",\"Price Pipeline\":\"2911\",\"Product Pipeline\":\"21389\",\"Store Pipeline\":\"25336\",\"URL
- If your build is in progress, a message similar to the following
will provide details of its current
status.
-
Verify search index data.
Using a REST client, issue the following POST commands to verify that the search index data are available. Ensure that the contents of the Body consist of
{ "query": { "bool": { "must": { "match_all": {} } } } }
.- Store data:
http://ElasticSearchServerName:30200/auth.store/_search
- Category data:
http://ElasticSearchServerName:30200/auth.storeId.category/_search
- Product data:
http://ElasticSearchServerName:30200/auth.storeId.product/_search
- Store data:
-
Build the index using the following REST call. This example assumes you
are indexing the
-
Verify that the Query service is running and can find data. Using a REST
client, issue the following command:
This call will return the top categories for the specified store.https://CommerceServerName:30901/search/resources/store/storeId/categoryview/@top
-
Build B2B Contract Price Index
After you create Contracts with price adjustment, build the full index as in step #2, then build the contract price Index. Issue the following command from within a REST client.
POST: https://CommerceServerName:ts-app-port/wcs/resources/admin/index/dataImport/build?connectorId=price&indexSubType=Price&storeId=12
Leave the Body empty, and use basic authentication:POST: https://CommerceServerName:ts-app-port/wcs/resources/admin/index/dataImport/build?connectorId=auth.price&indexSubType=Price&storeId=12
Username:spiuser
andPassword: spiuserPassword
. In the example, the storeId is given as 12 (Sapphire); replace this with the storeId for your own stores.Check the build index status via the REST interface.
Where jobStatusId is the value of jobStatusId in the response from the above step. Use basic authentication with the username spiuser and the password spiuserPassword.GET: https://CommerceServerName:ts-app-port/wcs/resources/admin/index/dataImport/status?jobStatusId=jobStatusId
Note:- The default password for the spiuser user is
passw0rd
for HCL Commerce 9.1.0.0 to 9.1.8.0, andQxV7uCk6RRiwvPVaa4wdD78jaHi2za8ssjneNMdu3vgqi
for HCL Commerce 9.1.9.0 or greater.
Following are Optional Parameters to make a more targeted price update.
- contractId
- Optional: Used to build prices for the specified contract IDs.
- Specify comma-separated contract IDs to index.
- If no contract IDs are passed in, a full recalculation is performed by default.
- Example:
https://localhost/wcs/resources/admin/index/dataImport/build?indexSubType=Price&contractId=-10008,-10005
- catentry
- Optional: Used to build prices for the specified catalog entries.
- Specify comma-separated catalog entry IDs to index.
- If no catalog entry IDs are passed in, a full recalculation is performed by default.
- Example:
https://localhost/wcs/resources/admin/index/dataImport/build?indexSubType=Price&catentryId=10006,10005
- currency
- Optional: Used to build prices for the specified currencies.
- Specify comma-separated currencies to index.Note: Ensure that the CURLIST table contains any additional currencies that you want built by the calculate price utility.
- If no currencies are passed in, a full recalculation is performed by default.
- Example:
https://localhost/wcs/resources/admin/index/dataImport/build?indexSubType=Price¤cy=USD,CAD
- The default password for the spiuser user is
-
Activate the Authoring and Production Environments with Push to Live (PTL) end
to end. This step assumes that you have created the Push-to-Live
connectors.
- Set envType to live in the input.properties configuration file. Deploy the HCL Commerce live environment by following the instructions in Deploying HCL Commerce Version 9.1 with Docker Compose (for non-production usage).
-
To replicate search index to production environment, issue the
following command from within a REST client.
POST: https://CommerceServerName:ts-app-port/wcs/resources/admin/index/dataImport/build?connectorId=push-to-live&storeId=1
Leave the Body empty, and use basic authentication with the username spiuser and the password spiuserPassword.Note:- The default password for the spiuser user is
passw0rd
for HCL Commerce 9.1.0.0 to 9.1.8.0, andQxV7uCk6RRiwvPVaa4wdD78jaHi2za8ssjneNMdu3vgqi
for HCL Commerce 9.1.9.0 or greater. - The storeId is given as 1 (AuroraESite). Replace this value with the storeId for your own stores.
You can verify the status via the REST interface.GET: https://CommerceServerName:ts-app-port/wcs/resources/admin/index/dataImport/status?jobStatusId=jobStatusId
Restriction: If Catalog data changes are not available in your live stores after a push-to-live operation, trigger a WCT+ESINDEX invalidation operation when you make the update. For more information about the caches to be updated and the procedure, see Index changes are not reflected in storefront after Elasticsearch push-to-live.
- The default password for the spiuser user is
- For Management Center for HCL Commerce catalog changes, run a full index, run stagingprop as usual, then run PTL.