Configuring Query services in ZooKeeper
You can build profiles for Ingest service customizations in ZooKeeper. Endpoints for configuring both development and production environment customizations are available. Note that these endpoints are available from within the Data environment and Development environment only. They are not available for the shopper to use through the Query service in either the Commerce Authoring or Live environment.
About this task
ZooKeeper is used to store your custom configurations. At runtime, each microservice polls ZooKeeper for any custom configurations it may also have stored. It will automatically override default behaviours such as query replies, and load custom applications such as new NiFi connectors. This task describes how to set up profiles in ZooKeeper, as well as specific custom search options such as units of measure and product colors. You define endpoints for the REST API so that these customizations are accessible through the gateway.
Procedure
-
Set up your profiles.
ZooKeeper stores named profiles in one of its nodes. You can create, update, delete the profiles, and get profiles using the profiles endpoint of the Query service. The endpoint for the profile is:
No container needs to be restarted after you add a profile using this endpoint.https://data_environment_hostname:30921/search/resources/api/v2/documents/profiles/profileName
For the full REST API specification, see the Query REST API.
Each profile document is in .json format and has the following structure.{ "indexName": "string", "parentProfileName": "string", "profileName": "string", "query": { "highlight": { "fields": [ "name", "fieldName" ], "simplePre": "<strong><span class=font2>", "simplePost": "</span></strong>" }, "params": [ { "paramName": "value" } ], "postprocessor": [ "postprocessorClass" ], "preprocessor": [ "preprocessorClass" ], "provider": [ "providerClass" ], "queryFields": [ "field1", "filed2" ], "responseFields": [ "field1", "filed2" ], "sortFields": { "key": "value" }, "spellcheck": { "limit": "5" }, "group" : { "enable" : "false", "sort" : "score desc", "field" : "relationship.product.group" }, "hero" : { "overrideFieldName" : "url.thumbnail", "showGroupPriceRange" : "false", "field" : { "7741124012283334335" : "Hero" } } } }
-
Configure specific Search customizations.
The /configuration endpoint is used to add the customized configuration details in ZooKeeper node. There is a root node in ZooKeeper with the name /configuration, and inside the root node are child nodes where you can add the configuration. The available nodes are:
- xC cextensions
- wc-component
- Colors, organized by language
- Filters to aid in natural language processing
- Units of measure, and
- Relevancy
-
To register xC customizations for pre- and post-processing of the
actual search request to this child node, pass the configuration node
the nodeName="xC" parameter as a request, for
example:
For the full REST API specification, see the Query Service API.https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=xC
Here is an example structure for an xC configuration:
You can use Java, as in the following CustomProductViewResource.java file:{ "preProcess": [ { "storeId" : "1", "endpoint": { "query": "/productview/bySearchTerm/pre", "xC": "/commerceue/extension/search/productview/preBySearchTerm", "properties": [ {} ] } } ], "postProcess": [ { "storeId" : "1", "endpoint": { "query": "/productview/bySearchTerm/post", "xC": "/commerceue/extension/search/productview/postBySearchTerm", "properties": [ { "fields": [ "name", "shortDescription", "UserData" ] } ] } } ] }
package com.hcl.commerce.search.rest; /* * Licensed Materials - Property of HCL Technologies Limited. (C) Copyright HCL Technologies Limited 1996, 2020. */ import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; import org.springframework.http.HttpStatus; import org.springframework.http.MediaType; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.RestController; @RestController @RequestMapping("/productview") public class CustomProductViewResource { /** * This method is use to add the search criteria. */ @SuppressWarnings({ "rawtypes", "unchecked" }) @RequestMapping(value = "/preBySearchTerm", method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE) public ResponseEntity preFindProductsBySearchTerm(@RequestBody Map criteria) throws Exception { ArrayList list = (ArrayList) criteria.get("content"); List filter = new ArrayList(); filter.add("manufacturer.text:\"manufacturerName\""); Map filterMap = new HashMap(); filterMap.put("_wcf.search.internal.filterquery", filter); list.add(filterMap); criteria.put("content", list); ResponseEntity result = new ResponseEntity(criteria, HttpStatus.OK); return result; } /** * This method is use to manipulate the data post from server. */ @SuppressWarnings({ "rawtypes", "unchecked" }) @RequestMapping(value = "/postBySearchTerm", method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE) public ResponseEntity postFindProductsBySearchTerm(@RequestBody Map response) throws Exception { ArrayList list = (ArrayList) response.get("content"); for (Object obj : list) { if (obj instanceof Map) { Map m = (Map) obj; m.put("name", "Custom :: " + m.get("name")); } } ResponseEntity result = new ResponseEntity(response, HttpStatus.OK); return result; } }
-
You can also add a wc-component.json file to
override or to add new configuration properties. The endpoint is:
Here is a sample structure for the component configuration.https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=component
{ "valuemappingservice": { "valuemapping": [ { "internalName": "CatalogGroupUserDataFieldNameMapping", "externalName": "CatalogGroupUserDataFieldNameMapping", "valuemap": [ { "externalValue": "UserRating", "internalValue": "rating" } ] }, { "internalName": "CustomData", "externalName": "CustomData", "valuemap": [ { "externalValue": "CustomDataExternal", "internalValue": "CustomDataInternal" } ] } ] }, "extendedconfiguration": { "configgrouping": [ { "name": "LoadOnStartup", "property": { "name": "com.ibm.commerce.search.internal.runtime.bootstrap.SearchServiceWarmStarter", "value": "FALSE" } }, { "name": "MultipleContracts", "property": { "name": "operation", "value": "OR" } }, { "name": "TransactionServerSetting", "property": [ { "name": "TransactionServerSetting/SSLAcceleratorOption", "value": "Enabled" }, { "name": "TransactionServerSetting/inSSLPort", "value": "8888" }, { "name": "TransactionServerSetting/inSecure", "value": "true" } ] }, { "name": "CustomGroup1", "property": { "name": "customPropName", "value": "customPropValue" } }, { "name": "CustomGroup2", "property": [ { "name": "customPropName1", "value": "customPropValue1" }, { "name": "customPropName2", "value": "customPropValue2" } ] } ] } }
-
In the HCL Commerce Search you can do natural-language searches using
colors in product queries. In ZooKeeper, colors are registered as a root
node, and en_US is registered as a child of colors.
This hierarchy lets you maintain colors based on locale. To add colors
inside ZooKeeper using the configuration endpoint, pass the
nodeName and locale mandatory request
parameters. In case of English color names you can only update the list
of colors; for other locale specific color you can also add colors. To
update the color value pass a data json file. To detete the record use
the PATCH request method with an empty value for key.
The endpoint for this service is:
Following is a sample color_en_US.json configuration file for colors.https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=colors&locale=en_US
In this example,{ "ghost white": "white_248", "navajo white": "brown_255", "midnight blue": "blue_112", "darkslateblue": "blue_139", "lightgrey": "black_211", "dark khaki": "gold_183", "light salmon": "red_255", "saddlebrown": "brown_139", "moccasin": "yellow_228", "skyblue": "blue_235", "deepskyblue": "blue_235", "chartreuse": "green_255", "mediumpurple": "purple_147" }
"ghost white": "white_248"
means that “ghost white” belong to the white color family. The number 248 represents the most significant color code for ghost white from the RGB color wheel. -
Inside ZooKeeper is a node with the name filter,
which is used to parse or ignore redundant words from the search term
during Natural Language Processing. At present the operations supported
with filter are
update
,delete
andretrieve
. Theadd
operation is not currently supported.The endpoint for this service is:
Following is a sample filter.json file, for filtering or term-ignore.https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=filter;locale=en_US
{ "quarter": "TO_NUMBER~0.25" "half": "TO_NUMBER~0.50", "one and half": "TO_NUMBER~1.5", "but": "IGNORE_TERM", "usd": "IGNORE_TERM", "sterling": "IGNORE_TERM", "up to": "FILTER_LTE~1", "less than": "FILTER_LTE~1" "expensive": "FILTER_GTE~-1", }
-
The Natural Language Processing feature enables you to use units of
measure. This feature is available to both the query service, for
analyzing the search query string, and NiFi for analysing the attributes
of products. Units of measure are updated and deleted using the PATCH
request method. To delete a unit of measure, pass an empty value as the
key. At present the operations supported with units of measure are
update
,delete
andretrieve
. Theadd
operation is not currently supported.Here is the endpoint for this service:
Sample uoms.json for units of measure:https://localhost:3737/search/resources/api/v2/configuration?nodeName=uoms;locale=en_US
{ "milliseconds": "millisecond", "yard": "yard", "centimeters": "centimeter", "year": "year", "hr": "hour", "yd": "yard", "years": "year", "millimeter": "millimeter", "milliliter": "milliliter", "inches": "inch", "seconds": "second", "foots": "foot", "nanometer": "nanometer", "gallons": "gallon", "pounds": "pound", "yr": "year", "grams": "gram", "kilogram": "kilogram", "\"": "inch", "nanosecond": "nanosecond", "in": "inch", "mile": "mile", "liters": "liter", "'": "foot", "litre": "liter", "μg": "microgram", "μm": "micrometer", "microsecond": "microsecond", "μs": "microsecond", "centimetre": "centimeter", "micrometre": "micrometer", "weeks": "week", "microgram": "microgram", "millisecond": "millisecond", "milligram": "milligram", "sec": "second", "min": "minute", "kilograms": "kilogram", "microseconds": "microsecond", "kg": "kilogram", "foot": "foot", "st": "stone", "km": "kilometer", "d": "day", "gram": "gram", "milliliters": "milliliter", "cm": "centimeter", "l": "liter", "m": "meter", "s": "second", "t": "tonne", "gallon": "gallon", "lb": "pound", "kilometer": "kilometer", "days": "day", "y": "year", "stones": "stone", "yrs": "year", "inch": "inch", "micrograms": "microgram", "millimeters": "millimeter", "second": "second", "mg": "milligram", "millilitre": "milliliter", "nanometre": "nanometer", "mi": "mile", "day": "day", "ml": "milliliter", "mm": "millimeter", "mo": "month", "millilitres": "milliliter", "months": "month", "meter": "meter", "minutes": "minute", "ms": "millisecond", "liter": "liter", "hrs": "hour", "yards": "yard", "millimetre": "millimeter", "month": "month", "lbs": "pound", "centimeter": "centimeter", "nm": "nanometer", "millimetres": "millimeter", "week": "week", "ns": "nanosecond", "yds": "yard", "milligrams": "milligram", "micrometer": "micrometer", "ft": "foot", "meters": "meter", "miles": "mile", "pound": "pound", "gal": "gallon", "wk": "week", "hour": "hour", "nanoseconds": "nanosecond", "tonne": "tonne", "litres": "liter", "hours": "hour", "gm": "gram", "ltr": "liter", "oz": "ounce", "minute": "minute", "stone": "stone", "kilometers": "kilometer", "ounce": "ounce" }
To enable the dimension matchmaker, call the configuration POST method in the dimensions node of the following endpoint:
Configurations are entered in JSON format, as in the following example:“http://{host}:{port}/search/resources/api/v2/configuration?nodeName=dimensions&locale=en_US"
You can update a configuration using the PATCH method. Only one pattern can be specified in ZooKeeper's dimensions node, and currently, only the{ "pattern": “LWH” }
en_US
locale is allowed. -
You can use the node index_field_relevancy to add
the boost value for an index field at query time. This is used to boost
the result based on the field. You can add or update usin the POST
method; to delete the existing value use the PATCH method for the
endpoint by passing an empty value as its key.
The endpoint for the service is:
Following is sample data for index field relevency.https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=index_field_relevancy
{ "default.suggestion": "1", "category.name": "100", "description.text": "5", "name.text": "10", "manufacturer.text": "10", "default.search.text": "1", "identifier.pn.raw": "15", "keyword.text": "100" }