Configuring Query services in ZooKeeper

You can build profiles for Ingest service customizations in ZooKeeper. Endpoints for configuring both development and production environment customizations are available. Note that these endpoints are available from within the Data environment and Development environment only. They are not available for the shopper to use through the Query service in either the Commerce Authoring or Live environment.

About this task

ZooKeeper is used to store your custom configurations. At runtime, each microservice polls ZooKeeper for any custom configurations it may also have stored. It will automatically override default behaviours such as query replies, and load custom applications such as new NiFi connectors. This task describes how to set up profiles in ZooKeeper, as well as specific custom search options such as units of measure and product colors. You define endpoints for the REST API so that these customizations are accessible through the gateway.

Procedure

  1. Set up your profiles.
    ZooKeeper stores named profiles in one of its nodes. You can create, update, delete the profiles, and get profiles using the profiles endpoint of the Query service. The endpoint for the profile is:
    https://data_environment_hostname:30921/search/resources/api/v2/documents/profiles/profileName
    No container needs to be restarted after you add a profile using this endpoint.

    For the full REST API specification, see the Query REST API.

    Each profile document is in .json format and has the following structure.
    {
	"indexName": "string",
	"parentProfileName": "string",
	"profileName": "string",
	"query": {
		"highlight": {
			"fields": [
				"name",
				"fieldName"
			],
			"simplePre": "<strong><span class=font2>",
			"simplePost": "</span></strong>"
		},
		"params": [
			{
				"paramName": "value"
			}
		],
		"postprocessor": [
			"postprocessorClass"
		],
		"preprocessor": [
			"preprocessorClass"
		],
		"provider": [
			"providerClass"
		],
		"queryFields": [
			"field1",
			"filed2"
		],
		"responseFields": [
			"field1",
			"filed2"
		],
		"sortFields": {
			"key": "value"
		},
		"spellcheck": {
			"limit": "5"
		},
		"group" : {
			"enable" : "false",
			"sort" : "score desc",
			"field" : "relationship.product.group"
		},
		"hero" : {
			"overrideFieldName" : "url.thumbnail",
			"showGroupPriceRange" : "false",
			"field" : {
				"7741124012283334335" : "Hero"
			}
		}
	}
}
  2. Configure specific Search customizations.
    The /configuration endpoint is used to add the customized configuration details in ZooKeeper node. There is a root node in ZooKeeper with the name /configuration, and inside the root node are child nodes where you can add the configuration. The available nodes are:
    1. xC cextensions
    2. wc-component
    3. Colors, organized by language
    4. Filters to aid in natural language processing
    5. Units of measure, and
    6. Relevancy
    1. To register xC customizations for pre- and post-processing of the actual search request to this child node, pass the configuration node the nodeName="xC" parameter as a request, for example: 
      https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=xC
      For the full REST API specification, see the Query Service API.
      Here is an example structure for an xC configuration:
      {
    "preProcess": [
        {
        	"storeId" : "1",
            "endpoint": {
                "query": "/productview/bySearchTerm/pre",
                "xC": "/commerceue/extension/search/productview/preBySearchTerm",
                "properties": [
                    {}
                ]
            }
        }
    ],
    "postProcess": [
        {
        	"storeId" : "1",
            "endpoint": {
                "query": "/productview/bySearchTerm/post",
                "xC": "/commerceue/extension/search/productview/postBySearchTerm",
                "properties": [
                    {
                        "fields": [
                            "name",
                            "shortDescription",
                            "UserData"
                        ]
                    }
                ]
            }
        }
    ]
}
      You can use Java, as in the following CustomProductViewResource.java file:
      package com.hcl.commerce.search.rest;

/*
 * Licensed Materials - Property of HCL Technologies Limited. (C) Copyright HCL Technologies Limited 1996, 2020.
 */

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/productview")
public class CustomProductViewResource {

	/**
	 * This method is use to add the search criteria.
	 */
	@SuppressWarnings({ "rawtypes", "unchecked" })
	@RequestMapping(value = "/preBySearchTerm", method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE)
	public ResponseEntity preFindProductsBySearchTerm(@RequestBody Map criteria) throws Exception {

		ArrayList list = (ArrayList) criteria.get("content");

		List filter = new ArrayList();
		filter.add("manufacturer.text:\"manufacturerName\"");

		Map filterMap = new HashMap();
		filterMap.put("_wcf.search.internal.filterquery", filter);
		list.add(filterMap);
		criteria.put("content", list);
		ResponseEntity result = new ResponseEntity(criteria, HttpStatus.OK);

		return result;
	}


	/**
	 * This method is use to manipulate the data post from server.
	 */
	@SuppressWarnings({ "rawtypes", "unchecked" })
	@RequestMapping(value = "/postBySearchTerm", method = RequestMethod.POST, produces = MediaType.APPLICATION_JSON_VALUE)
	public ResponseEntity postFindProductsBySearchTerm(@RequestBody Map response) throws Exception {

		ArrayList list = (ArrayList) response.get("content");
		for (Object obj : list) {
			if (obj instanceof Map) {
				Map m = (Map) obj;
				m.put("name", "Custom :: " + m.get("name"));
			}
		}
		ResponseEntity result = new ResponseEntity(response, HttpStatus.OK);

		return result;
	}
}
    2. You can also add a wc-component.json file to override or to add new configuration properties. The endpoint is: 
      https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=component
      Here is a sample structure for the component configuration.
      {
    "valuemappingservice": {
        "valuemapping": [
            {
                "internalName": "CatalogGroupUserDataFieldNameMapping",
                "externalName": "CatalogGroupUserDataFieldNameMapping",
                "valuemap": [
                    {
                        "externalValue": "UserRating",
                        "internalValue": "rating"
                    }
                ]
            },
            {
                "internalName": "CustomData",
                "externalName": "CustomData",
                "valuemap": [
                    {
                        "externalValue": "CustomDataExternal",
                        "internalValue": "CustomDataInternal"
                    }
                ]
            }
        ]
    },
    "extendedconfiguration": {
        "configgrouping": [
            {
                "name": "LoadOnStartup",
                "property": {
                    "name": "com.ibm.commerce.search.internal.runtime.bootstrap.SearchServiceWarmStarter",
                    "value": "FALSE"
                }
            },
            {
                "name": "MultipleContracts",
                "property": {
                    "name": "operation",
                    "value": "OR"
                }
            },
            {
                "name": "TransactionServerSetting",
                "property": [
                    {
                        "name": "TransactionServerSetting/SSLAcceleratorOption",
                        "value": "Enabled"
                    },
                    {
                        "name": "TransactionServerSetting/inSSLPort",
                        "value": "8888"
                    },
                    {
                        "name": "TransactionServerSetting/inSecure",
                        "value": "true"
                    }
                ]
            },
            {
                "name": "CustomGroup1",
                "property": {
                    "name": "customPropName",
                    "value": "customPropValue"
                }
            },
            {
                "name": "CustomGroup2",
                "property": [
                    {
                        "name": "customPropName1",
                        "value": "customPropValue1"
                    },
                    {
                        "name": "customPropName2",
                        "value": "customPropValue2"
                    }
                ]
            }
        ]
    }
}
    3. In the V9.1 Data Platform you can do natural-language searches using colors in product queries. In ZooKeeper, colors are registered as a root node, and en_US is registered as a child of colors. This hierarchy lets you maintain colors based on locale. To add colors inside ZooKeeper using the configuration endpoint, pass the nodeName and locale mandatory request parameters. In case of English color names you can only update the list of colors; for other locale specific color you can also add colors. To update the color value pass a data json file. To detete the record use the PATCH request method with an empty value for key.
      The endpoint for this service is:
      https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=colors&locale=en_US
      Following is a sample color_en_US.json configuration file for colors.
      {
	"ghost white": "white_248",
	"navajo white": "brown_255",
	"midnight blue": "blue_112",
	"darkslateblue": "blue_139",
	"lightgrey": "black_211",
	"dark khaki": "gold_183",
	"light salmon": "red_255",
	"saddlebrown": "brown_139",
	"moccasin": "yellow_228",
	"skyblue": "blue_235",
	"deepskyblue": "blue_235",
	"chartreuse": "green_255",
	"mediumpurple": "purple_147"
}
      In this example, "ghost white": "white_248" means that “ghost white” belong to the white color family. The number 248 represents the most significant color code for ghost white from the RGB color wheel.
    4. Inside ZooKeeper is a node with the name filter, which is used to parse or ignore redundant words from the search term during Natural Language Processing. At present the operations supported with filter are update, delete and retrieve. The add operation is not currently supported.
      The endpoint for this service is:
      https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=filter;locale=en_US
      Following is a sample filter.json file, for filtering or term-ignore.
      {
	"quarter": "TO_NUMBER~0.25"
	"half": "TO_NUMBER~0.50",
	"one and half": "TO_NUMBER~1.5",
	"but": "IGNORE_TERM",
	"usd": "IGNORE_TERM",
	"sterling": "IGNORE_TERM",
	"up to": "FILTER_LTE~1",
	"less than": "FILTER_LTE~1"
	"expensive": "FILTER_GTE~-1",
}
    5. The Natural Language Processing feature enables you to use units of measure. This feature is available to both the query service, for analyzing the search query string, and NiFi for analysing the attributes of products. Units of measure are updated and deleted using the PATCH request method. To delete a unit of measure, pass an empty value as the key. At present the operations supported with units of measure are update, delete and retrieve. The add operation is not currently supported.
      Here is the endpoint for this service:
      https://localhost:3737/search/resources/api/v2/configuration?nodeName=uoms;locale=en_US
      Sample uoms.json for units of measure:
      {
    "milliseconds": "millisecond",
    "yard": "yard",
    "centimeters": "centimeter",
    "year": "year",
    "hr": "hour",
    "yd": "yard",
    "years": "year",
    "millimeter": "millimeter",
    "milliliter": "milliliter",
    "inches": "inch",
    "seconds": "second",
    "foots": "foot",
    "nanometer": "nanometer",
    "gallons": "gallon",
    "pounds": "pound",
    "yr": "year",
    "grams": "gram",
    "kilogram": "kilogram",
    "\"": "inch",
    "nanosecond": "nanosecond",
    "in": "inch",
    "mile": "mile",
    "liters": "liter",
    "'": "foot",
    "litre": "liter",
    "μg": "microgram",
    "μm": "micrometer",
    "microsecond": "microsecond",
    "μs": "microsecond",
    "centimetre": "centimeter",
    "micrometre": "micrometer",
    "weeks": "week",
    "microgram": "microgram",
    "millisecond": "millisecond",
    "milligram": "milligram",
    "sec": "second",
    "min": "minute",
    "kilograms": "kilogram",
    "microseconds": "microsecond",
    "kg": "kilogram",
    "foot": "foot",
    "st": "stone",
    "km": "kilometer",
    "d": "day",
    "gram": "gram",
    "milliliters": "milliliter",
    "cm": "centimeter",
    "l": "liter",
    "m": "meter",
    "s": "second",
    "t": "tonne",
    "gallon": "gallon",
    "lb": "pound",
    "kilometer": "kilometer",
    "days": "day",
    "y": "year",
    "stones": "stone",
    "yrs": "year",
    "inch": "inch",
    "micrograms": "microgram",
    "millimeters": "millimeter",
    "second": "second",
    "mg": "milligram",
    "millilitre": "milliliter",
    "nanometre": "nanometer",
    "mi": "mile",
    "day": "day",
    "ml": "milliliter",
    "mm": "millimeter",
    "mo": "month",
    "millilitres": "milliliter",
    "months": "month",
    "meter": "meter",
    "minutes": "minute",
    "ms": "millisecond",
    "liter": "liter",
    "hrs": "hour",
    "yards": "yard",
    "millimetre": "millimeter",
    "month": "month",
    "lbs": "pound",
    "centimeter": "centimeter",
    "nm": "nanometer",
    "millimetres": "millimeter",
    "week": "week",
    "ns": "nanosecond",
    "yds": "yard",
    "milligrams": "milligram",
    "micrometer": "micrometer",
    "ft": "foot",
    "meters": "meter",
    "miles": "mile",
    "pound": "pound",
    "gal": "gallon",
    "wk": "week",
    "hour": "hour",
    "nanoseconds": "nanosecond",
    "tonne": "tonne",
    "litres": "liter",
    "hours": "hour",
    "gm": "gram",
    "ltr": "liter",
    "oz": "ounce",
    "minute": "minute",
    "stone": "stone",
    "kilometers": "kilometer",
    "ounce": "ounce"
}
      To enable the dimension matchmaker, call the configuration POST method in the dimensions node of the following endpoint:
      “http://{host}:{port}/search/resources/api/v2/configuration?nodeName=dimensions&locale=en_US"
      Configurations are entered in JSON format, as in the following example:
      {
	"pattern": “LWH”
}
      You can update a configuration using the PATCH method. Only one pattern can be specified in ZooKeeper's dimensions node, and currently, only the en_US locale is allowed.
    6. You can use the node index_field_relevancy to add the boost value for an index field at query time. This is used to boost the result based on the field. You can add or update usin the POST method; to delete the existing value use the PATCH method for the endpoint by passing an empty value as its key.
      The endpoint for the service is:
      https://data_environment_hostname:30921/search/resources/api/v2/configuration?nodeName=index_field_relevancy
      Following is sample data for index field relevency.
      {
    "default.suggestion": "1",
    "category.name": "100",
    "description.text": "5",
    "name.text": "10",
    "manufacturer.text": "10",
    "default.search.text": "1",
    "identifier.pn.raw": "15",
    "keyword.text": "100"
}