HCL Commerce Version 9.1.4.0 or later

Extending the Query Service

HCL Commerce Version 9.1.4.0 or laterThe Query service builds the search expressions and then hands the expression to Elasticsearch. You can customize this service to create your own expression providers, pre-processors, post-processors, and custom handlers as per your business requirements. There is no JDBC connectivity from the Query service to the Commerce database. This architecture ensures that the application can be stateless and scaled independently. Business data can still be provided to the Query service either by indexing or through other micro-services.

Before you begin

Ensure that you have installed and set up the Eclipse workbench to use it as your own search toolkit.

Several new helper classes have been added to the query-api.jar file. These additions make it easier for you to use common functions defined in the default classes, and are listed in Query classes in the query-api.jar file for your convenience.

About this task

To customize the Query service:

Procedure

  1. The query-api.jar is delivered as a Git bundle, HCL_Commerce_Search_Bundle_9.1.x.x.zip. To get the latest version of the query-api.jar, review the list of the latest available download packages to ensure that you are obtaining the most up-to-date version.
  2. Open a web browser and log in to the HCL Software License & Delivery site to download and extract the latest version of the HCL Commerce Search bundle to obtain the HCL_Commerce_Search_Bundle_9.1.x.x.zip.
  3. Create a new Gradle project. Extract the file query-api-9.1.x.x.jar into a /lib directory at the root of the project. Create this directory if it does not already exist.
  4. Add the following dependencies to the build.gradle file:
    implementation 'org.elasticsearch:elasticsearch:7.9.3'
        implementation 'org.springframework.boot:spring-boot-starter-web:2.2.4.RELEASE'
        implementation 'org.springframework:spring-web:5.2.5.RELEASE'
        implementation 'org.springframework.boot:spring-boot-starter-validation:2.2.5.RELEASE'
        implementation 'org.elasticsearch.client:elasticsearch-rest-high-level-client:7.12.0'
        implementation 'org.elasticsearch:elasticsearch:7.12.0'
        implementation ('org.elasticsearch.client:elasticsearch-rest-client:7.12.0') {
    		exclude module: 'snakeyaml:1.23'
    	}
    implementation files('lib/query-api-9.1.x.x.jar')
    
    Replace 9.1.x.x with the correct version of the downloaded jar file.
  5. To create the custom handlers:
    1. Use the following spring boot annotations for the rest handler class:
      @RestController
      @RequestMapping
      
    2. You can make use of the query runtime environment if you want to implement a custom controller to create a new endpoint and fetch custom data from the index.
      1. Call the performSearch method from SearchServiceFacade provided as part of the query-api.jar. For an example, see the following code.
        package com.samplecompany.search.rest;
        
        import org.springframework.http.ResponseEntity;
        import org.springframework.web.bind.annotation.RequestMapping;
        import org.springframework.web.bind.annotation.RequestParam;
        import org.springframework.web.bind.annotation.RestController;
        
        import com.hcl.commerce.search.expression.SearchCriteria;
        import com.hcl.commerce.search.internal.runtime.SearchServiceFacade;
        
        @RestController
        @RequestMapping("/api/url")
        public class URLResource {
        	
        	@RequestMapping("/id")
        	private ResponseEntity findUrlsByIds(@RequestParam("storeId") Integer iStoreId,
        			@RequestParam("id") String id) throws Exception {
        		
        		ResponseEntity result = null; 
        		SearchCriteria searchCriteria = SearchCriteria.getCriteria();
        		//Set all the necessary control parameters in the searchCriteria. Below is just a sample about how to set parameters.
        		searchCriteria.setControlParameterValue("_wcf.search.profile","Mycompany_customProfile");
        		searchCriteria.setControlParameterValue("_wcf.search.term",id);
        		searchCriteria.setControlParameterValue("_wcf.search.language","-1");
        		result = SearchServiceFacade.getInstance().performSearch(searchCriteria);
        		return result;
        	
        	}
        
        }
        
      2. 2. If you do not need the query runtime architecture and want to query the elasticsearch index directly, you can do it using the java APIs provided by ElasticSearch. For example, see the following sample code snippet.
        package com.samplecompany.search.rest;
        
        import java.util.Arrays;
        import java.util.HashMap;
        import java.util.Map;
        
        import org.springframework.http.HttpStatus;
        import org.springframework.http.MediaType;
        import org.springframework.http.ResponseEntity;
        import org.springframework.web.bind.annotation.PathVariable;
        import org.springframework.web.bind.annotation.RequestMapping;
        import org.springframework.web.bind.annotation.RequestMethod;
        import org.springframework.web.bind.annotation.RestController;
        
        
        import org.apache.http.HttpHost;
        import org.elasticsearch.action.search.SearchRequest;
        import org.elasticsearch.action.search.SearchResponse;
        import org.elasticsearch.client.RequestOptions;
        import org.elasticsearch.client.RestClient;
        import org.elasticsearch.client.RestHighLevelClient;
        import org.elasticsearch.index.query.BoolQueryBuilder;
        import org.elasticsearch.index.query.QueryBuilders;
        import org.elasticsearch.index.query.QueryStringQueryBuilder;
        import org.elasticsearch.search.builder.SearchSourceBuilder;
        
        @RestController
        @RequestMapping("/store/{storeId}/attrview")
        public class AttributeResource {
        	
        	private static final String ES_SCHEME = System.getenv("ELASTICSEARCH_SCHEME");
        	private static final String ES_HOST = System.getenv("ELASTICSEARCH_HOST");
        	private static final String ES_PORT = System.getenv("ELASTICSEARCH_PORT");
        	private static final String ENV_TYPE = System.getenv("ENVTYPE");
        
        	private static final String BY_ATTR_ID = "/{attrId}";
        
        	@RequestMapping(value = BY_ATTR_ID, method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
        	public ResponseEntity findAttrById(@PathVariable("storeId") String storeId, @PathVariable("attrId") String attrId) throws Exception {
        
        		ResponseEntity result = null;
        		SearchResponse searchResponse = null;
        		
        		RestHighLevelClient elasticServer = new RestHighLevelClient(RestClient.builder
        				(new HttpHost(ES_HOST, Integer.valueOf(ES_PORT), ES_SCHEME)));
        		
        		String strIndexName = "auth" + "." + storeId + "." + "attribute";
        		SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        		
        		BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        		
        		QueryStringQueryBuilder queryStringStore = QueryBuilders.queryStringQuery(attrId.toString());
        		queryStringStore.field("id.attribute");
        		boolQuery.filter(queryStringStore);
        		
        		searchSourceBuilder.query(boolQuery);
        		searchSourceBuilder.fetchSource(false);
        		searchSourceBuilder.storedFields(Arrays.asList("*"));
        		searchSourceBuilder.size(1000);
        		
        		SearchRequest request = new SearchRequest(strIndexName);
        		// Set search source builder in request
        		request.source(searchSourceBuilder);
        		
        		searchResponse = elasticServer.search(request, RequestOptions.DEFAULT);
        		
        		searchResponse.getHits().getAt(0).getId();
        		
        		Map resultMap = new HashMap();
        		
        		resultMap.put("id", searchResponse.getHits().getAt(0).getId());
        		
        		result = new ResponseEntity(resultMap, HttpStatus.OK);
        
        		return result;
        	}
        }
        
    3. For component scanning, declare your custom handler package com.samplecompany.search.rest against key scan.packages as comma separated values in the application.properties located at: /opt/WebSphere/Liberty/usr/servers/default/apps/search-query.ear/search-query.war/WEB-INF/classes inside the query-service container.
  6. To add custom expression providers:
    Expression Providers are the java classes that take SearchCriteria as a parameter. SearchCriteria is a java object which encapsulates the criteria attributes sent by the storefront, and some internal attributes required for generating Elasticsearch’s SearchSourceBuilder. SearchCriteria is the Java version of the search expression. Providers read and process the criteria attributes from it and add certain internal attributes which are further utilized by pre-processors.
    1. Create a new class that extends the AbstractSearchExpressionProvider class and implements the SearchExpressionProvider interface from query-API dependency.
      public class SearchByCustomProvider extends AbstractSearchExpressionProvider
      	implements SearchExpressionProvider {
      
      	private static final String CLASSNAME = SearchByCustomProvider.class.getName();
      	private static final Logger LOGGER = LoggerFactory.getLogger(CLASSNAME);
      
      	@Override
      	public void invoke(SearchCriteria searchCriteria) throws Exception {
      		// your logic here
      	}	
      }
      
    2. Declare your custom provider’s class name in the provider section of the relevant zookeeper profile using the Search-Profile-Resource endpoint of query-service.
  7. To add a custom pre-processor:
    Expression pre-processors are the java classes that take SearchCriteria and queryRequestObjects as parameters. queryRequestObjects is a java varargs (Variable Arguments) object type which contains Search Profiles and an empty SearchSourceBuilder instance. The SearchSourceBuilder instance is the native binary object to be used against the Search Engine (Elasticsearch). The pre-processor’s main responsibility is to prepare the SearchSourceBuilder object that can be used to query the Elasticsearch index.
    1. Create a new class that extends the AbstractSearchQueryPreprocessor class and implements the SearchQueryPreprocessor interface from query-API dependency. Refer to the following code snippet:
      public class SearchCustomQueryPreprocessor extends AbstractSearchQueryPreprocessor 
      	implements SearchQueryPreprocessor {
      
      	private static final String CLASSNAME = SearchCustomQueryPreprocessor.class.getName();
      	private static final Logger LOGGER = LoggerFactory.getLogger(CLASSNAME);
      
      	@Override
      	public void invoke(SearchCriteria searchCriteria, Object... queryRequestObjects) throws Exception {
      		// your logic here
      	}	
      }
      
    2. Declare the name of your custom pre-processor class in the pre-processor section of the relevant zookeeper profile using the Search-Profile-Resource endpoint of query-service.
  8. To add a custom post-processor:
    Expression post-processors are the java classes that take SearchCriteria and queryResponseObjects as parameters. queryResponseObjects is a java varargs (Variable Arguments) object type which contains an object of datatype SearchResponse. SearchResponse is a native response object which represents the search response from Elasticsearch. The post-processor’s main responsibility is to process the results returned from Elasticsearch and transform them into the format required by the storefront.
    1. Create a new class that extends the AbstractSearchQueryPostprocessor class and implements SearchQueryPostprocessor from query-API dependency. Refer to the following code snippet:
      public class SearchCustomQueryPostprocessor extends AbstractSearchQueryPostprocessor 
      implements SearchQueryPostprocessor {
      
      	private static final String CLASSNAME = SearchCustomQueryPostprocessor.class.getName();
      	private static final Logger LOGGER = LoggerFactory.getLogger(CLASSNAME);
      
      	@Override
      	public void invoke(SearchCriteria searchCriteria, Object... queryResponseObjects) throws Exception {
      		// your logic here
      	}	
      }
      
      
    2. Declare the name of your custom post-processor class in the post-processor section of the relevant zookeeper profile using the Search-Profile-Resource endpoint of query-service.
  9. Build the project at the root of the project, using the gradlew build command. The built jars can be found in the /build/libs directory, under the root of the project. Copy the new jar files to the designated extension directory (/opt/WebSphere/Liberty/usr/servers/default/apps/search-query.ear/search-query.war/WEB-INF/lib) in your host machine. This directory is mounted as a volume on your query-service docker container.
    Note:
    • Refer to the following command as an example to provide the extension jar as a volume mount to the query-service docker container:
      docker run -it -p 3737:3737 --name query-service -v /home/qsuser/ext/extension.jar:/opt/WebSphere/Liberty/usr/servers/default/apps/search-query.ear/search-query.war/WEB-INF/lib/extension.jar
    • Although the aforementioned step suggests using external mounts for storing your customizations, this is only a recommended set up in an agile development environment where frequent code changes are made and being retested over and over again. But when it gets to higher deployment environments such as QA, and Production environments, it is recommended to have these customization built into the application image through a CI/CD pipeline for consistency and ease of deployment.
  10. Restart the query-service container for the changes to take effect.