Customizing the Elasticsearch-based search solution

The topics in this section describe tasks performed by an application developer to customize the Elasticsearch-based search solution. You can extend the Ingest Service and Query Service to meet your business requirements.

These topics cover customization of the Search solution. Administrators working with and tuning the service on an ongoing basis can learn about the concepts and tools for this in covwcsearch.html.

Basic architecture

The Elasticsearch Search solution is one component of a unified data environment. This environment is designed as a generic data-management system. The environment includes several major components, including the gateway, data services, and the ZooKeeper configuration management system.

The Redis gateway and ZooKeeper/Registry support the data services. It is the data services that control Search behavior, so you customize Search by extending the Ingest data-uploading subsystem or the Elasticsearch-based Query subsystem. There are four main ways you can do this, as outlined below. In addition, you can develop migration and continuous-delivery strategies based on templates and examples provided by HCL.

Note: All SQL in the Version 9.1.15 HCL Commerce Search Ingest system have been moved to the GenerateSQL processor. If you want to add custom SQL to any processor, place the customized SQL in the CustomSQL box in the NiFi canvas, and change the value of Use Custom Variable to true.

Table 1. Customization topicsHCL Commerce Search documentation
Concept	Details and audience	For more information
Introduction	Learn about the customization points in HCL Commerce Search and how they can be used in your implementation: The Ingest service and NiFi pipelines Elasticsearch and its highly scalable indexing system The Query service, which include Natural Language Processing and multi-language capabilities. Migration tools to modernize your Solr customizations. These topics are for HCL Commerce developers who are learning to customize HCL Commerce Search.	Review the #C_CustomizeElasticsearch__section_z1s_yf5_szb of the integrated information system. Familiarize yourself with the csdelasticsearchlifecycle.html. To learn which indexing model is best for your implementation, see csdassetstoreindexes.html. The key concepts you need to extend the Ingest service are described in csdelasticsearch.html. Read ../tasks/t_customizeQueryService.html before extending search-related functions such as Natural Language Processing (NLP) and text relevancy. You can use the Query APIs to add your own services or extend existing ones.
Ingest service	For an overview of how HCL Commerce Search brings in data for indexing, see csdsearchconnectors.html.	Customizing Ingest typically involves the creation of new NiFi pipelines, which consist of connectors, processors, and process groups. To learn how to create new NiFi connectors, see ../tasks/tsdconnector_create.html. Read C_CustomNiFiProcessGroup.html and ../tasks/t_customizeNiFi.html to understand process groups and how to create them. To optimize Ingest performance, see ES_Ingesttuning.html. You can find a reference for optimal performance settings in ../refs/ES_configparameters.html. For a hands-on introduction, use the Elasticsearch file ingestion tutorial.
Search Query API	HCL Commerce Search uses a REST API by default to communicate with its microservices. This API is useful for developers who want to extend existing browsing, NLP, and site content features. This API is managed from a root node in ZooKeeper with the name `/configuration`. You can add new extensions in child nodes under the root. For example, suppose you need to show products of a particular color and allow search based on color. You can extend the API with a custom endpoint for product search and color as parameter.	If you want to use existing Query service features or an existing index, you can extend the existing Query API by calling the performSearch method from the SearchServiceFacade service provided as part of the query-api.jar. If you need a new endpoint unique to your requirements, you can create it using it JAVA APIs provided by Elasticsearch. For a detailed description of the Query API, see ../refs/rsdQuerySpec.html. Its formal definition is in the Query Service REST API section.
Search profiles	Use search profiles to control the storefront search experience at a page-level. Profiles group together sets of search runtime parameters. You can call default profiles or your own customized ones to control the behavior of the Query service.	Locations for the default profiles, how to load them, and examples of how to use them can be found in ../refs/ES_SearchProfiles.html. Refer to ../refs/zkp_customsearchprofile.html when creating your own profiles.
Natural Language Processing profiles	Natural Language Processor (NLP) profiles are used to control the preprocessing flow of search queries. You can find the default HCL_NLPProfile.json file in the resources\profiles\nlp directory of query runtime. A NLP profile can also be created through the /profiles REST endpoint and is stored inside of the Zookeeper “nlpprofiles” node.	To understand the basics of customizing NLP profiles, see ../refs/ES_NLP_profile_customization.html. You can explore more detail about the configuration process and learn how to customize them in ../tasks/tsdprofiles_nlp.html. Understand and improve csdquery_relevance.html.
Tutorials	Learn about HCL Commerce Search by completing tutorials.	You can find more tutorials for the Ingest, Query, and indexing services in the ../../tutorials/concepts/csdelasticsearch.html section.
Reference	Core concepts for administering HCL Commerce Search can be found in the covwcsearch.html section of the Help Center.	Learn about day to day tasks in Administering HCL Commerce search Ensure that your system is current by csdESreleasechanges.html. HCL Commerce provides a strategy for migrating your Solr customizations to an Elasticsearch-specific CI/CD pipeline. See csdESmigration.html. For a guide to implementing the pipeline, see C_CICDModel.html. If you encounter problems while you use HCL Commerce Search, refer to the../refs/rsdsearchtroubleshooting.html section of the Troubleshooting guide.