Tutorial: Customizing Catalog Asset Store connectors with Ingest Profile

This tutorial demonstrates how to use Ingest profiles with your Catalog Asset Store (CAS) connectors to separate custom configurations and logic code from the product release logic. This logical separation makes it easier to upgrade to future product releases.

In the default development environment, the search profile indexes the preset attribute seller for Marketplace stores. The seller is the vendor who sells the products on the marketplace store.

In this tutorial, you learn how to index an additional attribute seller description for the marketplace store by extending the default SQL in a custom Ingest processor. You will write custom Ingest processor code in Java and build and unit test this custom Ingest processor logic exclusively in a NiFi toolkit, to extend the default connectors. Overriding the default query search profile allows you to query on the new attribute, which is indexed using the custom Ingest logic in the response fields.

Skill level

This tutorial is intended for advanced HCL Commerce developers responsible for creating and updating customizations for HCL Commerce Search. The steps are mainly based on Customizing Ingest profiles.

Familiarity with the following terms and concepts is recommended:

HCL Commerce Search. For an introduction, see Using the HCL Commerce Search service and Customizing the Elasticsearch-based search solution.
The Catalog Asset Store (CAS) indexing model, as described in Choosing your index model.
HCL Commerce services. For more information, see Working with HCL Commerce SOI services.
The Ingest service. For an introduction, see Ingest Service, and for a customization guide, see Extending the Ingest service.
Ingest profiles .
Apache NiFi.
Programming in Java.
Apache Maven.
Structured Query Language (SQL).

Time required

Expect this tutorial to take three hours to complete. The tutorial takes longer if you explore concepts that are related to this tutorial.

Prerequisites

Perform the following steps to prepare your system for the tutorial.

Ensure that you have an HCL Commerce Developer environment configured for the Elasticsearch data platform. For more information, see Setting up the HCL Commerce Developer Search environment.
Download and extract HCL_Commerce_Search_Bundle_9.1.x.x.zip from the HCL License and Delivery portal. After you unzip the file, locate the file commerce-custom-search-marketplace-seller.zip in the extracted directory \dependencies\nifi\Tutorials.
Extract commerce-custom-search-marketplace-seller.zip into a temporary directory in your development environment. Use this to set up the NiFi toolkit, as instructed later in this tutorial. This file includes the sample Java code and Junit tests that you use in the tutorial.
Enable the Marketplace preview feature in the HCL Commerce Developer Tooling environment. For more information, see Enabling Marketplace in the HCL Commerce Developer Legacy environment.
Enable the Marketplace feature using the following Query service configuration endpoint.
```
PATCH http://QUERY_HOST:QUERY_PORT/search/resources/api/v2/configuration?nodeName=ingest&envType=auth 
```
with this request body:
```
{ 
	"global": { 
		"connector": [ 
			{ 
				"name": "attribute", 
				"property": [ 
					{ 
						"name": "flow.marketplace", 
						"value": "true" 
					} 
				] 
			} 
		] 
	} 
}  
```
Note: All APIs mentioned throughout the tutorial must be executed by adding basic authentication for the spiuser user, where the user is spiuser, and the password is
```
QxV7uCk6RRiwvPVaa4wdD78jaHi2za8ssjneNMdu3vgqi  
```
Ensure CAS Index Model is enabled by running an SQL query such as:
```
SELECT * FROM STORECONF WHERE name = 'wc.search.CASIndexModel';
```
You should receive a true response.
If it is not true run the following update query to update it:
```
UPDATE STORECONF SET VALUE= 'true' WHERE name = 'wc.search.CASIndexModel';
```
Trigger a full index for the EmeraldMP store (storeId=12001) using the following Ingest endpoint.
```
POST http://INGEST_HOST:30800/connectors/auth.reindex/run?storeId=12001 
```
Once the build is complete and the NIFI console shows 0/0 bytes queued data, run the following GET commands using a REST client.
1. Check the response for the default search profile HCL_V2_findProductByPartNumber_Details, by running the following GET call.
```
GET http://QUERY_HOST:30920/search/resources/api/v2/products?storeId=31&seller=7000000000000007002&partNumber=MPHBA031_310301 
```
2. Check the response for the default search profile HCL_findCatalogEntrySKUs by running the following GET call.
```
GET http://QUERY_HOST:30920/search/resources/api/v2/products?storeId=31&seller=7000000000000007002&id=14503 
```
The response contains the seller attribute. At the end of the tutorial, an additional attribute, seller_description appears in the response for the above GET calls.
Prepare the development toolkit for NiFi following the instructions in Custom NiFi processors. The NiFi toolkit must be configured with Eclipse set to use the Java 1.8 compiler, with Maven Version 3.8.4 or above. Set the Java build path for the imported project to use JavaSE-1.8 libraries.
Patch the Nifi container on the data platform and obtain the latest JAR files from the running patched container before you configure and build tutorial assets in the NiFi toolkit. As an example, for the Version 9.1.12 environment the JAR files are:
1. commerce-search-processors-9.1.12.0.jar
2. hcl-cache-core-9.1.12.0-20221201.221643-25.jar
3. cf-base-9.1.12.0-20221123.064414-9.jar
Initiate a Maven Build... on the root bundle project. In the Project Explorer, right-click on the root (commerce-search-custom-bundle-9.1.x.x) and select Run As > Maven build.... The Edit Configuration window opens.
Enter the following command nto the Goals entry field:
```
clean install -U -Denforcer.skip=true
```
Click the Run button.
Once the build is successful, the JAR files can be viewed in the target subdirectory of the commerce-custom-search-processors project.