Migrating WebSphere Commerce search customization assets

You can migrate WebSphere Commerce search customization assets to the REST-based search architecture. This topic provides a high-level overview of the assets you must change, and the steps required for updating several of the WebSphere Commerce search extension points.

There are key architectural differences between the search components under the WebSphere Commerce server, and under the WebSphere Commerce search server.

The WebSphere Commerce server contains the following characteristics:

Its architecture is based on the BOD programming model, using predefined nouns such as the CatalogEntry noun.
It contains other architectures such as SOA, SOI, EJBs, EMFs, and SDOs.

The WebSphere Commerce search server contains the following characteristics:

Its architecture is based on the REST programming model, using native Java objects.
The storefront uses getData tags to call WebSphere Commerce services that return BOD-structured data. RESTful APIs are used to call the WebSphere Commerce search server services that return simpler name-value formatted data.

The migration effort and complexity varies depending on which component is customized, and the dependency levels of the customized assets on WebSphere Commerce resources.

Topic overview

This topic contains several sections of interest that are related to migrating WebSphere Commerce search customization assets.

The following list helps summarize the key areas of interest:

Custom index fields and indexing scripts
Custom configuration files
Custom Java classes
Storefront services and response data format
#csdsearchmigratecustassets__DeployingTheCustomizations

Custom index fields and indexing scripts

Any customization made to any of the index schemas, or to the preprocessing and indexing scripts can be reapplied. The search index schema and indexing scripts are shared between the WebSphere Commerce server and the WebSphere Commerce search server, and are therefore compatible with no additional changes required.

Custom configuration files

Search-related configuration updates under the WebSphere Commerce EAR application extension directories can be moved into the corresponding extension directories under the WebSphere Commerce search EAR application. For example, the such as the com.ibm.commerce.catalog-ext and com.ibm.commerce.foundation-ext directories.

Note: You must restart the WebSphere Commerce server after making any changes to the search configuration files.

Registering custom search profiles defined in the wc-search.xml file

Any custom search profile defined in the WebSphere Commerce server must be redefined in the WebSphere Commerce search server extension directory.

Any custom search profile which extends a default search profile in the WebSphere Commerce server must be updated, as new default search profiles are introduced in the search server that contain different names or naming conventions.

Any custom search profile that uses any of the default search query providers, processors, and search result filters must be updated, as new alternatives are introduced in the search server that contain different names or naming conventions.

Custom search profiles must be associated to a REST service:

Locate the Search-Rest\WEB-INF\config\com.ibm.commerce.rest\wc-rest-resourceconfig.xml file.
Identify which REST service the custom search profile should be listed under.
Append your custom search profile to the end of the defined list of search profiles.

For more information about the search profiles mappings, see Mapping between WebSphere Commerce search profiles.

For more information about the search configuration properties in the wc-search.xml file, see WebSphere Commerce search configuration file (wc-search.xml) (Search EAR).

Custom configurations made in the wc-component.xml file

Most of the search-related custom properties defined in the wc-component.xml file can be reused in the search server, with the exception of the global price mode property. The price mode configuration is instead performed using the STORECONF table.

For more information about the search configuration properties in the STORECONF table, see Search configuration properties in the STORECONF table.

For more information about the search configuration properties in the wc-component.xml file, see Search properties in the component configuration file (wc-component.xml) (Search EAR).

Custom object mediators made in the wc-business-object-mediator.xml file

The search server does not support business object mediators. Therefore, any customizations performed under the wc-business-object-mediator.xml file must be moved to the wc-component.xml file. For example, mappings between a userData custom field to an internal index field or database field must instead use the correct mappings under the wc-component.xml file.

The following example shows how an existing userData mapping under the WebSphere Commerce server wc-business-object-mediator.xml file can be moved into the search server custom wc-component.xml file:


<_config:mediator-property name="CatalogEntryView/UserData[(Name='SKU')]" value="partNumber_ntk" />

In the search server, the mapping between internal and external names is done using the valuemappingservice defined in the wc-component.xml file. There are different maps for CatalogEntry UserData and CatalogGroup UserData. The corresponding mapping in the search server wc-component.xml file can be achieved as:


<_config:valuemapping externalName="CatalogEntryUserDataFieldNameMapping" internalName="CatalogEntryUserDataFieldNameMapping">
   <_config:valuemap externalValue="SKU" internalValue="partNumber_ntk" />
</_config:valuemapping>

The following is the CatalogGroup UserData mapping service:


<_config:valuemapping externalName="CatalogGroupUserDataFieldNameMapping" internalName="CatalogGroupUserDataFieldNameMapping">
</_config:valuemapping>

Where the mappings are being populated by the following postprocessors:


<_config:postprocessor classname="com.ibm.commerce.foundation.server.services.rest.search.postprocessor.solr.SolrRESTSearchCatalogEntryViewUserDataQueryPostprocessor" />

<_config:postprocessor classname="com.ibm.commerce.foundation.server.services.rest.search.postprocessor.solr.SolrRESTSearchCatalogGroupViewUserDataQueryPostprocessor" />

Note: Ensure that the required postprocessors are included in the search profile you are using.

Custom database query template (.tpl) files

The search server supports DSL. However, it does not support EMFs, SDOs and logical schemas. Therefore, any retrieved data from the database must be parsed by custom code and added into the main response as applicable. Any search related custom queries can be reused in the search server. For more information, see Creating a custom query postprocessor.

The search server does not support any Query template tags. That is, each of the query parameters must be passed into the query services as query parameters. For schema parameters, create two versions of the query; one as the original query, and another where the table names are prefixed by the schema name. For example:


BEGIN_SQL_STATEMENT
	name=X_GET_CUSTOM_FIELD_QUERY
	base_table=tableName
	sql=SELECT *
	FROM tableName
	WHERE CATALOG_ID=?catalogId? AND LANGUAGE_ID=?languageId? AND STOREENT_ID=?storeentId?
END_SQL_STATEMENT

BEGIN_SQL_STATEMENT
	name=X_GET_CUSTOM_FIELD_QUERY_WORKSPACE
base_table=tableName
	sql=SELECT *
	FROM $SCHEMA$.tableName
	WHERE CATALOG_ID=?catalogId? AND LANGUAGE_ID=?languageId? AND STOREENT_ID=?storeentId?
END_SQL_STATEMENT

At runtime, use the following helper method to get the schema name, and therefore call the correct query template:


queryParameters.put("languageId", Arrays.asList("-1"));
queryParameters.put("catalogId", Arrays.asList("10001"));
queryParameters.put("storeentId", Arrays.asList("10051"));

String readSchema = SolrSearchConfigurationRegistry.getInstance(
"com.ibm.commerce.catalog").getReadSchema();
if (readSchema != null && readSchema.length() > 0) {
queryParameters.put("$SCHEMA$",Arrays.asList(readSchema));
results = service.executeQuery("X_GET_CUSTOM_FIELD_QUERY_WORKSPACE", queryParameters);
} else {
results = service.executeQuery("X_GET_CUSTOM_FIELD_QUERY", queryParameters);
}

Custom Java classes

It is recommended to first evaluate all the search-related custom Java classes under the development environment before deploying them into the QA environment. Areas to focus on when evaluating the search-related custom Java classes are compilation errors. Ensure that none of the import statements reference an API that does not exist under the search EAR. Expand the search EAR to see a full list of available JAR files and packages. Then, evaluate the search-related custom code under the QA environment to uncover any runtime errors. For example, if there are JAR files and packages that exist in the development environment that do not exist in the QA and production environment.

Note: The WebSphereCommerceServerExtensionsLogic project is not configured under the search EAR application. Either configure it, or create your own utility project and add it to the search EAR application class path. For more information, see Creating a custom project for search-related customizations.

Custom expression providers

Most custom expression providers can be reused without any changes in the search server, as the search server supports Selection Criteria objects and control parameters.

Custom processors

All custom search query preprocessors can be reused in the search server, as search query preprocessors operate on the native physical SolrQuery object defined in the AbstractSolrSearchQueryPreprocessor parent class that all preprocessors extend from, which is also available in the search server.

Search query result postprocessors might require re-implementation. If your custom code operates on the native QueryResponse object, the custom code is reusable. However, if the custom postprocessor operates on the SolrCatalogNavigationViewImpl, it cannot be use reused.

Alternatively, change your custom code to operate on the SearchResponse. The following snippet is an example on how to use the SearchResponse:


List<Map<String, Object>> catalogEntryViews = (LinkedList<Map<String, Object>>)iSearchResponseObject.getResponse().get("external object name");

Where the object name is the external object name. Refer to the sample custom postprocessors in the wc-search.xml file for samples on resolving external names.

Custom search query result filters

Custom result filters operate on the logical CatalogNavigationViewType noun. This logical noun is not supported in the search server. All custom result filters must be re-implemented using search query postprocessors instead.

The main key difference in result filters is that the catalogEntryViewList was a list of CatalogEntryViewType objects, but in the search query postprocessor, it is a list of Map.

For example, the following code exists in the search query result filters:


CatalogNavigationViewType catalogNavigationView = (CatalogNavigationViewType) dataObject;

List<CatalogEntryViewType> catalogEntryViewList = catalogNavigationView
				.getCatalogEntryView();

The following is an alternative code on a search query postprocessor:


List<Map<String, Object>> catalogEntryViews = (LinkedList<Map<String, Object>>)iSearchResponseObject.getResponse().get("external object name");

Where the external object name is the external object name of the catalogEntryView.

Business Object Mediators

Similar to search query result filters, mediators also operate on the logical CatalogNavigationViewType noun. Mediators are not supported in the search server. An alternative is search query postprocessors, as they are used instead. All custom mediators that extend the AbstractReadBusinessObjectPartMediatorImpl parent class must be re-implemented using search query postprocessors instead.

Storefront services and response data format

Any custom storefront pages that use any of the search related CatalogNavigationView BOD services must be updated to use the corresponding new search server REST services.

For example, the following is a getData BOD search service which is used to populate a catalog entry recommendations widget:


<wcf:getData type="com.ibm.commerce.catalog.facade.datatypes.CatalogNavigationViewType" var="catalogNavigationView" 
expressionBuilder="getCatalogEntryViewDetailsByID" scope="request" varShowVerb="showCatalogNavigationView" 
maxItems="100" recordSetStartNumber="0" scope="request">
	<c:forEach var="marketingSpotData" items="${marketingSpotDatas.baseMarketingSpotActivityData}">
		<c:if test='${marketingSpotData.dataType eq "CatalogEntryId"}'>
			<wcf:param name="UniqueID" value="${marketingSpotData.uniqueID}"/>
		</c:if>
	</c:forEach>
	<wcf:contextData name="storeId" data="${storeId}" />
	<wcf:contextData name="catalogId" data="${catalogId}" />
	<wcf:param name="searchProfile" value="IBM_findCatalogEntryDetails_PriceMode"/>
</wcf:getData>
<c:set var="eSpotCatalogIdResults" value="${catalogNavigationView.catalogEntryView}"/>

The following is the equivalent REST-based service:


<wcf:rest var="catalogNavigationView" url="${searchHostNamePath}${searchContextPath}/store/${WCParam.storeId}/productview/byIds" >
<c:forEach var="id" items="${catentryIdQueryList}">
	<wcf:param name="id" value="${id}"/>
</c:forEach>
<wcf:param name="langId" value="${langId}" />
<wcf:param name="currency" value="${env_currencyCode}" />
<wcf:param name="responseFormat" value="json" />
<wcf:param name="catalogId" value="${WCParam.catalogId}" />
<wcf:param name="profileName" value="IBM_findProductByIds_Summary" />
</wcf:rest>
<c:set var="eSpotCatalogIdResults" value="${catalogNavigationView.catalogEntryView}"/>

The response data is being formatted in a BOD-like dot notation response to minimize the changes required in the storefront. In some cases, the response has been simplified and flattened to simpler name-value pairs, rather than using internal maps to group certain fields.

You can examine the JSON response by printing the response object using the following code:


<wcf:json object="${catalogNavigationView}"/>

You can also invoke the REST service directly from the web browser or by using HTTPRequester or an equivalent client to validate you custom code:


http://localhost/search/resources/store/10152/productview/byId/10001?langId=-1&catalogId=10001&currency=USD&responseFormat=json&pageSize=12&pageNumber=1

For more information about the BOD to REST services mappings, see Mapping between WebSphere Commerce search BOD services and REST services.

For more information about the WebSphere Commerce REST API for WebSphere Commerce search, see the complete list of supported query parameters for the following search services: CategoryViewHandler, ProductViewHandler, and SiteContentHandler.