WebSphere Commerce search interactions

WebSphere Commerce search includes several key service areas, including the compatibility paths, server architecture and programming model.

Important: If you need to continue to receive support for WebSphere Application Server, you must upgrade to WebSphere Application Server to version 8.5.5 with Java 8. Earlier versions of WebSphere Application Server are no longer supported. You can only update to use IBM SDK, Java Technology Edition Version 8 if your existing WebSphere Commerce runtime instance is on WebSphere Commerce version 7 Feature Pack 6, 7, or 8, and has Fix Pack 9 installed. For more information, see Support for JDK 8, WebSphere Application Server V8.5.5.5, and Rational Application Developer V9.5.0.2

Search compatibility paths

The following diagram illustrates the two search runtime access paths that exist:

Where:

Only one search index works with storefronts from previous feature packs (Feature Pack 5 and 6) and the current storefront (Feature Pack 7 or later).
A template configuration maps to the BOD-based dot notation on the JSP files to east the storefront JSP file migration process.
Both runtime access paths can coexist in Feature Pack 7 or later, so that all stores from Feature Pack 5 and above can be deployed to the same WebSphere Commerce instance and be available at the same time.
Each access path has its own search configurations, for example, the wc-search.xml file, and therefore allow independent search runtime behavior. That is, for example, the Feature Pack 6 version of the wc-search.xml file still exists on the WebSphere Commerce server, and is responsible for handling BOD-based CatalogNavigationView requests, while newer Feature Pack versions of the file can only be found on the remote search server that interacts with the REST-based search services.
For more information about the BOD-based search programming model, see WebSphere Commerce search interactions.

Search server architecture

A direct access path to site navigation and the search server exists on the Solr search server. This server consists of a set of REST services, a search runtime framework that reuses the current search programming model, and a set of WebSphere Commerce foundation services that also provide access to the WebSphere Commerce database.

The following diagram shows an overview of the search server architecture:
Search interaction diagram

Where:

The REST service mapper is an Apache Wink application that uses configurations to map a REST URL to a resource handler. This handler then calls the embedded search runtime.
The search runtime is a pluggable programming framework that allows search expressions to be run using properties that are defined in a configuration-based search profile. Once a search expression is composed, the runtime then forwards the request to the embedded Solr runtime for processing.
The foundation service is the WebSphere Commerce core runtime that can provide basic server services, such as the JDBC Query Service to access the WebSphere Commerce database.

Search programming model

The search runtime framework is mainly driven by a programming pattern that is very similar to a factory assembly line, where each business component can contribute its own piece of the search expression into the main query, which is then run against the Solr search engine.

The following diagram illustrates the BOD-based programming model being used on the WebSphere Commerce server, where the storefront accesses WebSphere Commerce to process the search request using a BOD-based message:
Search programming model

In comparison a similar, simplified search runtime programming model is introduced in Feature Pack 7 Feature Pack 8

or later and is used on the search server. Instead of using BOD and performing object-based mediation, the search programming model does not rely on any SDO, and instead uses POJO and raw data returned from the search server to perform simple name-value-pair mapping:

This simplified search runtime also contains two main components: the search expression provider and the search expression processor:

Search Expression Provider

Depending on the nature of the request, that is, the given search profile, other business components might get involved, such as Marketing for search-based merchandising rules, or contracts for entitlement. Each business component is responsible for contributing a portion of the search expression, which gets combined with the main search expression generated by the Search REST services.

As a result, the consolidated search expression is run by the search processor. A factory class, SolrRESTSearchExpressionProvider, manages all of these expression providers at runtime. The implementation of this search expression provider factory class is defined in the SearchExpressionProviderFactory.properties file on the search server.

The default SolrRESTSearchExpressionProvider performs the following high-level steps in this given order:

Validates that a search profile is given by calling SolrSearchProfileNameValidator and immediately stops if none is provided.
Validates the corresponding index name by calling SolrSearchIndexNameValidator and immediately stops if invalid.
Validates the corresponding workspace information by calling SolrSearchWorkSpaceValidator and sets the information to let the processor know.
Validates the search expression to ensure the query expression is not empty and does not contain special characters by calling SolrSearchExpressionValidator.
Launches a list of search query expression providers defined in the given search profile where each provider contributes a portion of the search expression.

Search Expression Processor

A search processor is the central processing unit for integrating with the search engine. Its responsibility is to run the given Solr expression, based on the given search profile attributes, and capture the response from the search engine.

The SolrRESTSearchExpressionProcessor is a default implementation of the expression processor that is defined in the SearchExpressionProcessorFactory.properties file on the search server.

The default SolrRESTSearchExpressionProcessor composes the final Solr expression and sets up all the required bootstrap settings for running in the Solr search runtime. The result set is then reformatted into a JSON object to be returned to the caller.

The expression processor performs the following high-level steps in the following sequence:

Injects a debug parameter option into the SolrQuery object by calling SolrSearchDebugQueryPreprocessor.
Generates a list of index fields to be included in the search result set by calling SolrSearchResultFieldQueryPreprocessor.
Includes a relevancy score and additional tracing information into the SolrQuery object by calling SolrSearchPreviewQueryPreprocessor.
Enables spell correction and includes its associated parameter options into the SolrQuery object by calling SolrSearchSpellCorrectionQueryPreprocessor.
Enables highlighter and includes its associated parameter options into the SolrQuery object by calling SolrSearchHighlighterQueryPreprocessor.
Injects pagination parameter options into the SolrQuery object by calling SolrSearchPaginationQueryPreprocessor.
Injects sorting parameter options into the SolrQuery object by calling SolrSearchSortingQueryPreprocessor.
Composes a list of facet fields, along with the proper settings, to be included in the search result set by calling SolrSearchFacetQueryPreprocessor.
Injects the dismax query related expression by calling SolrSearchEDismaxQueryPreProcessor as eDismax handler must occur before the main query because the boost query control parameter might need to be removed.
Injects the response format parameter option by calling SolrSearchResponseFormatQueryPreprocessor.
Injects the main search query string into the SolrQuery object by calling SolrSearchMainQueryPreprocessor.
Injects additional custom SolrQuery parameters into the final Solr query expression by calling SolrSearchCustomQueryPreprocessor.
Launches a list of search query preprocessors defined in the given search profile to allow modifications to the SolrQuery object right before sending it to Solr for processing.
Issues a request against Solr to process the SolrQuery object.
Launches a list of search query postprocessors defined in the given search profile to allow modifications to the physical data object, SearchResponse, right after the QueryResponse is returned from the Solr server.

A query preprocessor is used to modify the SolrQuery object right before sending it to the Solr server for processing.

A query postprocessor is used to modify the QueryResponse object that is returned from the Solr server.