Unstructured content in the storefront

Searching for unstructured content requires two queries since unstructured content is indexed in a different core. One query gets the related IDs by searching the unstructure field of the structured content, while the other query searches the unstructured index using the keywords and generated IDs scope from the first search.

Two queries are used because the result list must contain information from both the structured content index and the unstructured content index. Since both of the results have their own sorting and paging parameters, they are arranged in the following order:

Structured content search results.
Unstructured content search results.

The search process invokes the SolrJ API to post the query string to the request handler. There are two invocations involved:

The first query posts keyword and search scopes to the product index core, to retrieve the catalog entry ID list. The search scope additionally includes the unstructure field.
The second query posts keyword and the return ID list of the first query to the unstructured content index core, to retrieve the highlighted result, and catalog entry ID.

Search profiles for unstructured content

Searching for unstructured content at runtime follows a customized version of the structured content search. Based on the current search service facade, profiles are created for unstructured content search.

The following diagram illustrates the relationship between the storefront and the Data Service Layer:

Storefront and DSL relationship.

Where, for example, a sample search profile for unstructured content resembles the following snippet:


<_config:profile name="IBM_Global_Unstructured" indexName="UnstructuredContent">
  <_config:query>
       <_config:param name="maxRows" value="20"/>
	<_config:param name="maxTimeAllowed" value="3000"/>
	<_config:param name="debug" value="false"/>
	<_config:provider 
         classname="com.ibm.commerce.catalog.facade.server.services.search.expression.solr.SolrSearchProfileNameValidator"/>
	<_config:provider 
         classname="com.ibm.commerce.catalog.facade.server.services.search.expression.solr.SolrSearchIndexNameValidator"/>
	<_config:provider 
         classname="com.ibm.commerce.catalog.facade.server.services.search.expression.solr.SolrSearchByKeywordExpressionProvider"/>
	<_config:provider 
         classname="com.ibm.commerce.catalog.facade.server.services.search.expression.solr.SolrSearchByCustomExpressionProvider"/>
	<_config:provider classname="com.ibm.commerce.catalog.facade.server.services.search.expression.solr.SolrSearchMetaTokenExpressionProvider"/>
  </_config:query>
  <_config:sort />
  <_config:result>
	<_config:field name="attachment_id"/>
	<_config:field name="catentry_id"/>
	<_config:field name="path"/>
	<_config:field name="name"/>
	<_config:field name="mimetype"/>
	<_config:field name="identifier"/>
	<_config:field name="image"/>
	<_config:field name="shortdesc"/>
	<_config:field name="longdesc"/>
   </_config:result>
   <_config:highlight simplePre="</em&gt;</b&gt;"
     simplePost="</em&gt;</b&gt;">
   </_config:highlight>
   <_config:facet />
   <_config:spellcheck>
	<_config:param name="limit" value="5"/>
   </_config:spellcheck>
   <_config:mapping />
</_config:profile>
<_config:profile name="IBM_findAttachmentByContent" extends="IBM_Global_Unstructured">
<_config:query inherits="true">
	<_config:field name="tika_content"/>
</_config:query>
<_config:highlight inherits="true">
	<_config:field name="tika_content" />
</_config:highlight>
</_config:profile>

As a result, the unstructured content search request for the expression builder resembles the following snippet:


{_wcf.ap='IBM_Store_CatalogEntrySearch';
_wcf.search.profile='IBM_findAttachmentByCatentryId';
_wcf.search.expr='catentry_id(<CATENTRY_IDs >)';
_wcf.search.facet='';
_wcf.search.sort='';
_wcf.search.meta=''}/CatalogNavigationView

Search noun part mediators to assemble the unstructured content into BOD

When the product index search finishes, another query is sent to the unstructured content index core. With the return results, the search process command composes the unstructured content-related noun part and assembles the unstructured content noun part to the structured content noun. This second query invocation occurs in the InsertMoreCommand logic. For product and product attachment searches, the logic occurs in InsertMoreCatalogNavigationViewDataCmdImpl.

Unstructured content search results returned from the WebSphere Commerce search server through SolrJ client are in document form. A mediator is registered to transform the returned document into the related noun part.

For example, a sample mediator registration resembles the following snippet:


<_config:part-mediator-implementation 
className="com.ibm.commerce.catalog.facade.server.services.dataaccess.bom.mediator.solr.SolrReadAttachmentAssetViewPartMediator" />

Note: There is no need for faceted searching of attachments for product search result. The AttachmentAssetViewType noun is used for the structure of the mediator.

Search results displayed in the storefront

The following enablement steps display product attachment search result in the storefront:


<expression-builder>
		<name>getCatalogNavigationAttachmentView</name>
		<data-type-name>CatalogNavigationView</data-type-name>
		<expression-template>{_wcf.ap='$accessProfile$';_wcf.search.profile='$searchProfile$';_wcf.search.term='$searchTerm$';_wcf.search.type='$searchType$';_wcf.search.exclude.term='$filterTerm$';_wcf.search.exclude.type='$filterType$';_wcf.search.manufacturer='$manufacturer$';_wcf.search.price.minimum='$minPrice$';_wcf.search.price.maximum='$maxPrice$';_wcf.search.facet='$facet$';_wcf.search.sort='$orderBy$';_wcf.search.meta='$metaData$'}/CatalogNavigationView</expression-template>
		<param>
			<name>accessProfile</name>
			<value>IBM_Store_CatalogEntrySearch</value>
		</param>
		<param>
			<name>searchType</name>
			<value>0</value>
		</param>
		<param>
			<name>searchProfile</name>
			<value>IBM_findCatalogEntryByUnstructureField</value>
		</param>
	</expression-builder>

In the storefront JSP file, point the expressionBuilder parameter to the newly created expression builder. For example:


<wcf:getData type="com.ibm.commerce.catalog.facade.datatypes.CatalogNavigationViewType" var="catalogNavigationView" 
	expressionBuilder="getCatalogNavigationAttachmentView" scope="request" varShowVerb="showCatalogNavigationView" 
	maxItems="${pageSize}" recordSetStartNumber="${WCParam.beginIndex}" scope="request">
<wcf:param name="searchProfile" value="${searchProfile}" />
	<wcf:param name="searchTerm" value="${WCParam.searchTerm}" />
	<wcf:param name="searchType" value="${searchType}" />
	<wcf:param name="metaData" value="${WCParam.metaData}" />
	<wcf:param name="orderBy" value="${WCParam.orderBy}" />
	<wcf:param name="facet" value="${WCParam.facet}" />
	<wcf:param name="filterTerm" value="${WCParam.filterTerm}" />
	<wcf:param name="filterType" value="${WCParam.filterType}" />
	<wcf:param name="manufacturer" value="${WCParam.manufacturer}" />
	<wcf:param name="minPrice" value="${WCParam.minPrice}" />
	<wcf:param name="maxPrice" value="${WCParam.maxPrice}" />
	<wcf:contextData name="storeId" data="${WCParam.storeId}" />
</wcf:getData>

In the store page, parse the returned BOD and extract the related attachment information for showing it on the page. For example:


<c:forEach var="attachment" items="${catEntry.attachments}" varStatus="att_status">
<c:set var="attachmentName" value="${attachment.metaData['name']}" />
<c:set var="attachmentPath" value="${attachment.attachmentAssetPath}" />
<c:set var="attachmentID" value="${attachment.attachmentAssetID}" />
<c:set var="attachmentImage" value="${attachment.metaData['image']}" />
<a  href="${storeImgDir}${attachmentPath}" id="WC_CatalogEntryDBThumbnailDisplayJSPF_<c:out value='${attachmentID}'/>_attachment_links_11" class="h_tnav_but">
	<img src="${jspStoreImgDir}${attachmentImage}" alt="" align="left"/>
	<c:out value="${attachmentName}" escapeXml="false"/>
</a>
</c:forEach>