Caching strategies for REST services

REST services use both server-side caching and client-side caching. Server-side caching is achieved by using dynacache, and client-side caching is achieved by using cache directives in the response header.

Sample cachespec.xml file

Download the following sample cachespec.xml file, for reference: cachespec_sample.zip. It contains examples that show how to set up caching for REST services URLs.

REST cache framework

The following diagram shows the REST cache framework interactions flow:

Where:

RESTCacheFilter parses the request URI, tokenizes it, and adds it to the attribute. It also gets the other attributes, such as previewToken, from the HTTP header.
The path token is taken from the URL and set to the attribute with the servlet cache filter.
The cache-id in the cachespec.xml file defines the rule for caching the REST request. The cache entry of a specific resource is generated with the cache-id.
The dependency-id in in the cachespec.xml specifies the additional cache identifiers and associates them with existing dependency-id types.
If the identifier-id needs to be converted, RESTMetaDataGenerator generates the dependency-id with an ID.
If the cache-id exists in the DynaCache instance, it returns the cache entry directly from the cache. If the cache-id does not exist in the DynaCache instance, it calls the backend servlet to create the cache entry and returns the data to the requester.

To generate dependencies on the WebSphere Commerce server:

Define the dependency-id for identifier, such as CategoryDisplay:storeId:categoryIdentifier, and find the existing related dependency-id, such as CategoryDisplay:storeId:categoryIdentifier. This ensures that the current invalidation function can invalidate the cache entry.
In RESTMetaDataGenerator, it gets the current cache entry-related dependency-id when the request is received.
In RESTMetaDataGenerator, it sends the query to the backend system to get the ID of the item with the identifier as a query parameter, such as storeId, and categoryIdentifier. To improve the performance of queries, it can use the local data cache to cache the query.
In RESTMetaDataGenerator, set the ID as the dependency-id such as CategoryDisplay:storeId:categoryId.

For example, to convert the dependency-id in RESTMetaDataGenerator:


Enumeration dataIds = fragmentInfo.getDataIds();
    while(dataIds.hasMoreElements()){
         String dataId = (String)dataIds.nextElement();
         String covertedDataId = convertDataId(dataId, req);

        if(covertedDataId!=null){
       fragmentInfo.addDataId(covertedDataId);
    }
}

Generating dependencies on the WebSphere Commerce search server is not supported, because there is no default metagenerator for mapping partNumber and categoryIdentifier to their corresponding product ID or category ID.

The com.ibm.commerce.rest.caching.RESTKeyListMetaDataGenerator class can split multiple IDs in REST request queries into multiple dependency IDs for each individual ID. This class can be used on both the WebSphere Commerce and WebSphere Commerce search servers.

Server-side caching

Server-side caching is achieved by using dynacache. You can customize the provided sample cachespec.xml file to suit your business needs, providing custom rules for caching and invalidation, and generating dependencies between them. The sample cachespec.xml files can be found at the following locations:

WebSphere Commerce server: WCDE_installdir\components\foundation\samples\dynacache\REST
Search server: WCDE_installdir\components\foundation\samples\dynacache\Search-Rest

The com.ibm.commerce.rest.caching.RESTCacheFilter extracts the path parameters and sets it as request attributes. The request attributes are then used to build cache-ids. The cache-id rules are then applied to determine which of the following URLs are cacheable:

/wcs/resources/store/storeID/categoryview/@top
/wcs/resources/store/storeID/categoryview/categoryIdentifier
/wcs/resources/store/storeID/categoryview/byId/categoryId
/wcs/resources/store/storeID/categoryview/byIds
/wcs/resources/store/storeID/categoryview/byParentCategory/parentCategoryId
/wcs/resources/store/storeID/productview/partnumber
/wcs/resources/store/storeID/productview/byId/productId
/wcs/resources/store/storeID/productview/byCategory/categoryId
/wcs/resources/store/storeID/productview/bySearchTerm/searchTerm
/wcs/resources/store/storeID/espot/espotIdentifier
/wcs/resources/store/storeID/espot/espotIdentifier/category/categoryID
/wcs/resources/store/storeID/espot/espotIdentifier/product/productID

The following URLs are cacheable for the WebSphere Commerce search server:

/search/resources/store/storeID/categoryview/@top
/search/resources/store/storeID/categoryview/categoryIdentifier
/search/resources/store/storeID/categoryview/byId/categoryId
/search/resources/store/storeID/categoryview/byIds
/search/resources/store/storeID/categoryview/byParentCategory/parentCategoryId
/search/resources/store/storeID/productview/partNumber
/search/resources/store/storeID/productview/byId/productId
/search/resources/store/storeID/productview/byIds
/search/resources/store/storeID/productview/bySearchTerm/searchTerm
/search/resources/store/storeID/productview/byCategory/categoryId
/search/resources/store/storeID/productview/bySearchTerm/searchTerm
/search/resources/store/storeID/sitecontent/suggestions

RESTCacheFilter sets the following request attributes:

Request attributes and path parameters in REST URLs
Request attribute	Path parameter in REST URL
storeId	`storeID`
catalogId	`catalogID`
productId	The product ID of the `partnumber`, `productID`, and `uniqueID` of the product URL
categoryId	The category ID of the `groupID`, `categoryID`, `category_unique_ID`, and `uniqueID` of the category URL
espotId	`espotIdentifier`
searchTerm	`searchTerm`
action	Accepted values: topCategories productDisplay categoryDisplay
urlType	Accepted values: espot search Note: The urlType request attribute is not used.

The dependency-ids are generated to match the invalidation scheme that is defined in other sample cachespec.xml files.

Note: This cachespec must be used with cachespecs where the invalidation schemes are defined. Check the cachespec.xml files under the invalidation/store and invalidation/catalog directories for more detailed definitions.

Server-side caching for REST services that are provided by the WebSphere Commerce server that uses servlet cache and JSP files cannot be used due to local binding on the store web application. However, external applications outside the WebSphere Commerce EAR can use the server-side caching that is provided by the WebSphere Commerce server. In contrast, the REST services that are provided by the Search server can be cached by using server-side caching because they are deployed outside the WebSphere Commerce server. For more information, see Storefront JSP files that use client-side caching.

Client-side caching

Client-side caching is achieved by using cache directives in the response header. The directives that can be set in the response headers are:

Expires
Cache-Control

The Cache-Control directive supports only whether the resource is private or publicly cacheable.

The REST services web module configures the following resources in the WEB-INF/config/com.ibm.commerce.rest/wc-rest-clientCaching.xml file:


<cache>
  <!-- resource name="example" expiresAfter="60 (mins)" isPrivate="true|false"-->

  <resource name="categoryview" expiresAfter="60" isPrivate="false"/>
  <resource name="productview" expiresAfter="60" isPrivate="false"/>
  <resource name="espot" expiresAfter="60" isPrivate="false"/>
</cache>

The caching directives for the REST resources can be overwritten or added to. To overwrite the expiration time or enable client caching for more REST resources, create or modify a customized WEB-INF/config/com.ibm.commerce.rest-ext/wc-rest-clientCaching.xml file.

Storefront JSP files that use client-side caching

Storefront JSP files can use client-side caching when you call REST services by using the <wcf:rest> tag library.

The following conditions are required:

Client-side caching is enabled for the REST service. To set up client-side caching, see the previous section.

The cached="true" attribute must be specified. For example:


<wcf:rest var="sdb" url="store/{storeId}/databean" cached="true">
<wcf:var name="storeId" value="${storeId}" encode="true"/>
<wcf:param name="profileName" value="IBM_Store_Details" encode="true"/>
<wcf:param name="langId" value="${langId}" encode="true"/>
<wcf:param name="jspStoreDir" value="${jspStoreDir}" encode="true" />
</wcf:rest>

Response of REST requests based on the preceding URL and parameters are cached inside the data cache that is called RESTTagCache, with the services/cache/WCRESTTagDistributedMapCache cache JNDI name. This caching is applicable to REST services provided by both WebSphere Commerce and WebSphere Commerce search servers.

Important: Any cached JSP file or JSP file fragment that uses the <wcf:rest> tag to start REST services must mark its cache entry's consume-subfragments to true in the store's cachespec.xml file. If not done, cached JSP files or JSP file fragments might incorrectly start REST services and cause runtime errors.