Troubleshooting Near-Real-Time (NRT) index building

Some operations, such as updating a product description, trigger a Near-Real-Time (NRT) delta build index with NiFi. uses invalidation messages to notify the NiFi server of the event. This document will help you troubleshoot the aspects of the NRT process.

Troubleshooting steps

Confirm the Transaction Server is sending messages

When the operation is performed, the Transaction Server will write the event on the WCNifiDistributedMapCache channel. Use the SUBSCRIBE command to confirm the event is being written to the queue:

redis-cli subscribe "{cache-demoqaauth-services/cache/WCNifiDistributedMapCache}-invalidation"

The namespace will vary from system to system.

If the SUBSCRIBE command does not capture any events, there might be a problem with the Redis connection from the Transaction Server, or the specific event might not be configured to do so.

Check if NiFi is listening on the Redis NRT queue

The NiFi server registers PUBSUB listeners with Redis to receive the events.

Use the PUBSUB CHANNELS command on all the master servers to confirm NiFi's listeners are enabled:

redis-cli pubsub channels | grep -i nifi
{cache-demoqaauth-services/cache/WCNifiDistributedMapCache}-invalidation
{cache-demoqaauth-services/cache/WCNifiBatchDistributedMapCache}-invalidation

If the listener (WCNifiDistributedMapCache) is not active, NiFi might not be running or might not be operating correctly.

Ensure that WCNifiDistributedMapCache is configured as follows.

"[${TENANT:-}${ENVIRONMENT:-}auth]:services/cache/WCNifiDistributedMapCache":
    localCache:
      enabled: false
    remoteCache:
      enabled: false
    remoteInvalidations:
      publish: true
      publishFormat: plain

Use Tracing in NiFi to confirm listeners are active

The next step is to use tracing in NiFi to confirm that the NRT listeners are triggering when events are posted to the channels (WCNifiDistributedMapCache and WCNifiBatchDistributedMapCache).

Enable the following traces in logback.xml:

<logger name="com.hcl.commerce.cache" level="TRACE" />
<logger name="org.redisson" level="TRACE" />

Confirm the listeners trigger by looking for the onMessage method in the trace:

... onMessage: onMessage(CharSequence pattern, CharSequence channel, String stringMessage) ..

Also, confirm that WCNifiBatchDistributedMapCache is configured as follows.

"[${TENANT:}${ENVIRONMENT:-}auth]:services/cache/WCNifiBatchDistributedMapCache":
    localCache:
      enabled: false
    remoteCache:
      enabled: false
    remoteInvalidations:
      publish: true
      publishFormat: plain