Troubleshooting: Search indexing performance

Preprocessing and building the search index are required to propagate site changes into WebSphere Commerce search.

Problem

Depending on multiple factors, the use of the di-preprocess and di-buildindex utilities to preprocess and build the search index can take hours. If the use of these utilities is unreasonably slow for your environment, or if time is of the essence for propagating specific changes to your production environment, consider these tips to investigate and improve search processing performance.

Note: These tips were taken from the Search 'n' Rescue series of support blog posts. To get a deeper, more thorough explanation of the tips noted here, see Search 'n' Rescue blog series.

Solution

Preprocessing search data

Ensure the efficient use of the di-preprocess utility:

Specify the use of delta indexing. Delta indexing simply updates your existing preprocessed index with minor changes. This approach is a lot faster than building a full index, but does have its limitations when it comes to larger search index changes. You can perform a delta preprocess by setting the fullbuild parameter to false. To learn about the limitations of delta indexing, see Common business tasks and their impact to the WebSphere Commerce Search index.
Specify the langTables and nonLangTables parameters to avoid redundant preprocessing. For example, if you have five languages, then you might be preprocessing all language insensitive tables five times. The duplication of processing these tables is wasteful because this data isn't changing for each locale. To avoid this redundant processing, you can perform an initial run of preprocessing with -nonLangTables true to preprocess the language insensitive tables, and then process all of your locale-specific tables with -langTables true.
Specify the appropriate preprocessing XML files. If you are planning on building specific search cores, you do not need to spend time preprocessing for them all. Use a combination of the parameters fullpath and onelevel to specify exactly which preprocessing XML files to process.
For example, to preprocess for just the CatalogGroup core, you can use the following parameters:
```
-fullpath WC_installdir/instances/instance_name/search/pre-processConfig/MC_masterCatalogId/databaseType/CatalogGroup/
```

If the di-preprocess utility appears to be performing poorly:

Investigate the data import times within the wc-dataimport-preprocess.log log file. If data import is taking abnormally long, investigate the breakdown of query time, read time, and write time.
If queries are taking too long, check to see if any inefficient or unnecessary customizations to those queries are the cause.
If it is read times, investigate which tables are taking longest, and consider adjusting their fetchSize. If it is write times, consider adjusting their batchSize. Increasing these variables will increase the memory footprint of the utility, so be sure to monitor and test after any changes are made.
Note:
- These queries, the fetchSize parameter, and the batchSize parameter, are located within the preprocessing XML file that contain the table definition.
- Some tables are inherently more complex to preprocess. For example, while you might not have many records inside of the TI_APGROUP table, since this table contains the catalog hierarchy data, there is a significant chunk of time needed to determine the catalog hierarchy based on the retrieved data. As a result, this table might take up most of your indexing time, even if there isn't as much data inside of this table as other tables.

Building search data

Ensure the efficient use of the di-buildindex utility:

If you only need to build specific search cores, be sure to specify them with the indextype and the indexSubType parameters to avoid building more than you need.
For example, if you are only planning on updating the CatalogEntry core, you can use the following parameters:
```
-indextype CatalogEntry -indexSubType Structured
```

If the di-buildindex utility appears to be performing poorly:

Investigate the search index build wc-dataimport-buildindex.log log file. Each core and the time it took to build is individually referenced. If a specific core is taking an abnormally long time for relatively few entries, you might have a performance issue that can be corrected.
For each poorly performing search core, you can increase the ramBufferSizeMB parameter within the solrconfig.xml configuration file for that specific core. This parameter has a maximum value of 2 gigabytes. This setting will increase the memory footprint of the utility, so be sure to monitor and test after any changes are made.
Investigate the size of the search core relative to its number of documents. If a search core is relatively large for its number of documents, this might be caused by the amount of data being processing for each document. For example, the EdgeNGramFilterFactory can be used to break up indexed words into multiple pieces to allow for better partial matches (For example, coffee becomes co, cof, coff, coffe, coffee). However, if you set minGramSize to be too small, then you will get much more data than expected indexed for the field. You should review the different filter factories being used for the search core in the schema.xml configuration file. This helps identify if any individual settings are a cause for your poorer than expected performance.

In addition, custom optimization techniques can be used to improve preprocessing and indexing performance:

Parallel preprocessing and distributed indexing (sharding)
Sharding avoids the need to sequentially index a search core from beginning to end. Instead of indexing each category entry (catentry) or category group (catgroup) one after another, you can break up larger collections into smaller, more easily processed chunks. For example, shard A contains catentry_id 1 - 300,000, shard B contains catentry_id 300,001 - 600,000, etc. After processing is complete, the completed index is merged. These parallel processes use multiples of the system resources that are required during indexing (processor threads, memory, storage). Therefore, depending on your hardware constrictions, sharding might exponentially improve indexing time. Similarly, performance degradation can occur if multiple indexing processes must compete for limited system resources.
Intelligent scheduling
If you are updating specific search cores on a schedule, it doesn't make sense to preprocess and index all of these search cores together. You should build each search core according to the frequency of its update, and to a predetermined tolerance for having old data. For example, if you only update your catalog entry core once a day after a dataload, while the inventory core is updated hourly, you should not be indexing the catalog entry core with your hourly inventory core. This data remains unchanged until the dataload operation.