HCConcurrencyLimiter configuration

Server performance can be affected when too many query requests are concurrently sent to the Elasticsearch servers. You can control the load by limiting the number of threads that can concurrently issue query requests.

Server performance can be affected when too many query requests are concurrently sent to the Elasticsearch servers. Response times can be significantly impacted, leading to a situation where all query requests timeout and return error return codes. By limiting the number of threads that can concurrently issue query requests to the Elasticsearch servers, the load can be controlled so that some requests quickly fail with HTTP return code 429 (Too Many Requests), and many will typically complete without timing out.

The effect of using two limiters, one to limit concurrent first requests and another to limit concurrent query requests (each with its own maxTransactionTimeMilliseconds setting), is to be able to quickly timeout threads that have not issued their first query request. This allows more time for those threads that have already issued at least one query request to complete normally.

Using a limited capacity setting also ensures that a thread is always available to execute requests that do not require Elasticsearch query requests, such as cache hits and no-query requests.

Note: The default capacity and maxWaiters settings assume the thread pool size is configured to provide at least 100 threads. If the thread pool size is reduced, the limiter capacity and maxWaiters settings may also need to be reduced.

The Elasticsearch query server uses two HCConcurrencyLimiter instances to limit how many concurrently executing request threads can send query requests to the Elasticsearch servers.

The first limiter instance is named com.ibm.commerce.datatype.HCConcurrencyLimiter.com.hcl.commerce.search.internal.config.RestHighLevelClientHelper.request and can be configured with the following settings:

capacity

The maximum number of concurrently executing request threads that can send query requests to the Elasticsearch servers. Request threads in excess of this number must wait in a limited capacity queue of threads waiting to issue their first query request to the Elasticsearch servers. The default is 40.

maxWaiters

The maximum number of concurrently executing threads that can wait to send their first query request to the Elasticsearch servers. Request threads in excess of this maximum immediately return an error return code of HTTP status 429 (Too Many Requests). The default is 10.

maxTransactionTimeMilliseconds

The maximum number of milliseconds a request thread can execute before issuing its first query request to the Elasticsearch servers. Threads that attempt to issue their first query request after executing or waiting in excess of the specified maximum milliseconds do not issue any Elasticsearch queries and instead return an error return code of HTTP status 429 (Too Many Requests). The default is 60000.
The second limiter instance is named com.ibm.commerce.datatype.HCConcurrencyLimiter.com.hcl.commerce.search.internal.config.RestHighLevelClientHelper.call and can be configured with the following settings:

capacity

The maximum number of concurrently executing request threads that can concurrently send query requests to the Elasticsearch servers. Request threads in excess of this number must wait to issue query requests to the Elasticsearch servers. The default is 15.

maxTransactionTimeMilliseconds

The maximum number of milliseconds a request thread can execute while sending query requests to the Elasticsearch servers. Threads that attempt to issue query requests after executing or waiting in excess of the specified maximum milliseconds terminate and return an error return code of HTTP status 429 (Too Many Requests). The default is 180000.

In V9.1.13, these configurations are entirely new. The wc-component.json configuration file for the Elasticsearch query server has the following HCConcurrencyLimiter configuration settings:

{
"name": "com.ibm.commerce.datatype.HCConcurrencyLimiter",
"property": [

{ "name": "com.ibm.commerce.datatype.HCConcurrencyLimiter.com.hcl.commerce.search.internal.config.RestHighLevelClientHelper.request/capacity", "value": "40" }
,
{ "name": "com.ibm.commerce.datatype.HCConcurrencyLimiter.com.hcl.commerce.search.internal.config.RestHighLevelClientHelper.request/maxWaiters", "value": "10" }
,
{ "name": "com.ibm.commerce.datatype.HCConcurrencyLimiter.com.hcl.commerce.search.internal.config.RestHighLevelClientHelper.request/maxTransactionTimeMilliseconds", "value": "60000" }
,
{ "name": "com.ibm.commerce.datatype.HCConcurrencyLimiter.com.hcl.commerce.search.internal.config.RestHighLevelClientHelper.call/capacity", "value": "15" }
,
{ "name": "com.ibm.commerce.datatype.HCConcurrencyLimiter.com.hcl.commerce.search.internal.config.RestHighLevelClientHelper.call/maxTransactionTimeMilliseconds", "value": "180000" }
]
}

For more information on the wc-component.json configuration file for the Elasticsearch query server, see Configuring Query services in ZooKeeper .