
Elasticsearch-based search performance tuning
You have numerous options when tuning the performance of NiFi and Elasticsearch. The following documents introduce key tuning parameters, and provide a performance tuning strategy that you can use with the component.
The NiFi and Elasticsearch component may be perceived as relatively slow in processing speed. This may appear to be the case if you are testing the default configuration, which has not been modified or tuned to improve processing performance. This default configuration includes a mostly single threaded configuration of NiFi, and a minimal configuration of the Elasticsearch cluster. This setup can result in performance issues when building the search index. However, the performance of the solution is not only due to the default configuration.
It is also important to choose an optimal hardware configuration when testing. While CPU and memory resources are not critical, the I/O bandwidth of the disk subsystem on the Elasticsearch cluster emerges as a critical factor. Thus, careful selection of the environment from the start can avoid several simple performance pitfalls.
- A minimal configuration, that is applicable for use with a minimal hardware specification, and
- A recommended configuration, that should be applied on the recommended hardware specification.