Troubleshooting: Ingest start-up issues

If the default NiFi connectors were not properly created, restarting the Ingest service does not create them.

Problem

If an environment had issues with the default connectors are not getting created, or are getting created with missing process groups, restating Ingest will do nothing to create the missing connectors.

Ingest start-up behaviour

The Ingest service is started with the other search services. It will connect to NiFi, and either create the framework or check its validity if the framework already exists. Ingest will then check Zookeeper and NiFi to see if they contain any connector descriptors or connectors. Based on this, Ingest will either create all connectors from default descriptors, or not create anything related to connectors.

Information presented in NiFi after start-up


Connectors	Auth environment	Live environment	Toolkit environment	Data environment
`auth.reindex`	Yes	Yes	Yes	Yes
`auth.url`	Yes	No	Yes	Yes
`auth.attribute`	Yes	No	Yes	Yes
`auth.category`	Yes	No	Yes	Yes
`auth.delete`	Yes	No	Yes	Yes
`auth.inventory`	Yes	No	Yes	Yes
`auth.price`	Yes	No	Yes	Yes
`auth.product`	Yes	No	Yes	Yes
`auth.sta`	Yes	No	Yes	Yes
`push-to-live`	Yes	Yes	No	Yes
`live.reindex`	Yes	Yes	No	No
`live.price`	No	Yes	No	No
`live.inventory`	No	Yes	No	No

Solution

For each missing connector, first query ZooKeeper to check whether the missing NiFi connector is there. Using the REST API interface, send a GET /connectors query.

For more information on querying the Ingest service, see Search Ingest Service API..

This request returns a list of connectors that have been created, or were attempted to be created. If the missing connector descriptor is found, use the DELETE /connectors request, which will delete the connector data from ZooKeeper.

Note: The request will return a failure code because there is no connector in NiFi. Having removed the connector from Zookeeper, create the connector manually by putting the connector descriptor in the body of the POST /connectors request.

If process groups are missing in a NiFi connector, use the POST /connectors request to remove the entire connector from both NiFi and ZooKeeper. Following this, issue a POST /connector request in which the body of the NiFi connector descriptor includes the information used to create the information in Zookeeper and the connector in NiFi.

Both of these solutions will disregard any new customizations that have been done outside of the connectors. The only way to keep such customizations is to change the body of the POST request to include the custom process group into the connector.

Refresh Search Services

Stop all search related containers especially Nifi, Zookeeper and Ingest. Delete the NiFi and Zookeeper data folder or pods if using Kubernetes. This is done because NiFi and Zookeeper store the data of connectors and connector descriptors, respectively. Start the services up again and go into the logs for Ingest to see that start-up connectors are loaded.