Troubleshooting Steps for iCrawler (Advanced Knowledge component)

These troubleshooting steps provide solutions for common issues with iCrawler, including the service not starting, internal server errors, 404 resource not found errors, and "could not get any response" errors.

HCL.iAutomate.iCrawler service is not starting

Table 1. Table 100 - iCrawler: Scenario 1
Issue Description HCL.iAutomate.iCrawler service is not starting
Modules Impacted iCrawler
Probable Root Cause Invalid “iknowledgeCrawler.conf” file content

Steps to resolve

  1. Check for existence of the file iknowledgeCrawler.conf at location <Apache_Root_Folder_Path>/conf.

For e.g. - “C:/Apache24/conf” has <Apache_Root_Folder_Path> as “C:/Apache24”.

  1. Ensure that paths are mentioned correctly in the iknowledgeCrawler.conf file.
  2. In the conf file, inside < VirtualHost *> tags, there are two lines that have path to the iCrawler directory as follows:

WSGIScriptAlias /iKnowledge <<Path to Crawler>>/crawler_v5/main.wsgi

<Directory <<Path to Crawler>>/crawler_v5>

  1. Ensure that these paths are correct.

<VirtualHost *>

WSGIScriptAlias /iKnowledge C:/HCL.iAutomate.Crawler/crawler_v5/main.wsgi

SSLVerifyClient require

SSLVerifyDepth 10

<Directory C:/HCL.iAutomate.Crawler/crawler_v5 >

WSGIApplicationGroup %{GLOBAL}

Require all granted

</Directory>

</VirtualHost>

  1. Ensure that service was created through Service Account and not through the Local Account. Refer to below figure to check the status.
Figure 1. Figure 145 - HCL.iAutomate.iCrawler
crawler_pic1
  1. If not, open Windows Command Prompt in Administrator mode and run the below command:

sc config HCL.iAutomate.iCrawler obj= “ID” password= “PSWD”

where ID is the [Domain\SA ID] and PSWD is the [Password of SA] and SA is

[Service Account].

Log On As’ field corresponding to the service HCL.iAutomate.iCrawler should be changed to ‘Service Account’. Click Start to start the service.

Service returns Internal Server Error - status code 500

Table 2. Table 101 - iCrawler: Scenario 2
Issue Description Service is reachable but returns Internal Server Error with status code 500
Modules Impacted iCrawler
Probable Root Cause Python package not available

Steps to resolve

  1. Check the Apache error log at <Apache_Root_Folder_Path>/logs/error.log.
  1. If the error is because of the missing package, perform the following steps:
  1. Open the Windows Command Prompt in the Administrative mode in Apache_Root_Folder_Path:
  1. If open internet access is available, run the below command:

pip install package_name

  1. If open internet access is not available, download wheel file for package from where user has internet connectivity and move onto server at same location where it needs to be installed. E.g. https://pypi.org has a repository of most used wheel files, else other sources are available as well. Subsequently, run the below command:

pip install package_name.whl

‘404 Resource Not Found’

Table 3. Table 102 - iCrawler: Scenario 3
Issue Description Error message ‘404 Resource Not Found’ when iCrawler is hit.
Modules Impacted iCrawler
Probable Root Cause
  • Paths are not correct in iKnowledgeCrawler.conf
  • Port is not correct

Steps to resolve

  1. If the issue is due to the incorrect paths, refer to resolution steps mentioned for issue in Table 100 - iCrawler: Scenario 1.
  1. Also, check if correct port is being used. To identify the correct port, perform the following steps:
  • Open iKnowledgeCrawler.conf
  • Search for Listen xxxx

e.g. – Here the image below shows the port set for iCrawler component is 9000

#

Listen 9000

# ports, instead of the default. See also the <VirtualHost>

# directive

  1. The set port is one where iCrawler is deployed.

‘Could not get any response’

Table 4. Table 103 - iCrawler: Scenario 4
Issue Description Error message ‘Could not get any response’ appears when iCrawler is being hit.
Modules Impacted iCrawler
Probable Root Cause HCL.iAutomate.iCrawler service is not running

Steps to resolve

  1. Press Win+R, type services.msc and press Enter to open Windows Services.
Figure 2. Figure 146 - HCL.iAutomate.iCrawler
  1. Find service HCL.iAutomate.iCrawler and select it.
  2. Click Start to start the service, if it’s not running already.
Figure 3. Figure 147 - HCL.iAutomate.iCrawler
1