Upgrading or Migrating to Compass Search

As and administrator, you can upgrade or migrate to HCL Compass Search functionality.

About this task

Note: If your existing Compass full-text search is running on a machine separate from Compass, you can keep the existing Compass full-text search and continue using it with Compass 2.0.1 or later and upgrade to Compass Search 2.0.1 at a later time. If you do so, then the only new feature you will get is the by-ID lookup search of Compass Search.

Attention: There is no upgrade path from the old full-text search to the new Compass Search. When upgrading from any earlier version of HCL Compass or migrating from IBM ClearQuest to HCL Compass 2.0.1 or later, before you install Compass 2.0.1, you will need to first execute the “--prep_upgd_was_profiles” or “--archive_fts” command using “cqtsadmin.pl” tool. The “--prep_upgd_was_profiles” command will allow you to go back to the old full-text search after upgrading or migrating. The “--archive_fts” command will keep important files only and delete the index. If disk space is not a concern and you want the option of going back to the old full-text search, then use “--prep_upgd_was_profiles”.

Once you uninstall the old full-text search, by executing “--prep_upgd_was_profiles” or “--archive_fts”, you can now re-setup and enable your new Compass Search feature using the new search tool. This requires a full re-indexing of your Compass records which can take anywhere from few hours to several hours depending on the number of records you have in your database.

Procedure

How to upgrade from old full-text search to the new search
The following steps assumes you already have full-text search enabled and you are upgrading from HCL Compass 2.0 or earlier or you are migrating from any IBM ClearQuest version to Compass 2.0.1.

Attention: The upgrade and migration process will take some time depending on the number of data you have enabled for search. During this time, Compass search functionality will be disabled and when indexing, search results will not be complete. You should notify your end users of the downtime and the partial results issue before starting the upgrade or migration process.
Steps to upgrade or migrate to Compass 2.0.1 or later:
1. While still on the older version of Compass or ClearQuest, using “cqtsadmin.pl” script, run the command “--prep_upgd_was_profiles” or “--archive_fts”.
2. Upgrade or migrate to Compass 2.0.1 or later using Installation Manager.
3. Using “cpsearch.pl” script, execute the “initSearch” command.
4. If you want to keep the same record types and fields that you had originally set for search, copy the content of your old entity file to the new entity file, otherwise, re-customize the entity file to meet your new need.
5. Using “cpsearch.pl” script, execute the “deploySearch” command.
6. Once “deploySearch” has completed, start the incremental indexer. On Windows, start it from the Windows services, on Linux add the command “startSearchServer” to systemd, init.
Enabling Compass Search for more than on Compass user database

If you have two or more Compass user databases and you want to enable Compass Search on those too, the process is the same as enabling Compass Search for any user database. The only thing changes is to use the proper dbSet and userDb name for those additional databases everything when running the “inistSearch”, “deploySearch” and “startSearchServer”. In addition, you will need to use a different port than the one already used. For Windows, a new Windows service will be created reflecting the name of the search instance.

Note: It is highly recommended that you use the same “-searchHome” and “-solrHome” for the additional Compass user databases for which you are enabling search. This way, all of your instances are in a common location on the file system.
cpsearch.pl script
The “cpsearch.pl” script drives the Compass Search tool. The most important commands are “initSearch”, “deploySearch” and “startSearchServer”. All those were covered earlier. This section covers all the available commands under this script. Most you will never need to use unless if instructed by HCL Support.

Any command that requires “-username” and “-password” means such command will access the Compass database either for reading and / or for writing to the database.

Any command that supports “-overwrite” means the command can overwrite an existing data or file. If so, then be causes on using it and should only be used when directed by HCL Support.

None of the commands and their arguments are case sensitive. However, the database set name and user database name are case sensitive.

Use the following command to disable and archive a Compass Search instance. Once this command completes, Compass Search, for selected user database will be disabled and the home directory of both search and Solr will be renamed to contain “Archived-<timestamp>”.
Command:
```
archiveSearch -- Disables and archives Search instance
```
.
Usage:
```
archiveSearch -dbset <dbset> -userdb <userdb> -searchhome <path>
```
Use the following command to clear the state in which “cpsearch” tool maybe in so that you can overwrite and unblock “cpsearch”. This command should only be used when a command, such as “deploySearch” fails to complete and you want to finish full indexing from where “deploySearch” failed.
Command:
```
clearState -- Clears the current state cpsearch
```
.
Usage:
```
clearState -dbset <dbset> -userdb <userdb> -searchhome <path>
```
Use the following command to copy the raw Apache Solr’s schema to your search instance. The copied Solr schema will be customized to meet your Compass schema. The customization happens when the command “deploySearch” is executed. Note: This is an internal command and it should only be used when instructed by HCL Support.
Command:
```
copySolrSchema -- Copies the generic Solr schema for Search
```
.
Usage:
```
copySolrSchema -dbset <dbset> -userdb <userdb> -searchhome <path> [-overwrite <yes | no>]
```
Use the following command to copy the Apache Solr’s server to your search instance. Note: This is an internal command and it should only be used when instructed by HCL Support.
Command:
```
copySolrServer -- Copies Apache Solr server
```
.
Usage:
```
copySolrServer -dbset <dbset> -userdb <userdb> -searchhome <path> [-overwrite <yes | no>]
```
Use the following command to create the entity file. The entity file is created when you execute the command “initSearch”. The entity file contains a list of all record types and their fields based on your Compass schema. Note: This is an internal command and it should only be used when instructed by HCL Support.
Command:
```
createEntityFile
```
-- Creates the entity file which contains the list of record types and their fields that are candidates for searching.
Usage:
```
createEntityFile -username <user> -password <password> -dbset <dbset> -userdb <userdb> -searchhome <path> [-overwrite <yes | no>]
```
Use the following command to create the Compass search XML file. The search XML file contains settings about the record types and fields enabled for search, settings about the location of Compass Search server, settings about both full and incremental indexing and setting about Compass database connection. Note: This is an internal command and it should only be used when instructed by HCL Support.
Command:
```
createSearchFile
```
-- Creates the Search file which contains the list of record types, their fields, indexing and Search parameters.
Usage:
```
createSearchFile -username <user> -password <password> -dbset <dbset> -userdb <userdb> -searchhome <path> [-overwrite <yes | no>] 
```
Use the following command to create the Windows service for Compass Search instance.
Command:
```
createWindowsService
```
-- Creates a Windows service to start / stop Search services. Available on Windows only.
Usage:
```
createWindowsService -dbset <dbset> -userdb <userdb> -searchhome <path>
```
Use the following command to deploy a Compass Search instance. This command is executed after executing the “initSearch” command. This command will customize Solr’s schema based on your Compass schema and then start the full indexer which will read all record type and their fields that you have enabled for search. This command can take anywhere from few hours to several hours.
Command:
```
deploySearch
```
-- Deploys and enables Search. Can be resumed to complete full indexing.
Usage:
```
deploySearch -username <user> -password <password> -dbset <dbset> -userdb <userdb> -searchhome <path> [-overwrite <yes | no>]
```
Use the following command to disable Compass Search so that the user interface (UI) no longer provides search capability. In general, you should not need to use this command.

Note: If the incremental indexer and Apache Solr search server are running, they will continue to run.
Command:
```
disableSearch
```
-- Disables Search feature. Warning: you may have to run full indexing if records are touched in the database while Search is disabled.
Usage:
```
disableSearch -username <user> -password <password> -dbset <dbset> -userdb <userdb> -searchhome <path> 
```
Use the following command to enable Compass Search so that the UI provides search capability. In general, you should not need to use this command.

Note: If the incremental indexer and Apache Solr search server are not running, any user who attempts to use the search feature will get an error. Also, if Apache Solr search server is running, but the incremental indexer is not running, search result may not be in sync with Compass data.
Command:
```
enableSearch
```
-- Re-enables a previously disabled Search or updates the Search configuration data.
Usage:
```
enableSearch -username <user> -password <password> -dbset <dbset> -userdb <userdb> -searchhome <path>
```
Use the following command to initialize a new Compass Search instance for a Compass user database. If there is already a search instance at the destination “-searchome” or “-solrhome” the command will fail. Also, the value for “-solrport” must be an available port. The command has no way of determining if that port is available or not.

The “-typeahead” parameter is optional and should only be used if you are enabling type-ahead.

If you have more than one Compass user database for which you are enabling search for, it is recommended that you use the same “-searchhome” and “-solrhome” value. This way you keep all of your search instances in a common place.
Command:
```
initSearch
```
-- Initializes Search. Use as first step for Search setup.
Usage:
```
initSearch -username <user> -password <password> -dbset <dbset> -userdb <userdb> -searchhome <path> -solrhome <home> -solrport <port> [-typeahead <file>] [-solrArgs "args"] 
```
Use the following command to ping the Apache Solr server of a search instance. If Solr is running, the ping will response with a success, if not it will response with a failure.
Command:
```
pingSolrServer -- Ping Apache Solr server
```
Usage:
```
pingSolrServer -dbset <dbset> -userdb <userdb> -searchhome <path>
```
Use the following command to query Apache Solr for any search term. This command issues a search against Solr directly bypassing Compass. This command can be very helpful to test and see not only is Solr finding hits, but also the response time for executing a search that Solr consumes.
Command:
```
querySolr -- Executes a query directly against Apache Solr server; returns raw Solr result
```
.
Usage:
```
querySolr -dbset <dbset> -userdb <userdb> -searchhome <path> [-querytext <text>] [-page <page>] [-size <size>] [-timeallowed, <value>] 
```
Use the following command to search Compass for any search term. This command issues a search against Compass as if a user is doing so from Compass UI. This command can be very helpful to test and see not only is Compass is finding hits, but also the response time for executing a search that Compass consumes without going through the UI.
Command:
```
searchCompass
```
-- Executes a search using Compass API. Can be used by any user account.
Usage:
```
searchCompass -dbset <dbset> -userdb <userdb> -searchhome <path> [-searchtext <text>] [-page <page>] [-size <size>] [-timeallowed, <value>] 
```
Use the following command to start the full indexer. The full indexer is an indexing mode in which all the records for record types enabled for search are read from the Compass database and sent to Apache Solr for indexing. This process happens by default when you execute the command “-deploySearch”. However, if “-deploySearch” fails before it completes, while it indexing, you will use this command to finish off from where it failed.

The command can be run over and over, even if it fails or if you stop it before it completes. If you do so, it will continue indexing from where it left off. However, if you need to, you can force it to start from the beginning by setting the parameter “-overwrite” to “yes”.

Note: This command can take anywhere from few minutes to several hours and even over a day. There is no easy way to determine how long it will take to complete because that all depends on several factors such as, primarily, the size of your database (number of records and their fields and the data in those records), the performance of your database and the network traffic. The search server setup, such as CPU, RAM and disk speed are also a factor but data size and database performance are the main factors.
Command:
```
startFullIndex
```
-- Starts full indexing. Can be resumed to complete indexing from last check>] point.
Usage:
```
startFullIndex -username <user> -password <password> -dbset <dbset> -userdb <userdb>>] -searchhome <path> [-overwrite <yes | no>] 
```
Use the following command to start the incremental indexer of Compass Search. This command is executed when you start the Windows services for a search instance or on Unix, when you execute the command “startSearchServer”.
Command:
```
startIncrIndex
```
-- Runs the incremental indexer. Resumes indexing from last check point.
Usage:
```
startIncrIndex -dbset <dbset> -userdb <userdb> -searchhome <path> >>] -startas <continues | once> [-overwrite <yes | no>]
```
Use the following command, on Unix, to start the search server. On Windows, this command is executed when you start the Windows service for your search instance.

This command will start the Apache Solr server as well as the incremental indexer. If Solr server is not started, then any search operation will fail. If the incremental indexer is not started, then index will not be in sync with changes to the Compass database and thus search results will not be complete or accurate.

Attention: On Linux, you will need to add this command to your “systemd init”. If you do not, the Solr server and the incremental indexer will not be running resulting with errors and / or incomplete search results.

Attention: On Windows, you should not use this command. Instead use the Windows services to start it. If you use this command, and you logoff or Windows reboots, Solr server and the incremental indexer will not start and thus search will not be available.
Command:
```
startSearchServer
```
-- Starts Search server; Apache Solr and incremental indexer.
Usage:
```
startSearchServer -dbset <dbset> -userdb <userdb> -searchhome <path>
```
Use the following command to start the Apache Solr server. This command is executed when “deploySearch” command is executed. It is also executed when “startSearchServer” is invoked either from the Windows services or from “system init” on Linux.

Note: This is an internal command and it should only be used when instructed by HCL Support.
Command:
```
startSolrServer
```
-- Starts Apache Solr server.
Usage:
```
startSolrServer -dbset <dbset> -userdb <userdb> -searchhome <path>
```
Use the following command to stop the Apache Solr server. This command is executed when “archiveSearch” or when “startSearchServer” is stopped (on Windows this happens when the service is stopped, on Linux it also happens when “system init” is stopped).

Note: This is an internal command and it should only be used when instructed by HCL Support.
Command:
```
stopSolrServer -- Stops Apache Solr server
```
.
Usage:
```
stopSolrServer -dbset <dbset> -userdb <userdb> -searchhome <path>
```