cqtsadmin.pl

cqtsadmin.pl automates the steps to set up, configure, and administer HCL Compass full-text search.

Synopsis

: cqperl cqtsadmin.pl --username superusername --password password --dbset dbset --userdb connectionname --ftshome cqftshome [options]

Description

The cqtsadmin.pl script is one of two components of the full-text search administrator tool. The other component is cqtsadmin-dbset-userdb.xml, which is generated by the cqtsadmin.pl script. The cqtsadmin-dbset-userdb.xml file provides and holds data about full-text search deployment. You edit cqtsadmin-dbset-userdb.xml to complete your deployment. All command-line options work in the same way on Windows™, UNIX™ or Linux™ with the exception of scrub_oplog which is not available on UNIX or Linux.

Important: To run the cqtsadmin.pl command, the CQFTS_AppServer_HOME environment variable must be set to the directory where WebSphere® Application Server is installed. Under certain circumstances, such as when you install HCL Compass into an existing WebSphere Application profile, or when you install the full-text search feature on a separate system that remotely accesses HCL Compass Web servers, the CQFTS_AppServer_HOME environment variable is not set and an error similar to this is displayed:

Cannot determine WebSphere rsqos AppServer home. See the HCL Compass Full-Text Search Administrator Guide on how to set it via the CQFTS_AppServer_HOME environment variable.

To resolve the issue, set the CQFTS_AppServer_HOME variable to the directory where you installed WebSphere Application Server. For example, set the variable as follows:

On Windows operating systems:: set CQFTS_AppServer_HOME=C:\Program Files\IBM\WebSphere\AppServer
On the UNIX system and Linux operating systems:: setenv CQFTS_AppServer_HOME /opt/IBM/WebSphere/AppServer

Important: The arguments to the dbset, userdb and ftshome arguments are case sensitive. You must maintain the same case throughout the use of the cqtsadmin.pl script. Otherwise, the full-text search deployment might be reconfigured.

The ftshome option

When you deploy full-text search or refer to a deployment, the required command-line options ftshome, dbset, and userdb arguments define where your deployment data is located. The following example shows how to create a new HCL Compass full-text search deployment on the D drive in the CQ.Search directory. The directory is created if it does not exist. The TextSearch_SAMPL subdirectory is created in the preceding directory. The subdirectory name is generated based on your HCL Compass database set name and your HCL Compass logical user database name. The subdirectory contains full-text search data for this deployment.

: cqperl cqtsadmin.pl --username admin --password ldquordquo --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --init_cq_fts --create_fts_was_profile manual --fts_was_profile_home D:\FTS.WASprofiles

After the D:\CQ.Search\TextSearch_SAMPL directory is created, it is the location of the full-text search deployment for the TextSearch database set and the SAMPL user database. All data and settings that are related to this deployment are made in this directory. Subsequent commands that you issue that use the same value for the ftshome, dbset, and userdb arguments are applied to this directory.

To deploy full-text search for a second HCL Compass user database, specify the required command-line options that pertain to that second HCL Compass user database. Additional HCL Compass user databases will deploy full-text search configurations that are based on the specified ftshome arguments and the generated subdirectory structure.

: cqperl cqtsadmin.pl --username admin --password ldquordquo --dbset TextSearch --userdb PROD --ftshome D:\CQ.Search --init_cq_fts --create_fts_was_profile manual --fts_was_profile_home D:\FTS.WASprofiles

Table 1. Full-text search deployment directory structure
Directory or file	Description
`ftshome`\`dbset_userdb`\logs\	This directory holds logs of every command that you issue against this deployment. Reference this directory to get a history of which commands you have used, when you used them, and their status. HCL customer support might examine these logs when it works with you on an issue. Passwords are displayed as asterisks (*) in the logs and screen output, not in plain text.
`ftshome`\`dbset_userdb`\Solr\solr\conf\schema.xml	This file is one of the configuration files that the search engine uses to determine which fields to index and search. When the cust_solr_files command is issued, the fields in this file are customized to match the fields in your record types as specified by your entity file. You might have to edit this file to further customize it if your HCL Compass database is not in English. See Enabling full-text search on a non-English database for more information.
`ftshome`\`dbset_userdb`\Solr\solr\data\index\	This directory holds the actual index of your deployment. Attention: Do not modify the contents of this directory. Modifying the directory might compromise the integrity of your deployment and might require reindexing or redeployment.
`ftshome`\`dbset_userdb`\AboutThisFTS.txt	This file is generated once during the initial deployment of full-text search. It contains information about the deployment that you might have to refer to. HCL customer support might examine this file when it works with you on an issue.
`ftshome`\`dbset_userdb`\CQ`-dbset-userdb`.xml	This file is the HCL Compass full-text search properties XML file. The file contains data about batch and update indexing, the search server, your connection profile, and record types, and fields to index and search. HCL customer support examines this file when it works with you on an issue.
`ftshome`\`dbset_userdb`\cqtsadmin`-dbset-userdb`.xml	This file contains the full-text search administrator configuration. It contains data about your deployment. Most of the data is set during deployment, but you might have to edit this file to customize some settings. HCL customer support examines this file when it works with you on an issue.
`ftshome`\`dbset_userdb`\Entity`-dbset-userdb`.txt	The entity file contains a list of entity types and their fields, for which searching is enabled. During deployment, you might have to edit this file to remove record types or fields that you do not want to search. After you complete the deployment, do not modify this file. HCL customer support examines this file when it works with you on an issue.

National language support

To use cqtsadmin.pl on a non-English operating system, set the LANGUAGE system environment variable to one of the following supported values:

en English (United States; default)
de German (Germany)
fr French (France)
it Italian (Italy)
br Portuguese (Brazil)
es Spanish (Spain)
cn simplified Chinese (China)
hk traditional Chinese (Hong Kong, Special Administrative Region of China)
tw traditional Chinese (Taiwan; same as hk)
ja Japanese (Japan)
ko Korean (South Korea)

If LANGUAGE specifies an unsupported value, cqtsadmin.pl fails with an error message.

Options and arguments

The cqtsadmin.pl script has required command-line options and optional command-line options. You must provide the required command-line options every time you run the cqtsadmin.pl script. If any of the parameters to the required options are incorrect, the tool fails with an error message. The tool authenticates the user against the HCL Compass database before it takes action. You must provide at least one optional command-line option when you run the cqtsadmin.pl script.

Required command-line options

username superusername: HCL Compass user name with super user privileges

password password: HCL Compass user password

dbset dbset: HCL Compass database set name. The value is case sensitive.

userdb connectionname: HCL Compass user database name. The value is case sensitive.

ftshome cqftshome: HCL Compass full-text search home directory. This options contains all configuration files that are related to this deployment, and Solr files, settings, and the Lucene index. The value is case sensitive.

Optional command-line options

The optional command-line options take specific actions on the HCL Compass full-text search deployment. All commands generate informational, progress, warning, error, and instructional output. The instruction output helps you recover from an error. The displayed output is also logged in the log directory. This log data is helpful when you try to debug or trace your action on a deployment, because you do not have to redirect the screen output to a file. Typically, commands do not fail. If a failure does occur, most commands revert all changes. When a change cannot be reverted, an error message is displayed with instructions on what to do. You can issue an optional command-line option multiple times. When multiple optional command-line options are given, they run in the order that they appear on the command line. If an option fails, subsequent command evaluation and execution ceases, and the tool exits with an error message. The optional command-line options can be grouped into two categories: commonly used and rarely used options.

add_record_type

Summary: Adds one or more new record types to the index. This option temporarily disables the full-text search feature for HCL Compass Web users while the command is running. The list of new record types and their associated fields are provided through the <addRecordType> tag that is in the full-text search administrator configuration file.

Usage

Use this command-line option to add a record type if you omitted a record type during your initial full-text search deployment or if you added a new record type to your HCL Compass schema after your initial deployment and you want to search on the new record type. Use this command if you renamed, added, or removed a field in a record type that is already indexed. To reflect the change in your index, issue the remove_record_type command to remove the record type, and then issue this command to add it again. To add two or more record types, use semicolon as a separator. For example, the following code adds the Customer and Product record types.

<newValue
required="no">Customer=CustomerNum,address,phone,product;Product=name,version</newValue>

If you list a record type without a field list, then all fields of that record type are added. In the following example, all fields for the Customer and Product record types are added.

<newValue required="no">Customer;Product</newValue>

Effect: This command affects the HCL Compass full-text search index, the HCL Compass full-text search properties XML file, the entity file, and the Solr schema.xml file. Before you issue this command, back up your deployment. Run this option at off-peak hours. The operation is time consuming and causes brief full-text search downtime.

Stateful status: The command is stateful. If the command fails during one of its execution points, you should be able to correct the issue and then run the command again. The operation continues from where it stopped. If a failure occurs, an error message tells you what to do.

Example

You have to add a new record type called Customer.

Edit the cqtsadmin-TextSearch-SAMPL.xml file, and change the <addRecordType> tag.
<newValue required="no">Customer=CustomerNum,address,phone,product</newValue>
Create a backup.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --backup_fts E:\FTSBackup
Add the new record type.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --add_record_type
Edit the cqtsadmin-TextSearch-SAMPL.xml file, and remove what you added for to the <addRecordType> tag. This step is primarily a clean-up task.

archive_fts

Summary: Archives a HCL Compass full-text search deployment. It disables the active full-text-search deployment, and then removes a deployed WebSphere Application Server profile. It preserves all configuration data and the index. This option can be used to start over with a new configuration and refer to your old configuration.

Usage: Use this command-line option to start a fresh deployment, or use this option if you no longer require the full-text search feature of a deployment. This command-line option stops full-text search services and archives all relevant resources such as services and files. You can refer to the deployment after you archive it.
Note: Do not use this command-line option if you plan a future restoration. Use the prep_upgd_was_profiles command-line option instead.

Effect: This command-line option disables full-text search. It also removes and deletes all files, resources, and settings that were used and set under WebSphere Application Server for this deployment. The deployment data under ftshome remains intact, but is renamed to dbset_userdb.Archived-time-stamp.

Stateful status: This option is not stateful. If the command fails during one of its execution points, you might have to complete the archiving manually. A progress report and an error message instruct you how to recover from the error.

Example

The following example shows how to archive a deployment.

cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --archive_fts

After you run the script, the archived full-text search deployment is called D:\CQ.Search\TextSearch_SAMPL.Archived-time-stamp.

backup_fts destination

Summary: Creates a backup copy of your HCL Compass full-text search deployment. After a backup is created, you can recover data from the backup or from the whole deployment. For best results, first create a backup before running commands that considerably alter your existing deployment. Before you back up your deployment, make sure that you have the same amount of disk space at your backup location that your deployment at ftshome uses.

Usage: Use this command to create a backup when you add or update a record type or when your organization policy requires that you maintain periodic backups.
Note: Do not use this command in lieu of the prep_upgd_was_profiles command-line option, which also handles the backup WebSphere Application Server profiles for each user database.

Effect: This command temporarily disables update-mode indexer while the backup takes place. Full-text searches might not be up-to-date for the duration of the backup. The duration depends on the size of your index, the speed of your hard drive, and your network, if you are backing up over a LAN or WAN.

Stateful status: This option is not stateful. If the command fails during one of its execution points, you must complete the backup manually or start over, depending on the failure type and the error message that you receive. The most likely failure is having insufficient disk space on the destination device. No deployment data is changed during the backup.

Example

You want to create a backup of your deployment before adding new record types.

cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --backup_fts E:\FTSBackup

After running the script, the archived full-text search deployment is called E:\FTSBackup\TextSearch_SAMPL.Backup-time-stamp.

clear_state

Summary: Resets the state in the cqtsadmin.pl tool procedure so that there is no state. In effect, whatever state the tool was in which might have been an incomplete state, is cleared.

Usage: Use this command-line option to clear the state of a stateful command so that you can issue other commands or reissue the stateful command. The time to clear the state depends on which stateful command you plan to clear, the state that the stateful command was last in, and the error message and the provided corrective instructions.

Effect: The effect of running this command-line option depends on which stateful command was stopped, and how much of the command completed before stopping. The log and error message indicate if you can reset the state.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You have to clear the state of a stateful action so that you can recover from an unrecoverable error, according to the error message instructions.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --clear_state

copy_fts_template

Summary: Copies and sets the HCL Compass full-text search default template. When you deploy full-text search on a database that is not yet enabled for full-text search, you must start from a clean default template and copy it to your ftshome directory. If you attempt to use this command over an existing deployment, it fails with an error.

Usage: In general, you do not need to use this command-line option directly because it is called when you issue the init_cq_fts command-line option. This command-line option is provided in case you need to fine-tune or debug your deployment.

Effect: This command-line option copies the default data needed for the full-text search feature to the specified ftshome directory.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL Compass Support has instructed you to issue this command to debug a deployment issue, or to customize a deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --copy_fts_template

create_fts_was_profile startup-type { manual | automatic | disabled }

Summary: Creates a HCL Compass full-text search WebSphere Application Server profile. Required for all new deployments.

Usage

Each HCL Compass full-text search deployment must have its own WebSphere Application Server profile, one WebSphere Application Server profile for each HCL Compass full-text search enabled user database.

The parameter value configures Windows service status for Windows operating system deployments only. This value is ignored on UNIX and Linux operating systems for which you must configure the WebSphere Application Server profile to start at boot time as a daemon.

The WebSphere Application Server profile name is determined from your database set and user database name. However, you can override it with the <ftsWASProfileName> tag.

Specify the location for creating the WebSphere Application Server profile with the fts_was_profile_home command-line option. If you do not specify the location, the %COMPASS_HOME%/cqweb/ default location is used. For best results, specify your own location.

Note: Always use this command-line option with the init_cq_fts command-line option so that a WebSphere Application Server profile is created and customized based on your HCL Compass full-text search deployment. Otherwise, the deployment will fail unless the WebSphere Application Server profile has been previously created with this tool and is being reused.

Effect: This command-line option creates a new WebSphere Application Server profile under the WebSphere Application Server using the next available port. When the operation completes, the disk space utilization is about 200 MB. If the profile is set to Automatic on Windows, the service starts automatically when Windows restarts.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example

You plan to deploy full-text search for a second user database. The name of your database set is MASTR, and the user database name is SAMPL.

Issue the following command to set up an initial deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset MASTR --userdb SAMPL --ftshome D:\CQ.Search --init_cq_fts --create_fts_was_profile automatic --fts_was_profile_home D:\FTS.WASprofiles
Modify your entity file to contain only the record types and fields that you are interested in.
Issue this command to complete your deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset MASTR --userdb SAMPL --ftshome D:\CQ.Search --setup_cq_fts

cust_fts_files

Summary: Customizes the HCL Compass full-text search Properties XML file based on your user database, entity file, and full-text search administrator configuration file. When you deploy HCL Compass full-text search for the first time, you must customize the full-text search properties file CQ-dbset-userdb.xml.

Usage: The default full-text search template contains generic settings that you customize based on your user database. One file that you must customize is the full-text search properties XML file. This file holds a list of all record types and their fields based on what is specified in your entity file. This file also contains parameters such as which field to use as the display field, how often to check for changes to your HCL Compass user database, and how to communicate with your HCL Compass database and server. In general, you do not have to use this command-line option directly because it is called when you issue the setup_cq_fts command-line option. This command-line option is provided in case you have to fine-tune or debug your deployment.

Effect: If you use this command-line option on a deployment, it overwrites the CQ-dbset-userdb.xml file. All changes that were made to the file, either manually or by issuing commands, are lost.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support asks you debug a full-text search deployment issue or to customize a deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --cust_fts_files

cust_solr_files

Summary: When you deploy HCL Compass full-text search for the first time, you must customize the Solr schema file which is based on your current HCL Compass entity file.

Usage: The default full-text search template is generic and contains default settings that you must customize based on your user database. One file that you must customize is the Solr schema.xml file. This file contains all the fields of all the record types that you have set to be searched by using the entity file. In general, you do not have to use this command-line option directly because it is called when you issue the setup_cq_fts command-line option. This command-line option is provided in case you need to fine-tune or debug your deployment.

Effect: This command-line option reads data from your entity file and then customizes the Solr schema.xml file. If you use the option over an existing deployment, it refactors the schema.xml file. If you changed the entity files after your initial deployment, then the previous values are lost.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support asks that you debug a full-text search deployment issue, or to customize a deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --cust_solr_files

delete_fts_was_profile

Summary: To delete a WebSphere Application Server profile or you want to start over again, run this command-line option. It deletes the HCL Compass full-text search WebSphere Application Server profile that is associated with this HCL Compass full-text search depolyment. This option is different from archive_fts, in that only the WebSphere Application Server profile is deleted. However, the full-text search home directory and associated metadata is not removed. To fully remove a HCL Compass full-text search deployment, use the archive_fts option instead.

Usage: In general, do not use this command directly because it is called when you issue the archive_fts command-line option. This command is provided if you have to fine-tune or debug your deployment.

Effect: Resources that are used by WebSphere for this WebSphere Application Server profile are released. If HCL Compass Web full-text search was not disabled with the disable_cqweb_fts command-line option, then full-text search searches result in errors.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example

You have to rename or change the location of your WebSphere Application Server profile, but you do not want to completely redeploy your full-text search solution.

Delete the WebSphere Application Server profile.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --delete_fts_was_profile
Edit thecqtsadmin-dbset-userdb.xml file and change the <ftsWASProfileName> tag from the automatically generated default name to the new name for the WebSphere Application Server profile. The name must be unique. Otherwise, the command fails.
Recreate the WebSphere Application Server profile.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --create_fts_was_profile automatic --fts_was_profile_home D:\FTS.WASprofiles
If you were also planning to change the port number for this WebSphere Application Server profile, edit the cqtsadmin-dbset-userdb.xml file, and update the port number in the <ftsWASProfilePort> tag prior to step 3 above. Or, issue this command to redefine the ports instead:
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --create_fts_was_profile automatic --fts_was_profile_home D:\FTS.WASprofiles --was_profile_ports_file D:\CQ.Search\TextSearch_SAMPL\cqftsportdef.props

disable_cqweb_fts

Summary: This command-line option disables the Full Text radio button in HCL Compass Web, and stops recording operation logs (oplogs) in a non-replicated HCL Compass user database. This command-line option has no effect on creating oplogs in replicated environments.
Attention: Use this command-line option with caution. It disables oplog recording if your HCL Compass database is not replicated. If HCL Compass records are changed when oplogs are not generated, and the changed records are not reindexed. You will have to perform a reindexing of the entire user database. Either block users from accessing HCL Compass or permit only read-only operations until the full-text search capability is reenabled.

Usage: You do not have to use this command unless you are in a test environment or HCL customer support instructs you to do so.

Effect: Oplog generation stops, if your database is not replicated, and the Full Text radio button is disabled.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support instructs you to disable full-text search to help you resolve issues.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --disable_cqweb_fts

enable_cqweb_fts

Summary: Enables the Full Text radio button in HCL Compass Web. Oplog generation is also enabled if your HCL Compass database is not replicated and is at feature level 7. If your HCL Compass user database is replicated, no change is made to oplog generation. If your deployment is not correctly configured, users who attempt to use full-text search receive error messages.

Usage: You do not have to use this command unless you are in a test environment, or HCL customer support instructs you to use it.

Effect: Oplog generation starts, if your database is not replicated and is at feature level 7. The Full Text radio button in HCL Compass Web is enabled.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support instructs you to enable full-text search to help you resolve issues you have.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --enable_cqweb_fts

fresh_batch_idx

Summary: This command-line options enables you to force a full reindex in batch mode after the initial indexing. Batch-mode indexing is the process of reading all HCL Compass records that are configured for searching and sending the data of those records to the search engine (Solr) for indexing. Batch-mode indexing is performed as part of running the setup_cq_fts command-line option. Before you issue this command, issue the stop_update_idx command-line option to stop the update-mode indexer. If you do not, the batch-mode indexer might replace the data of a more recent record indexed by the update-mode indexer. When the re-indexing is complete, your index is fragmented. Typically, this fragmentation has no performance impact on your searches. However, the index size might grow to up to twice the current size. To optimize the index and reduce its size, run the optimize_idx command-line option.

Usage: You do not need to use this command unless you are in a test environment, or HCL customer support instructs you to do so.

Effect: The search index is updated. Its size grows to up to twice the current size. Therefore, confirm that you have enough disk space before using this command. While reindexing, search results might not be complete because the update-mode indexer is disabled.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example

HCL customer support instructs you to completely reindex your searchable records.

Stop the update-mode indexer.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --stop_update_idx
Force batch mode reindexing.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --fresh_batch_idx
Run index optimization.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --optimize_idx
Enable update-mode indexer.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --start_update_idx
Note: You can combine these steps into one command. You might want to do this because this operation takes a long time if you reindex many records.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search -- stop_update_idx --fresh_batch_idx --optimize_idx --start_update_id

fresh_update_idx

Summary: Forces a full reindex of all recorded oplog record changes by the update-mode indexer. Use this option cautiously. The update mode indexer is single threaded. This operation takes a long time, especially if your HCL Compass database contains many oplogs.
Update-mode indexing is the process of monitoring the HCL Compass database for changes on record types that are configured for searching. The monitoring is done by checking for oplogs in the HCL Compass database. Update-mode indexing is enabled as part of the setup_cq_fts command-line option. Unlike the fresh_batch_idx command-line option, this command does not require you to stop the update-mode indexer. When this command completes against a populated index, your index will be fragmented. Typically, this fragmentation has no performance impact on your searches. However, the index size might grow to be up to twice the current size. To optimize the index and reduce its size, run the optimize_idx command-line option. Consider carefully whether to start update-mode indexer from the first recorded oplog. Over time, you amass an oplog for every action taken in a record and you might not want to index from the first recorded oplog, especially if you have not been consistently purging oplogs. This command is intended for use with testing environments and to debug full-text search deployment issues working with HCL customer support.

Usage: You do not have to use this command unless you are in a test environment or HCL customer support instructs you to do so.

Effect: The search index is updated. Its size could grow to up to twice the current size. Ensure that you have enough disk space before you use this command.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support instructs you to force a reindexing of the update-mode indexer.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --fresh_update_idx

fts_was_profile_home destination

Summary: Use this command-line option with create_fts_was_profile command-line option to specify where the WebSphere Application Server profile for this HCL Compass full-text search deployment will be created. If you omit this option, the default location of ${COMPASS_HOME}/cqweb/ is used. The default location is not always appropriate because additional files that are created in the HCL Compass installation directory may interfere with HCL Compass installation, such as uninstalling or upgrading. This command-line option can also be used with the restore_was_profile command-line option. You can specify a location that is different from the original where the full-text search WebSphere Application Server profiles will be created.

Usage: You want to deploy HCL Compass full-text search for a HCL Compass database, and you do not want the default location to be used to create and store WebSphere files related to your deployment. Use this command-line option in conjunction with the create_fts_was_profile command-line option to specify where to create your full-text search WebSphere Application Server profile. If you have more than one full-text search deployment on this machine, use the same location for all of them so that all of your WebSphere Application Server profiles are in the same location.
Effect: A directory is created, if it does not exist, in the fts_was_profile_home location. It contains the data related to the WebSphere Application Server profile. A subdirectory within this directory is created. It represents the WebSphere Application Server profile name, which is cqsearchprofile_dbset_userdb or cqfts_dbset_userdb.
Stateful status: This option is not stateful. This command should never fail unless if there is an I/O error or the path is invalid such as a drive letter that does not exist or a nonexistent Unix or Linux mount path.
Example: You need to deploy HCL Compass full-text search. You do not want to use the default location for WebSphere Application Server profiles.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --init_cq_fts --create_fts_was_profile manual --fts_was_profile_home D:\FTS.WASprofiles

gather_diagnostic_data

Summary: Collects data that helps HCL customer support diagnose potential HCL Compass full-text search issues. If you require support, run this diagnostic command-line option and send the data that you collect with this command-line option to support to expedite assistance. This command-line option gathers relevant data about your HCL Compass full-text search deployment. The data is copied into a directory that you might be instructed to send in to HCL customer support. Before you send the data, verify that it contains no confidential information. Typically, the most sensitive data are the record-type names that you enabled for searching and a history of search terms that your organization submitted. The history is the log that WebSphere maintains for your WebSphere Application Server profile. Passwords are converted to asterisks when output is sent to the screen or log files. They are never stored in plain text.

Usage: Use this command to gather and send diagnostic data to HCL customer support to help you diagnose issues with full-text search.

Effect: none of your data or configuration settings is changed. A new directory is created with the name of your deployment and a time stamp. The cumulative size of the diagnostic data varies depending on the total sizes of logs in your deployment, which is usually in megabytes. The actual index is not part of the diagnostic data.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support instructs you to send diagnostic data to help you resolve issues with full-text search. The following command creates the diagnostic data and places it in a D:\CQ.Search\TextSearch_SAMPL.Diag-time-stamp file.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --gather_diagnostic_data

gen_entity_file

Summary: This command-line option generates the entity file, which contains all record types that can be submitted to the HCL Compass database. By default, the entity file contains all record types and all their fields of your HCL Compass schema. And, they are candidates for full-text search. You can customize the file to select only the record types and associated fields that you plan to index. When you deploy full-text search against a HCL Compass user database for the first time, you must have an entity file that holds all record types and their fields that the HCL Compass user database schema refers to. This entity file is used as an input to generate the full-text search properties XML file, the Solr intermediate XML file, and the Solr schema.xml file.

Usage: In general, you do not have to use this command directly. It is called when you issue the init_cq_fts command-line option. This command is provided in case you have to fine-tune or debug your deployment.

Effect: If you use this command on an existing deployment, it overwrites the Entity-dbset-userdb.txt file, and you lose the edits that you have made.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support Support has instructed you to debug a full-text search deployment issue or to customize a deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --gen_entity_file

gen_fts_files

Summary: This command-line option generates HCL Compass full-text search setup files based on your user database, entity file, and full-text search administrator configuration file. When you deploy full-text search on a user database for the first time, you must generate the full-text search properties XML file. This file holds information about your deployment settings such as how often to index, the batch size for indexing, and search server information.

Usage: In general, you do not use this command-line option directly. It is called when you issue the setup_cq_fts command-line option. This command-line option is provided in case you have to fine-tune or debug your deployment.

Effect: If you use this command on an existing deployment, it overwrites the Entity-dbset-userdb.txt, Solr-dbset-userdb.txt, and CQ-dbset-userdb.xml files and you lose the edits that you made to them.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support has instructed you to debug a full-text search deployment issue or to customize a deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --gen_fts_file

help

Summary: Displays help text and then closes.

Usage: You are familiar with the cqtsadmin.pl script, but you want a quick refresher about the available commands.

Effect: none. This command does not change any data and does not require authentication to run.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You want a list of available commands.
cqperl cqtsadmin.pl --help

init_cq_fts

Summary: Copies the HCL Compass full-text search default template and then generates the default entity file. In effect, it runs both the copy_fts_template and gen_entity_file options. This command is one of the first commands that you run when you deploy full-text search. It creates a dbset_userdb directory in your ftshome directory. All default data and settings that are related to this deployment are placed in this directory. One of the key files that this option creates is the Entity-dbset-userdb.txt entity file. It is placed in your ftshome directory. When you customize your deployment, you decide whether to leave it as is or to remove record types and fields that you do not want to be searched. Another file that is created is the cqftsadmin-dbset-userdb.xml full-text search administrator configuration file. This file holds additional default settings that are specific for your deployment and environment such as the server name, your WebSphere Application Server profile name, the index batch size, and the index frequency.
Note: This command-line option is typically run in combination with the create_fts_was_profile and fts_was_profile_home command-line options so that a new WebSphere Application Server profile is created for this deployment. Otherwise, the HCL Compass full-text search deployment fails, unless you reuse a WebSphere Application Server profile that was previously deployed for full-text search.

Usage: Use this command to preconfigure your full-text search deployment. It creates your ftshome directory, copies the default files to it, and sets default values. Before you complete your deployment, you typically customize the entity file.

Effect: This command creates a new ftshome directory if it does not exist and copies the default data and settings for your deployment into it.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example

You want to start a new full-text search deployment for one of your user databases.

Run this command to preconfigure your deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --init_cq_fts --create_fts_was_profile Automatic --fts_was_profile_home D:\CQFTS.WASprofiles
Edit this deployment entity file to remove record types or fields that users should not be able to search. Also, add an ampersand (&) in front of a field for each record type. This will be the display field in the hit result set of full-text search.
D:\CQ.Search\TextSerch_SAMPL\Entity-TextSearch-SAMPL.txt
Complete your deployment by running this command:
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --setup_cq_fts

lock_cq_fts

Summary: This command-line option locks this HCL Compass full-text search deployment so that only non-destructive cqtsadmin.pl commands can be run. All commands, except gather_diagnostic_data and help, are disabled. Use the unlock_cq_fts option to enable the commands again.

Usage: After you deploy full-text search, issue this command to lock the deployment to prevent inadvertent modifications. This lock is weak. Anyone with correct file system access or HCL Compass privileges can unlock a deployment. The goal of this command is to enable administrators to signal that the deployment is complete. Further modification must be communicated and issued with care.

Effect: none. Your full-text search deployment data and settings are not affected.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You have completed a deployment and want to ensure that the deployment is not modified.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --lock_cq_fts

optimize_idx

Summary: This command-line option defragments the index. Optimization requires a minimum of 1.5 times as much free disk space as your current index size or a successful. For example, if your index uses 2 GB of disk space, you must have 3 GB of free disk space to run the command.

Usage

As you add or modify records in the search index, it can become fragmented. Fragmented indexes tend to grow larger than unfragmented ones. There also might be a slight performance degradation. To reduce index size and restore performance, optimize your index at least once a year. Optimize it more frequently if your HCL Compass database experiences heavy activity with record modifications, additions, or deletions.

Before you issue this command, ensure that you have enough free disk space. Otherwise the optimization will fail, but your original index will remain intact. Free disk space is required because the original index is rewritten during optimization. The old index is kept until the new index is regenerated.

The time required to optimize an index depends on the size of your index and the speed of your hard drive and I/O. Optimization can take a few hours on a 2 GB index. While the optimization is in progress, all full-text search services are available, including the update-mode indexer. However, there might be slight performance degradation. Plan index optimization for off-peak hours.

Effect: Your search index is rewritten. If an I/O error occurs during optimization, it is most likely because of insufficient disk space. The original index remains intact. The original index might be larger, but it will return to its original size after its optimization is complete.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You have to optimize your search index.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --optimize_idx

prep_upgd_was_profiles destination

Summary: During a scheduled downtime, this command-line option prepares for a HCL Compass modification or reinstallation. Use the backup that this command creates to restore the full-text search deployments including their WebSphere Application Server profile back to their original state.

Usage

All user databases that are deployed with full-text search require a corresponding WebSphere Application Server profile. These profiles might not be preserved when you upgrade, modify, or reinstall HCL Compass.

To delete the WebSphere Application Server profiles, you must change the <deleteFtsWASProfiles> tag value from FALSE to TRUE for one cqtsadmin-dbset-userdb.xml full-text search administrator configuration file. Ensure that you refer to this modified deployment when you run the command. Otherwise, the WebSphere Application Server profiles will not be correctly removed in preparation for their restoration and upgrade.

This command-line option can be run on any dbset, userdb, or ftshome. The option is not associated with a specific full-text search deployment. It evaluates and affects all full-text search deployments on the host in any ftshome directory.

Note: If you have more than one full-text search deployment on the current machine, this command-line option and the restore_was_profiles command-line option must be run only once. These two commands act globally across all of your full-text search deployment on your current machine.

Attention: When upgrading HCL Compass, you must run this command-line option with the <deleteFtsWASProfiles> tag set to TRUE. Otherwise, the result could be an incomplete upgrade of your full-text search deployments. After the upgrade is complete and you have run the restore_was_profiles command-line option, change this option to FALSE

Effect: If the <deleteFtsWASProfiles> tag is set to TRUE for one cqtsadmin-dbset-userdb.xml full-text search administrator configuration file, backup data of your HCL Compass full-text search WebSphere Application Server profiles is created, and these WebSphere Application Server profiles are deleted. Any full-text search request will fail until these profiles are restored. But, the radio button remains enabled, and creating oplogs continues for non-replicated Feature Level 7 user databases. This type of failure is acceptable because the only time that you use this command-line option is when you are upgrading, modifying, or reinstalling HCL Compass.

Stateful status: This option is not stateful. This command should never fail unless an I/O error occurs.

Example

You are about to upgrade, modify, or reinstall HCL Compass. Before you start the installation, you must back up all WebSphere Application Server profiles on this host.

Edit the cqtsadmin-dbset-userdb.xml file of a single full-text search deployment and change the <deleteFtsWASProfiles> XML tag as follows to permit WebSphere Application Server profiles to be backed up and deleted.
<newValue required="no">true</newValue>
Save the change. The cqtsadmin-TextSearch-SAMPL.xml file is used in this example.
Issue the command-line option to create backup data and delete the WebSphere Application Server profiles:
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --prep_upgd_was_profiles E:\FTSBackupWASProfiles
Change the <deleteFtsWASProfiles> XML tag value to FALSE or remove the TRUE value.
Complete the HCL Compass upgrade, full-text search feature modification or reinstallation.
Issue the command-line option to restore all full-text search deployments and deleted WebSphere Application Server profiles from the backup data. Two possible scenarios follows:
- For HCL Compass (all releases), preserve original WebSphere Application Server profile home locations, restore them to original unique locations:
  cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --restore_was_profiles E:\FTSBackupWASProfiles
- For HCL Compass (all releases), consolidate all WebSphere Application Server profiles into one WebSphere Application Server profile home directory when restored.
  cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --restore_was_profiles E:\FTSBackupWASProfiles --fts_was_profile_home D:\FTS.WASprofiles
If you encounter a problem, run the gather_diagnostic_data command-line option on the same dbset, userdb, ftshome to provide data to HCL customer support.

Attention: If you customized the Solr schema.xml file to support a language other than English, or if you made changes to the Solr analyzer or tokenizer, the changes will be lost. To preserve the changes, record your changes and introduce them again after you complete the upgrade. After you make the changes to the schema.xml file, stop and start the full-text search WebSphere Application Server profile for the changes to take effect.

remove_lucene_idx_lock

Summary: Use this command to remove a Lucene lock on the search index. The Lucene search engine uses locks to synchronize updates. In rare cases, if Lucene or the server encounters an error when a lock is obtained and Lucene cannot recover, the lock remains active. While a lock is active, the search index cannot be updated. Therefore, no HCL Compass records can be added or updated. Full-text searches on the index continue to work unless there is an integrity problem with the index. To recover from this type of lock, restart the WebSphere Application Server profile by issuing stop_fts_was_profile and start_fts_was_profile command-line options. Full-text searches are interrupted while the service restarts.

Usage: You notice that newly added HCL Compass records are not included in search results. You examine the WebSphere Application Server profile logs and discover that Lucene is reporting errors that the index is locked. Use this command to clear the lock.

Effect: none. The full-text search deployment data and settings are not affected.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You want to remove a Lucene index lock. Stop, start, and unlock the index in one step:
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --stop_fts_was_profile --remove_lucene_idx_lock --start_fts_was_profile

remove_record_type

Summary

This command removes one or more HCL Compass record types from the search index. Subsequent searches will not find matches to the record types that are removed. This command is used with data that you must to provide through the cqtsadmin-dbset-userdb.xml full-text search administrator configuration file. In the configuration file, list the names of record types to remove in the <removeRecordType> XML tag. To remove multiple record types, separate the names with a semicolon.

This command runs in states. If an error occurs during one of the states, an error message explains how to correct the error. When you restart the command, it continues from the point where the error occurred.

Create a backup of your deployment before you issue this command. Removing a record type removes only the data that is related to that record type from the search index. HCL Compass itself is not affected by this command.

Usage: After deploying full-text search, you are asked to no longer allow searches on certain record types. This command removes those record types from the index, which makes these record types unsearchable. Another use of this command is when record types that are indexed have changed in your schema. Perhaps you added new fields or renamed fields. To reflect this change in your search index, use the remove_record_type and add_record_type command-line options.

Effect: Your search index is altered so that any references to the record types that are removed no longer exist. Therefore, the record types that are removed are not searchable. References to these record types are removed from the full-text search properties XML file and entity file. While this command is running, search services are briefly interrupted when the WebSphere Application Server profile is restarted. Users might get an error that the server is down. Also, users who have a HCL Compass session open after the command is complete still see the removed record types in their Search Scope in HCL Compass Web. If they attempt to search these record types, there will be no matches. In order for their Search Scope to reflect the search index, these users must log in again. This command causes index fragmentation. Optimize the index after running this command so that both the index size and performance are optimal.

Stateful status: The command is stateful. If the command fails during one of its execution points, typically you can correct the issue and rerun the command. It will continue from where it stopped. If a failure occurs, an error message explains what to do.

Example

You must remove two record types from your search index.

In the cqtsadmin-TextSearch-SAMPL.xml file, specify the record types to remove in the <removeRecordType> XML tag. Separate the record types with a semicolon.
<newValue required="no">Email_Rule;Customer</newValue>
Run the command to remove the two record types.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --remove_record_type
In the cqtsadmin-TextSearch-SAMPL.xml file, remove the two record types that you added to the <removeRecordType> XML tag. This housekeeping task prevents accidental removal if you subsequently add these removed record types later.

repair_records

Summary: If a batch mode or update mode indexing process has issues that prevent reading records or sending them to the server for indexing, a repair file is created. The repair file lists the HCL Compass record IDs of the failed records. This command reads the repair file and reindexes one record at a time to reduce the chance of another failure.

Usage: As part of your full-text search deployment, run this command when batch indexing is finished to index records that have not been indexed. You should periodically check the ftshome directory for records that are not indexed during update-mode indexing. If records are not indexed, you see files with this naming convention record-type-nametime-stamp.xml. The following example shows an unindexed file:Defect1222923990646.xml. If you see these types of files, run this command to index the records.

Effect: Your search index includes data from the newly indexed records.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You have to index records that were not indexed during batch indexing or update indexing.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --repair_records

restore_was_profiles path-to-backup

Summary: Use this command-line option to restore your HCL Compass full-text search WebSphere Application Server profiles from backup data that was created with the prep_upgd_was_profiles command-line option.

Usage: WebSphere Application Server profiles might not be preserved when you upgrade, modify, or uninstall Compass. Full-text search fixes must be incorporated into current WebSphere Application Server profiles after you apply HCL Compass Fix Packs, all of which are handled by the prepare-restore procedure. After you upgrade, modify, or reinstall HCL Compass, this command-line option restores all HCL Compass full-text search WebSphere Application Server profiles that were created with the prep_upgd_was_profiles command-line option.
This command-line option fails if you attempt to restore a HCL Compass full-text search WebSphere Application Server profile that already exists or that has not been first backed up and deleted using the prep_upgd_was_profiles command-line option.
You can run this command-line option on any dbset, userdb or ftshome. It will affect all full-text search deployments and WebSphere Application Server profiles in any WebSphere Application Server profile home on this host.
Note: You can use this command-line option with the fts_was_profile_home command-line option to specify a consolidated new destination for the WebSphere Application Server profiles versus their original destination. Use this switch when you upgrade from HCL Compass (all releases) to ensure that a new WebSphere Application Server profile home is used.

Effect: Your full-text search WebSphere Application Server profiles are recreated and restored to their original settings and locations, or they are consolidated into the specified fts_was_profile_home directory path.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

run_batch_idx

Summary: This command starts the batch-mode indexer. It indexes all HCL Compass records that you configured for searching.
This command might fail if the batch size or number of threads is set too high, or if your HCL Compass records have a large amount of data. The most common failure is an out-of-memory error. In this case, either reduce the batch size or the number of threads in use. The result is an increase of the indexing time. Alternatively, you can increase Java™ virtual machine (JVM) memory for both the batch-mode indexer using the <batchIndexJVMParm> XML tag and the search serverrsquos memory using the <ftsWASProfileMaxHeapSize> XML tag. If you must increase memory, this is a temporary requirement until the batch indexer completes. HCL customer support can help you determine the best action in these circumstances. If the command stops before it completes, you can start it again. It will resume running the from where it stopped.

Usage: You do not have to use this command directly. It is called when you issue the setup_cq_fts command-line option. This command is provided in case you have to fine-tune or debug your deployment.

Effect: This command runs HCL Compass SQL queries against your HCL Compass database. Then it sends the records that were found to the Solr search server for indexing. In effect, while this command runs, your HCL Compass database server is busy sending data to HCL Compass full-text search, and your search index is updated. If you need to run this command, run it at off peak hours to reduce the load or effects on the performance of your database server.

Stateful status: This option is not stateful. This command should never fail unless an I/O error, an out-of-memory error or an unexpected configuration error occurs.

Example: You have to index all HCL Compass records.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --run_batch_idx

scrub_oplog before-date

Summary

This command is used to scrub oplogs from a non-replicated HCL Compass user database. Compass generates oplogs to keep track of changes that you make to records. HCL Compass full-text search monitors oplogs during update-mode indexing to synchronize the search index with these changes.

Oplogs are temporary data. You do not have to keep them indefinitely. To prevent continuous oplog growth on unreplicated databases, scrub old oplogs occasionally, but as infrequently as possible. Base scrubbing on the rate of oplog creation.

If your HCL Compass user database is replicated, use your replication tool and policy for oplog scrubbing. If you attempt to use this command on replicated databases, it fails with an error message that instructs you to use replication tools.

Never scrub all oplogs, especially if some are not yet processed by the update-mode indexer. Otherwise, your search index will not be synchronized with HCL Compass records. Searches might not be accurate or complete. This scenario requires batch-mode reindexing.

Aggressive oplog scrubbing where update-mode indexer throughput is insufficient might lead to index inaccuracy and missed hits. Ensure that the update-mode indexer is up-to-date, scrub only oplogs that are more than one month old, or do not scrub oplogs.

Usage: To conserve database space and to clean up unused data, you might have to scrub oplogs on a non-replicated user database. If your database is replicated, do not use this command.

Effect: HCL Compass oplogs in your oplog table that are created before the specified date are deleted.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: As a HCL Compass administrator, you plan to periodically scrub old oplogs. For information on date formats that are supported, see the scruboplog command reference.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --scrub_oplog "31-Oct-2009"

setup_cq_fts

Summary

This command is used to complete a full-text search deployment. First, the option customizes and configures the following files, based on your full-text search administrator configuration file, HCL Compass database, and operating system:

The entity file
The full-text search properties XML file
The Solr schema.xml file

Second, it enables HCL Compass full-text search in HCL Compass Web by enabling the Full Text radio button. Oplog generation begins if your HCL Compass database is Feature Level 7 and if it is not replicated. Third, it starts batch-mode indexing by indexing all HCL Compass records for the record types that you set for searching in the entity file. Finally, it enables update-mode indexing, which completes your deployment.

This command maintains its state. If an error occurs before it completes, the state is set and you must correct the error before you can continue. The error message and the logs contain instructions on how to recover from the error. The recovery steps depend on the error and the state in which the error occurred.

Usage: Use this command to complete your full-text search deployment. Typically, you run this command after you customize the entity file. While this command runs, the Full Text radio button is enabled in HCL Compass Web. Users who log in again are able to search. The search results are not complete until the deployment is complete. Particularly when the HCL Compass database is not replicated, the radio button must be enabled because oplogs must be generated to capture all record changes that occur during and after batch-mode indexing.
The computer on which you deploy full-text search undergoes high utilization because batch mode indexing relies heavily on the processor, I/O and memory, if you increase the default JVM memory setting. Depending on how aggressively you set your batch-mode indexer by increasing the batch size and number of threads, there could be high utilization of your HCL Compass database during batch-mode indexing.

Effect: Several files are created in your ftshome directory. The index is created. Search services under WebSphere Application Server are enabled. And, your database is updated to include the full-text search properties XML file. If your database is not replicated, creating oplogs is enabled. Replicated user databases continue to create oplogs.

Stateful status: The command is stateful. If the command fails during one of its execution points, you should be able to correct the issue and rerun the command. The command continues from where it left off. If a failure occurs, an error message tells you what to do. If an error occurs before it completes, the state is set and you must correct the error before you can continue. The error message and the logs contain instructions on how to recover from the error. The recovery steps depend on the error and the state in which the error occurred.

Example

You have run the init_cq_fts command-line option and edited your entity file. You now have to complete the deployment.

cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --setup_cq_fts

set_was_max_mem

Summary

This command is used to set the maximum JVM memory that the WebSphere Application Server profile can use. The default value is 300 MB, which might be too small for batch-mode indexing in these situations:

The batch size has been increased
The number of threads has increased
The record types are complex

This command reads the memory setting from the <ftsWASProfileMaxHeapSize> XML tag in your configuration file. Then it sets the JVM memory to this value.

Usage

If batch-mode indexing fails because the WebSphere Application Server profile server reports out-of-memory error, the memory setting is probably too low. Take one of the following actions to recover from an out-of-memory error:

Reduce the batch size and number of threads to increases the time it takes to complete indexing.
Temporarily increase the JVM memory. The JVM maximum memory typically should be set higher until batch-mode indexing completes.

Effect: The JVM maximum memory setting for the WebSphere Application Server profile changes to the new value. In effect, more system memory is allocated to the WebSphere Application Server profile

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example

You receive out-of-memory errors from the WebSphere Application Server profile while indexing. You must address this issue before you can continue.

Set the JVM memory to 1.5 GB in the <ftsWASProfileMaxHeapSize> XML tag in the cqtsadmin-TextSearch-SAMPL.xml file.
<newValue required="no">1536</newValue>
Run this command to set the new JVM memory setting:
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --set_was_max_mem

set_solr_home

Summary: This command is used to set the Solr home directory under WebSphere Application Server. Each deployment has its own schema.xml configuration file and index. If the Solr home directory is not specified correctly or if the wrong location is specified, the WebSphere Application Server full-text search profile might not start. If the WebSphere Application Server full-text search profile does not start, errors are logged to the corresponding full-text search WebSphere Application Server profile logs directory in the %your-WAS-profile-home%/<cqfts>_<dbset>_<userdb>/logs/server1/logs/ path.

Usage: You do not have to use this command directly. It is called when you issue the setup_cq_fts command-line option. This command is provided in case you have to fine-tune or debug your deployment.

Effect: The JVM property of the deployed WebSphere Application Server profile is changed so that the Solr home environment variable is set.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You have been instructed by HCL customer support to use this command to debug a full-text search deployment issue, or to customize a deployment.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --set_solr_home

show_scenarios [ID | all]

Summary: This command is used to display a list of scenarios with examples of how to use the cqtsadmin.pl tool. The scenarios are an abbreviated form of the scenarios that are listed in the information center and might not be the complete list or match one-to-one.

Usage: Use this command-line option to display a list of the most commonly used scenarios for the cqtsadmin.pl tool. Issue the command without a parameter to see the scenario IDs and headlines. Issue the command with a scenario ID parameter to see the full text of a scenario. Issue the command with the all parameters to see a complete list of scenarios, each with an ID, headline, and a full text description.

Effect: The full-text search deployment data and settings are not affected.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You are about to use the cqtsadmn.pl tool to complete a task, but you cannot remember how to use it. Use this command-line option to list the scenario headlines with their IDs. Then run this command again and pass the ID of the scenario that you are interested in to see the full text of it.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --show_scenarios

start_fts_was_profile

Summary: This command is used to start the full-text search WebSphere Application Server profile service. A full-text search WebSphere Application Server profile must be started for search requests to be serviced. A profile must be started also for the update-mode indexer to start checking for new or updated records and send them to the search engine for indexing.

Usage: You do not have to use this command directly. It is called when you issue the setup_cq_fts command-line option. This command is provided in case you have to fine-tune or debug your deployment.

Effect: If the full-text search WebSphere Application Server profile was stopped, it will start, causing temporary memory and processor resource utilization. If search services and the update-mode indexer are enabled, they start running. If the WebSphere Application Server profile is already started, no change is made.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error, a Solr server failure, or a WebSphere Application Server profile startup failure, which is typically because of setup issues.

Example: HCL customer support instructed you to start the search WebSphere Application Server profile.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --start_fts_was_profile

start_update_idx

Summary: This command is used to enable and start the update-mode indexer, which runs under the WebSphere Application Server profile. The update-mode indexer synchronizes the search index with changes that are made to the database. The indexer monitors oplogs for new values to be indexed. You configure how often the index is synchronized by using the <updateIndexDelay> XML tag in your full-text search administrator configuration file.

Usage: You do not have to use this command directly. It is called when you issue the setup_cq_fts command-line option. This command is provided in case you have to fine-tune or debug your deployment.

Effect: The update-mode indexer is enabled and started. Modifications, additions, and removal of HCL Compass records are now indexed and appear in search results.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error, a Solr server failure, or a WebSphere Application Server profile startup failure, which is typically because of setup issues.

Example: HCL customer support instructed you to enable the update indexer.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --start_update_idx

stop_fts_was_profile

Summary: This command is used to stop the full-text search WebSphere Application Server profile service. When a WebSphere Application Server profile stops, then search services are unavailable, and the update-mode indexer stops synchronizing the search index with the changes that were made to your records.

Usage: You do not have to use this command directly. It is called when you issue the setup_cq_fts command-line option. This command is provided in case you have to fine-tune or debug your deployment.

Effect: The full-text search WebSphere Application Server profile stops, which also stops the search services and update indexing. Memory and processor power that the WebSphere Application Server profile uses are released. A search causes an error.
The oplog continues to receive messages as long as the Full Text radio button remains enabled in HCL Compass Web. This enables the update-mode indexer to catch up after the WebSphere Application Server profile is running again.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support instructed you to stop the search WebSphere Application Server profile.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --stop_fts_was_profile

stop_update_idx

Summary: This command is used to disable and stop the update-mode indexer. When the update-mode indexer stops, search services are available, but changes that are made to HCL Compass records are not reflected in the index until update-mode indexing is reenabled or resumed. Therefore, searches might yield results that are not up-to-date or accurate.

Usage: You do not have to use this command directly. It is called when you issue the setup_cq_fts and backup_fts command-line options. This command is provided in case you have to fine-tune or debug your deployment.

Effect: The update-mode indexer, which runs under the WebSphere Application Server profile, is disabled. New and modified records are not reflected in search results. The oplog continues to receive messages. HCL Compass record changes are reflected in full-text searches after the update-mode record indexer is reenabled and after it catches up with the current oplogs.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: HCL customer support instructed you to disable the update-mode indexer.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --stop_update_idx

unlock_cq_fts

Summary: This command is used to unlock a locked deployment of full-text search. After the command finishes, you can run all available full-text search Administrator commands.

Usage: Use this command to unlock a locked deployment of full-text search so that you can run all available commands. This command reverses the lock that the lock_cq_fts command-line option sets.

Effect: none. Your full-text search deployment data and settings are not affected, and all commands can be run.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You have to optimize your index, but your deployment is locked. You must unlock it, optimize the index, and then lock it again.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --unlock_cq_fts --optimize_idx --lock_cq_fts

update_fts_prop_files

Summary

Propagates changes that you make in the cqtsadmin-dbset-userdb.xml full-text search administrator configuration file. After you change any of the following XML tags in the configuration file, you must propagate the changes for them to take effect:

<batchIndexBatchSize>
<batchIndexDelay>
<batchIndexThreads>
<ftsServerName>
<ftsWASProfileName>
<ftsWASProfilePort>
<updateIndexBatchSize>
<updateIndexDelay>
<updateIndexLoginInterval>

For example, if you change the batch size for the batch-mode indexer, you must issue this command before running the run_batch_idx command-line option for the new value to be used.

Note: Whenever you change the password for the admin user, use update_fts_prop_files (specifying the new password) to ensure that the password file (pwd.txt) is updated and batch indexing of new records can resume; for example

cqperl cqtsadmin.pl --username admin --password new_password --dbset 7.0.0 --userdb SAMPL --ftshome C:\CQ.Search --fresh_batch_idx

Usage: When you deploy full-text search, you might have to change a default setting. When a change is made to the configuration file, the change must be propagated to the appropriate full-text search components.

Effect: The affected component depends on which XML tag value you change. See cqtsadmin-dbset-userdb.xml for more information.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error.

Example: You have to accelerate batch-mode indexing by increasing the batch size and number of threads. After you change the configuration file, issue this command.
cqperl cqtsadmin.pl --username admin --password "" --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --update_fts_prop_files --run_batch_idx

upgrade_solr_app

Summary: This command-line option checks whether Apache Solr of your current full-text search deployment is the latest version that is shipped with your version of HCL Compass. If not, the option upgrades Apache Solr.

Usage: A newer version of HCL Compass might support newer versions of Apache Solr than the version your current full-text search deployment uses. When you upgrade HCL Compass and your full-text search deployments that use the prep_upgd_was_profiles and restore_was_profiles command-line options, your Apache Solr application is not automatically upgraded. This upgrade approach is intentional, because the index format might have changed, which would require a full reindexing. Use this command to upgrade Apache Solr to the latest supported version.
Before you use this command-line option, you should back up your full-text search deployment by using the backup_fts command-line option. In addition, see the HCL Compass release notes to learn whether you have to reindex your HCL Compass database after you upgrade Apache Solr. You might have to reindex if the index format of Apache Solr changed.
After you run this command-line option, you must run the stop_fts_was_profile command-line option and then run the start_fts_was_profile command-line option for the upgrade to take effect.
Important: You must run this command-line option after you upgrade HCL Compass and after you upgrade your full-text search deployments by using the prep_upgd_was_profiles and restore_was_profiles command-line options. Otherwise, unknown issues might occur.

Effect: Your existing Apache Solr is upgraded to the latest version that is supported. HCL Compass full-text search is unavailable until the upgrade is complete.

Stateful status: This option is not stateful. This command should never fail unless there is an I/O error, or WebSphere Application Server issues.

Example

You want to upgrade Apache Solr for one of your full-text search deployments. Back up your full-text search deployment by running the cqtsadmin.pl command with the backup_fts option.

cqperl cqtsadmin.pl --username admin --password secret --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --backup_fts E:\FTS.Backup

Upgrade the Solr application by running the cqtsadmin.pl command again with the upgrade_solr_app command-line option:

cqperl cqtsadmin.pl --username admin --password secret --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --upgrade_solr_app

If a message displays that instructs you to re-index the user database, then run the cqtsadmin.pl command four times with the stop_update_idx, run_batch_idx, optimize_idx, and start_update_idx command-line options:

cqperl cqtsadmin.pl --username admin --password secret --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --stop_update_idx

cqperl cqtsadmin.pl --username admin --password secret --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --run_batch_idx

cqperl cqtsadmin.pl --username admin --password secret --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --optimize_idx

cqperl cqtsadmin.pl --username admin --password secret --dbset TextSearch --userdb SAMPL --ftshome D:\CQ.Search --start_update_idx

If you have additional full-text search deployments for other HCL Compass databases, run the previous commands on each database.

was_profile_ports_file

Summary: This command-line option enables you to optionally provide your own WebSphere Application Server profile ports file, which is used with the create_fts_was_profile command-line option when a full-text search profile is created.

Usage: When you create a new full-text search deployment, you create a new WebSphere Application Server profile by default. The full-text search administrator tool determines the next available port to use on your system. If you have to tune your deployment and specify your own ports to use, you must create your own ports file. Then, you use the was_profile_ports_file option to tell the full-text search administrator tool to use your ports file. There is a template that you can use in the portdef.props file in your WebSphere installation. The portdef.props file is typically located in the \IBM\WebSphere\AppServer\profileTemplates\default\actions\portsUpdate\ directory. To learn more about WebSphere Application Server profile ports file, see manageprofiles command.
This command-line option is used with create_fts_was_profile option.
Note: If you use your own ports, ensure that your ports are not used by other applications. Otherwise, you might encounter problems with full-text search and other applications.

Effect: The ports that you specify with this command are used instead of the ports specified by the full-text search administrator tool.

Stateful status: This option is not stateful. This command should never fail unless an I/O error occurs.

Example

You have to specify your own set of ports to use.

Make a copy of the portdef.props file, and then specify your own port value.
Run this command to create a new full-text search deployment that uses the was_profile_ports_file option to specify your own ports.
cqperl cqtsadmin.pl --username admin --password "" --dbset MASTR --userdb SAMPL --ftshome D:\CQ.Search --init_cq_fts --create_fts_was_profile automatic --fts_was_profile_home D:\FTS.WASprofiles --was_profile_ports_file D:\my-portdef.props
Continue to use your full-text search deployment.