Evaluating codepageutil output data

When you run the codepageutil analyze_tables command, the code page utility produces a report file that specifies which characters must be converted and where they occur in the database. This file can be produced either in a text file format easier for reading, or in XML file format which can be parsed and understood by another program.

No conversion needed

If no characters need to be converted, the code page utility displays a message listing the tables scanned, the number of records checked in each table, and the following confirmation:
All scanned data is valid for code page 932.

Note that you might see this message even if the current HCL Compass data code page does not match the target data code page. For example, even if the data code page is 1252 (Latin-1), it is possible that only ASCII characters (and no accented European Latin-1 characters) are stored in that database. In this situation, no conversion is required before changing the HCL Compass data code page to 932 (Japanese).

Conversion required

If any characters in the current database are not supported by the specified data code page, analyze_tables displays the following data:

  • The tables scanned and the number of records checked in each table.
  • The number of rows in each table that have a character that is not supported by the specified data code page.
  • A list of unsupported characters stored in the database, with the number of occurrences of each.
  • A list of the columns in each table in the database that contain an unsupported character, with the number of these characters in each listed column.
  • A list of HCL Compass objects (such as fields, scripts, and states) that contain unsupported characters, with the number of unsupported characters in each column.
  • For stateful tables, a breakdown of the number of records, by state, that contain unsupported characters. Stateful tables are tables whose records have a designate state, such as Assigned or Closed. Example records include Defect and Enhancement Request records.
  • The path and name of the file that contains the output report.
  • The ID of the record associated with the row, if any.

Information about the state of the record can be very useful during the conversion process. For example, you might choose to not convert defect records that have a Resolved or Closed state, because it is less likely that these records will be accessed in the future.

The analysis files for schema repositories and user databases are generated in the same way, but there are some differences. The output for a schema repository contains schema names and revisions for row information.

The report output for a user database contains no schema names, but contains other information. The state, such as Submitted, is included for Defects and other stateful records. The output contains HCL Compass entity names in addition to the database names for tables and fields. For columns, the user database report output provides both the database name (column HEADLINE) and the HCL Compass name (Defect.Headline). In addition, for stateful records, the report lists both the internal database ID (such as dbid=33554499) and the HCL Compass record name (such as SAMPL00000067).

The schema ID and revision number are shown only for metadata that derive from the schema repository. These are changed in the schema repository and propagated to the user database as part of a schema upgrade. The user database contains records only from the single schema and schema revision that the database currently uses.

Reports in XML format

The ability to produce a report output file in either text or XML format is very important. The information in text format can be read and used to make decisions and convert characters. However, if you want to write a program to automate the conversion task, a program can more easily parse and understand the XML format file. Although HCL does not provide an XML parser, many commercial and open-source XML parsers are available.